incubator-doris [Feature] Support for storage layer benchmark

rdrgkggo  于 2022-04-22  发布在  Java
关注(0)|答案(0)|浏览(140)

Is your feature request related to a problem? Please describe.

I have developed a Benchmark tool that is used to test some read and write performance tests related to the storage layer.

Describe the solution you'd like

It is currently testing segment and partial Page.
Dataset can be randomly generated or imported by data files.
It also supports testing functions written by users, used to get some effects of some third-party libraries at doris.

Describe alternatives you've considered

It will be compiled when the user executes ./run-be-ut.sh and placed in ut_build_ASAN/test/tools/benchmark_tool.
But it doesn't be executed when Run All Ut (- Run).

Additional context

Here are some usage examples:

./benchmark_tool --operation=Custom
Execute the benchmark of the custum function (need to implement the init() and run() functions)

./benchmark_tool --operation=BinaryDictPageEncode --rows_number=10000 --iterations=40

Execute the benchmark of BinaryDictPage encode 1000 randomly generated data,Iterate forty times to calculate the average time consumption.

./benchmark_tool --operation=SegmentScan --column_type=int,varchar --rows_number=10000 --iterations=0

Execute the benchmark of Segment scan data from file(data is randomly generated), iterations is set to 0 means the number of iterations is automatically set.

./benchmark_tool --operation=SegmentWrite --column_type=int --rows_number=10000 --iterations=10

Execute the benchmark of Segment write data to file(data is randomly generated).

./benchmark_tool --operation=SegmentScanByFile --input_file=./sample.dat --iterations=10

Execute the benchmark of Segment scan data from file(data from input files).

Data file format:
The first line defines Shcema
The rest of the content is DataSet

For example:

int,char,varchar
123,hello,world
321,good,bye

./benchmark_tool --operation=SegmentWriteByFile --input_file=./sample.dat --iterations=10

Execute the benchmark of Segment write data from file(data from input files).

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题