schrht / virt-perf-scripts Goto Github PK

$ GenerateBenchmarkReport.py rhel74_fio_report.csv rhel75_830_fio_report.csv benchmark_report.csv
Reading base samples from csv file "rhel74_fio_report.csv"...
Reading test samples from csv file "rhel75_830_fio_report.csv"...
Traceback (most recent call last):
File "/home/cheshi/bin/GenerateBenchmarkReport.py", line 364, in
generate_fio_benchmark_report()
File "/usr/lib/python2.7/site-packages/click/core.py", line 722, in call
return self.main(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/lib/python2.7/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python2.7/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/home/cheshi/bin/GenerateBenchmarkReport.py", line 353, in generate_fio_benchmark_report
fbr.generate_report()
File "/home/cheshi/bin/GenerateBenchmarkReport.py", line 297, in generate_report
self._complete_report_dataframe()
File "/home/cheshi/bin/GenerateBenchmarkReport.py", line 264, in _complete_report_dataframe
series, my_sub_base, my_sub_test, 'Util', 'Util(%)', True)
File "/home/cheshi/bin/GenerateBenchmarkReport.py", line 210, in _calculate_and_fill_report_series
series[label + '-TEST-AVG'] = df_test[source_label].mean()
File "/usr/lib64/python2.7/site-packages/pandas/core/generic.py", line 7315, in stat_func
numeric_only=numeric_only)
File "/usr/lib64/python2.7/site-packages/pandas/core/series.py", line 2577, in _reduce
return op(delegate, skipna=skipna, **kwds)
File "/usr/lib64/python2.7/site-packages/pandas/core/nanops.py", line 77, in _f
return f(*args, **kwargs)
File "/usr/lib64/python2.7/site-packages/pandas/core/nanops.py", line 131, in f
result = alt(values, axis=axis, skipna=skipna, **kwds)
File "/usr/lib64/python2.7/site-packages/pandas/core/nanops.py", line 356, in nanmean
the_sum = _ensure_numeric(values.sum(axis, dtype=dtype_sum))
File "/usr/lib64/python2.7/site-packages/pandas/core/nanops.py", line 825, in _ensure_numeric
.format(value=x))
TypeError: Could not convert 99.91494299.93327999.91327899.93828299.911606 to numeric
$

when run ./GenerateTestReport.py --result_path $path
there is error [ERROR] Error while extracting performance KPIs: 'jobs'

Now the scripts not support python3, please update to support python3.
Thanks very much!

Could you help to change size to parameter for fio test?

Currently, need to use file name to track compare kernel result, if can add base and test kernel version in the result, will be better, thank you.

Hi shichen,
when I try to run on RHEL by default,there is a error can not found yaml file, the yaml file is on upper folder.
could you help define the path?

./RunFioTest.py
[WARNING] Fail to get default value from yaml file. [Errno 2] No such file or directory: './virt_perf_scripts.yaml'

if we just run one round fio test, then run the generate test report script, there is error message.

[root@bootp-73-199-100 tmp]# ./GenerateTestReport.py 
  File "./GenerateTestReport.py", line 49
    for key, value in inputs.iteritems()
      ^
SyntaxError: invalid syntax
[root@bootp-73-199-100 tmp]# 

please check the reason.
Thanks
ldu

Error while extracting performance KPIs: 'disk_util'

[cheshi@cheshi-laptop2 ESXi]$ GenerateTestReport.py fio_result.rhel75 rhel75.csv 
Error while extracting performance KPIs: 'disk_util'
[cheshi@cheshi-laptop2 ESXi]$ 

$ GenerateTestReport.py ./rhel75_830 rhel75_830_fio_report.csv
Error while parsing disk_util: length != 1
Error while parsing disk_util: length != 1
Error while parsing disk_util: length != 1
Error while parsing disk_util: length != 1
Error while parsing disk_util: length != 1
......
Error while parsing disk_util: length != 1
Error while parsing disk_util: length != 1
Dumping data into csv file "rhel75_830_fio_report.csv"...
Finished!
$

Following list some suggestion about documents:

1. fio document added suggestion:

how many cpu needed? memory needed for seting up vm

2. Default parameter explanation, e.g runtime 1 minute, size =512M

3. Typical scenario 
不同场景的测试公式基本一致，只有3个参数（读写模式，iodepth，blocksize）的区别。下面举例说明使用block size为4k，iodepth为1来测试顺序读性能的命令。
FIO： https://blog.csdn.net/qq_36357820/article/details/79965014
block=4k iodepth=1 随机读测试，能反映磁盘的时延性能；
block=128K iodepth=32 能反映峰值吞吐性能 ;
block=4k iodepth=32 能反映峰值IOPS性能

4. Some parameter suggestions?
   read
   Sequential reads.
   write
   Sequential writes.
   rw,readwrite
   Sequential mixed reads and writes.

   randread
   Random reads.
   randwrite
   Random writes.
   read, write, rw -- numberjobs better 1
   randread, readwrite, rw, -- numberjobs better 16.

[root@bootp-73-199-203 block]# ./RunFioTest.py --backend NVME --driver SCSI --fs RAW --filename /dev/sdb --log_path /root--rounds 1
Usage: RunFioTest.py [OPTIONS]

Error: Got unexpected extra argument (1)