VerneMQ server generates a lot of errors and no results when testing fan_in.bdl about vmq_mzbench

[ioolkos]

Hi @syberkitten, this is not VerneMQ crashing, but the test environment. It runs into some sort of network troubles indicated by EINVAL errors, which can probably be caused by a couple of things.
Is your 'localhost' resolving ok?

[syberkitten]

Hello @ioolkos Yes, my localhost is ok.
I'm also getting now more errors from mzbench itself, even when testing a remote server

23:50:25.992 [error] [[email protected]] <0.266.0> Worker <0.467.0> on '[email protected]' has crashed: undef
Stacktrace: [{crypto,rand_bytes,[150],[]},{mzbl_stdlib,random_binary,4,[{file,"/private/tmp/bench_mzbench_api_Liams-MacBook-Pro_1500_578967_295800/deployment_code/node/_build/default/deps/mzbench_language/src/mzbl_stdlib.erl"},{line,79}]},{mzbl_interpreter,'-eval/4-fun-0-',4,[{file,"/private/tmp/bench_mzbench_api_Liams-MacBook-Pro_1500_578967_295800/deployment_code/node/_build/default/deps/mzbench_language/src/mzbl_interpreter.erl"},{line,19}]},{lists,foldl,3,[{file,"lists.erl"},{line,1263}]},{mzbl_interpreter,eval_function,6,[{file,"/private/tmp/bench_mzbench_api_Liams-MacBook-Pro_1500_578967_295800/deployment_code/node/_build/default/deps/mzbench_language/src/mzbl_interpreter.erl"},{line,33}]},{mzbl_interpreter,'-eval/4-fun-0-',4,[{file,"/private/tmp/bench_mzbench_api_Liams-MacBook-Pro_1500_578967_295800/deployment_code/node/_build/default/deps/mzbench_language/src/mzbl_interpreter.erl"},{line,19}]},{lists,foldl,3,[{file,"lists.erl"},{line,1263}]},{mzbl_loop,k_times,5,[{file,"/private/tmp/bench_mzbench_api_Liams-MacBook-Pro_1500_578967_295800/deployment_code/node/_build/default/deps/mzbench_language/src/mzbl_loop.erl"},{line,316}]}]

00:09:13.354 [error] [[email protected]] <0.266.0> Worker <0.1455.0> on '[email protected]' has crashed: undef Stacktrace: [{crypto,rand_bytes,[150],[]},{mzbl_stdlib,random_binary,4,[{file,"/private/tmp/bench_mzbench_api_Liams-MacBook-Pro_1500_578967_295800/deployment_code/node/_build/default/deps/mzbench_language/src/mzbl_stdlib.erl"},{line,79}]},{mzbl_interpreter,'-eval/4-fun-0-',4,[{file,"/private/tmp/bench_mzbench_api_Liams-MacBook-Pro_1500_578967_295800/deployment_code/node/_build/default/deps/mzbench_language/src/mzbl_interpreter.erl"},{line,19}]},{lists,foldl,3,[{file,"lists.erl"},{line,1263}]},{mzbl_interpreter,eval_function,6,[{file,"/private/tmp/bench_mzbench_api_Liams-MacBook-Pro_1500_578967_295800/deployment_code/node/_build/default/deps/mzbench_language/src/mzbl_interpreter.erl"},{line,33}]},{mzbl_interpreter,'-eval/4-fun-0-',4,[{file,"/private/tmp/bench_mzbench_api_Liams-MacBook-Pro_1500_578967_295800/deployment_code/node/_build/default/deps/mzbench_language/src/mzbl_interpreter.erl"},{line,19}]},{lists,foldl,3,[{file,"lists.erl"},{line,1263}]},{mzbl_loop,k_times,5,[{file,"/private/tmp/bench_mzbench_api_Liams-MacBook-Pro_1500_578967_295800/deployment_code/node/_build/default/deps/mzbench_language/src/mzbl_loop.erl"},{line,316}]}]
--
00:09:13.354 [error] [[email protected]] <0.266.0> Worker <0.449.0> on '[email protected]' has crashed: undef Stacktrace: [{crypto,rand_bytes,[150],[]},{mzbl_stdlib,random_binary,4,[{file,"/private/tmp/bench_mzbench_api_Liams-MacBook-Pro_1500_578967_295800/deployment_code/node/_build/default/deps/mzbench_language/src/mzbl_stdlib.erl"},{line,79}]},{mzbl_interpreter,'-eval/4-fun-0-',4,[{file,"/private/tmp/bench_mzbench_api_Liams-MacBook-Pro_1500_578967_295800/deployment_code/node/_build/default/deps/mzbench_language/src/mzbl_interpreter.erl"},{line,19}]},{lists,foldl,3,[{file,"lists.erl"},{line,1263}]},{mzbl_interpreter,eval_function,6,[{file,"/private/tmp/bench_mzbench_api_Liams-MacBook-Pro_1500_578967_295800/deployment_code/node/_build/default/deps/mzbench_language/src/mzbl_interpreter.erl"},{line,33}]},{mzbl_interpreter,'-eval/4-fun-0-',4,[{file,"/private/tmp/bench_mzbench_api_Liams-MacBook-Pro_1500_578967_295800/deployment_code/node/_build/default/deps/mzbench_language/src/mzbl_interpreter.erl"},{line,19}]},{lists,foldl,3,[{file,"lists.erl"},{line,1263}]},{mzbl_loop,k_times,5,[{file,"/private/tmp/bench_mzbench_api_Liams-MacBook-Pro_1500_578967_295800/deployment_code/node/_build/default/deps/mzbench_language/src/mzbl_loop.erl"},{line,316}]}]

Happened both in Erlang/OTP 20/19
Could this be related to an open files limit of 10000 ?
I will try to reduce the number of workers and test again.

[ioolkos]

crypto:rand_bytes/1 was removed in OTP 20, in favour of crypto:strong_rand_bytes/1. That immediately makes MZBench (the underlying Machinezone MZBench framework) incompatible. I guess I always used it on OTP 18.

And yes, 10k file descriptors is low, can you configure this for a higher value or run a lower scenario?

Many thanks for your ongoing effort & learnings for OS X!

[syberkitten]

@ioolkos
thanks for the feedback, I've switched to Erlang/OTP 18.3 using Kerl,
and the benchmark is working. 👍

[ioolkos]

Great!
One additional thing to note is that vmq_mzbench is asynchronous, that's why sometimes you have to play around with wait times, especially for subscribers.

[ioolkos]

I guess I can close this issue.