Hi,
I have a strange bug, and don't know if it's in uvw, libuv or my own code. I will explain here and maybe someone has any hints.
I have an Idle handle that start a process at some point. It works and at some random point in time my my code crashes with an assert exactly when the process is launched:
[INF] lua (ScriptExec.cpp:39) Starting script. (0xd7b480)
[INF] process (ExternProc.cpp:191) Starting process: /usr/bin/calaos_script --socket /tmp/calaos_proc_c79ad522-33ff-978a-3d31-08070a51c459_lua_21499 --namespace lua
calaos_server: ../libuv-1.19.1/src/unix/stream.c:1574: uv_read_start: Assertion `uv__stream_fd(stream) >= 0' failed.
What my code does is basically easy:
- it create a uvw::ProcessHandle
- setup uvw::ExitEvent and uvw::ErrorEvent for the process handle
- then create a uvw::PipeHandle for communicating with the process
- setup uvw::EndEvent and uvw::DataEvent for the pipe
- spawn the process
- start pipe->read() (aka. uv_read_start())
Full code of the function is here:
https://github.com/calaos/calaos_base/blob/master/src/bin/calaos_server/IO/ExternProc.cpp#L147
It works. Sometime it works for a long time (hours). And then it crashes with this backtrace:
Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at /usr/src/debug/glibc/2.25-r0/git/sysdeps/unix/sysv/linux/raise.c:51
51 }
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at /usr/src/debug/glibc/2.25-r0/git/sysdeps/unix/sysv/linux/raise.c:51
#1 0x000000310e23450a in __GI_abort () at /usr/src/debug/glibc/2.25-r0/git/stdlib/abort.c:89
#2 0x000000310e22bfe7 in __assert_fail_base (fmt=<optimized out>, assertion=assertion@entry=0x7ffff7deaca6 "uv__stream_fd(stream) >= 0",
file=file@entry=0x7ffff7dea998 "../libuv-1.19.1/src/unix/stream.c", line=line@entry=1574, function=function@entry=0x7ffff7deae48 <__PRETTY_FUNCTION__.9160> "uv_read_start")
at /usr/src/debug/glibc/2.25-r0/git/assert/assert.c:92
#3 0x000000310e22c092 in __GI___assert_fail (assertion=assertion@entry=0x7ffff7deaca6 "uv__stream_fd(stream) >= 0",
file=file@entry=0x7ffff7dea998 "../libuv-1.19.1/src/unix/stream.c", line=line@entry=1574, function=function@entry=0x7ffff7deae48 <__PRETTY_FUNCTION__.9160> "uv_read_start")
at /usr/src/debug/glibc/2.25-r0/git/assert/assert.c:101
#4 0x00007ffff7de316d in uv_read_start (stream=stream@entry=0xd8b418,
alloc_cb=alloc_cb@entry=0x48ffe0 <uvw::Handle<uvw::PipeHandle, uv_pipe_s>::allocCallback(uv_handle_s*, unsigned long, uv_buf_t*)>,
read_cb=read_cb@entry=0x498320 <uvw::StreamHandle<uvw::PipeHandle, uv_pipe_s>::readCallback(uv_stream_s*, long, uv_buf_t const*)>)
at /usr/src/debug/libuv/1.19.1-r0/libuv-1.19.1/src/unix/stream.c:1574
#5 0x0000000000497bf9 in uvw::Handle<uvw::PipeHandle, uv_pipe_s>::invoke<int (*)(uv_stream_s*, void (*)(uv_handle_s*, unsigned long, uv_buf_t*), void (*)(uv_stream_s*, long, uv_buf_t const*)), uv_stream_s*, void (*)(uv_handle_s*, unsigned long, uv_buf_t*), void (*)(uv_stream_s*, long, uv_buf_t const*)>(int (*&&)(uv_stream_s*, void (*)(uv_handle_s*, unsigned long, uv_buf_t*), void (*)(uv_stream_s*, long, uv_buf_t const*)), uv_stream_s*&&, void (*&&)(uv_handle_s*, unsigned long, uv_buf_t*), void (*&&)(uv_stream_s*, long, uv_buf_t const*)) (f=<optimized out>, this=0xd8b3b0) at /usr/src/debug/calaos-server/1_3.0-rc1+r0+gitAUTOINC+19f8b098fd-r0/git/src/lib/uvw/src/uvw/handle.hpp:62
#6 uvw::StreamHandle<uvw::PipeHandle, uv_pipe_s>::read (this=0xd8b3b0)
at /usr/src/debug/calaos-server/1_3.0-rc1+r0+gitAUTOINC+19f8b098fd-r0/git/src/lib/uvw/src/uvw/stream.hpp:228
#7 ExternProcServer::startProcess (this=0xd7b480, process=..., name=..., args=...)
at /usr/src/debug/calaos-server/1_3.0-rc1+r0+gitAUTOINC+19f8b098fd-r0/git/src/bin/calaos_server/IO/ExternProc.cpp:193
#8 0x000000000062b8f3 in ScriptExec::ExecuteScriptDetached(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<void (bool)>, Params) (script=..., cb=..., env=...) at /usr/src/debug/calaos-server/1_3.0-rc1+r0+gitAUTOINC+19f8b098fd-r0/git/src/bin/calaos_server/LuaScript/ScriptExec.cpp:150
#9 0x000000000063c021 in Calaos::ActionScript::Execute (this=<optimized out>)
at /usr/src/debug/calaos-server/1_3.0-rc1+r0+gitAUTOINC+19f8b098fd-r0/git/src/bin/calaos_server/Rules/ActionScript.cpp:39
#10 0x00000000006358f1 in Calaos::Rule::ExecuteActions (this=0xc441f0)
at /usr/src/debug/calaos-server/1_3.0-rc1+r0+gitAUTOINC+19f8b098fd-r0/git/src/bin/calaos_server/Rule.cpp:148
#11 0x0000000000626eae in Calaos::ListeRule::ExecuteRuleSignal (this=this@entry=0x977980 <Calaos::ListeRule::Instance()::inst>, id=...)
at /usr/src/debug/calaos-server/1_3.0-rc1+r0+gitAUTOINC+19f8b098fd-r0/git/src/bin/calaos_server/ListeRule.cpp:182
#12 0x00000000006241ce in Calaos::ListeRule::<lambda()>::operator() (__closure=0xd8b660)
at /usr/src/debug/calaos-server/1_3.0-rc1+r0+gitAUTOINC+19f8b098fd-r0/git/src/bin/calaos_server/ListeRule.cpp:101
#13 sigc::adaptor_functor<Calaos::ListeRule::ExecuteRuleSignal(std::__cxx11::string)::<lambda()> >::operator() (this=0xd8b660)
at /usr/include/sigc++-2.0/sigc++/adaptors/adaptor_trait.h:256
#14 sigc::internal::slot_call0<Calaos::ListeRule::ExecuteRuleSignal(std::__cxx11::string)::<lambda()>, void>::call_it(sigc::internal::slot_rep *) (rep=0xd8b630)
at /usr/include/sigc++-2.0/sigc++/functors/slot.h:114
#15 0x0000000000670602 in sigc::slot0<void>::operator() (this=0xd8b608) at /usr/include/sigc++-2.0/sigc++/functors/slot.h:513
#16 Idler::<lambda()>::operator() (__closure=0xd8b608) at /usr/src/debug/calaos-server/1_3.0-rc1+r0+gitAUTOINC+19f8b098fd-r0/git/src/lib/Timer.cpp:136
#17 sigc::adaptor_functor<Idler::singleIdler(sigc::slot<void>)::<lambda()> >::operator() (this=0xd8b600) at /usr/include/sigc++-2.0/sigc++/adaptors/adaptor_trait.h:256
#18 sigc::internal::slot_call<Idler::singleIdler(sigc::slot<void>)::<lambda()>, void>::call_it(sigc::internal::slot_rep *) (rep=0xd8b5d0)
at /usr/include/sigc++-2.0/sigc++/functors/slot.h:461
#19 0x000000000066f84a in sigc::internal::signal_emit0<void, sigc::nil>::emit (impl=0xd6ef20) at /usr/include/sigc++-2.0/sigc++/signal.h:794
#20 sigc::signal0<void, sigc::nil>::emit (this=<optimized out>) at /usr/include/sigc++-2.0/sigc++/signal.h:2800
#21 Idler::<lambda(const auto:32&, auto:33&)>::operator()<uvw::IdleEvent, uvw::IdleHandle> (__closure=<optimized out>)
at /usr/src/debug/calaos-server/1_3.0-rc1+r0+gitAUTOINC+19f8b098fd-r0/git/src/lib/Timer.cpp:125
#22 std::_Function_handler<void(uvw::IdleEvent&, uvw::IdleHandle&), Idler::createIdler()::<lambda(const auto:32&, auto:33&)> >::_M_invoke(const std::_Any_data &, uvw::IdleEvent &, uvw::IdleHandle &) (__functor=..., __args#0=..., __args#1=...) at /usr/include/c++/6.3.0/functional:1731
#23 0x000000000066f95e in std::function<void (uvw::IdleEvent&, uvw::IdleHandle&)>::operator()(uvw::IdleEvent&, uvw::IdleHandle&) const (__args#1=..., __args#0=...,
this=<optimized out>) at /usr/include/c++/6.3.0/functional:2127
#24 _ZZN3uvw7EmitterINS_10IdleHandleEE7HandlerINS_9IdleEventEE7publishES4_RS1_ENKUlOT_E_clIRSt4pairIbSt8functionIFvRS4_S6_EEEEEDaS8_ (element=..., __closure=<synthetic pointer>)
at /usr/src/debug/calaos-server/1_3.0-rc1+r0+gitAUTOINC+19f8b098fd-r0/git/src/lib/uvw/src/uvw/emitter.hpp:118
#25 _ZSt8for_eachISt16reverse_iteratorISt14_List_iteratorISt4pairIbSt8functionIFvRN3uvw9IdleEventERNS4_10IdleHandleEEEEEEZNS4_7EmitterIS7_E7HandlerIS5_E7publishES5_S8_EUlOT_E_ET0_SI_SI_SL_ (__first=..., __last=..., __f=...) at /usr/include/c++/6.3.0/bits/stl_algo.h:3769
#26 0x000000000066fedd in uvw::Emitter<uvw::IdleHandle>::Handler<uvw::IdleEvent>::publish (ref=..., event=..., this=<optimized out>)
at /usr/src/debug/calaos-server/1_3.0-rc1+r0+gitAUTOINC+19f8b098fd-r0/git/src/lib/uvw/src/uvw/emitter.hpp:123
#27 uvw::Emitter<uvw::IdleHandle>::publish<uvw::IdleEvent> (this=0xd8b268, event=...)
at /usr/src/debug/calaos-server/1_3.0-rc1+r0+gitAUTOINC+19f8b098fd-r0/git/src/lib/uvw/src/uvw/emitter.hpp:166
#28 uvw::IdleHandle::startCallback (handle=<optimized out>) at /usr/src/debug/calaos-server/1_3.0-rc1+r0+gitAUTOINC+19f8b098fd-r0/git/src/lib/uvw/src/uvw/idle.hpp:40
#29 0x00007ffff7dde3c4 in uv__run_idle (loop=loop@entry=0x7ffff7ff2cc0 <default_loop_struct>) at /usr/src/debug/libuv/1.19.1-r0/libuv-1.19.1/src/unix/loop-watcher.c:68
#30 0x00007ffff7dd8175 in uv_run (loop=0x7ffff7ff2cc0 <default_loop_struct>, mode=mode@entry=UV_RUN_DEFAULT) at /usr/src/debug/libuv/1.19.1-r0/libuv-1.19.1/src/unix/core.c:361
#31 0x0000000000413e99 in uvw::Loop::run<(uvw::details::UVRunMode)0> (this=<optimized out>)
at /usr/src/debug/calaos-server/1_3.0-rc1+r0+gitAUTOINC+19f8b098fd-r0/git/src/lib/uvw/src/uvw/loop.hpp:284
#32 main (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/calaos-server/1_3.0-rc1+r0+gitAUTOINC+19f8b098fd-r0/git/src/bin/calaos_server/main.cpp:185
When inspecting the stream->io_watcher.fd
it is -1. So the assert is correct that the uv_read cannot start- But why is this sometime -1... I really don't know...
(gdb) p *stream
$1 = {data = 0xd8b3b0, loop = 0x7ffff7ff2cc0 <default_loop_struct>, type = UV_NAMED_PIPE, close_cb = 0x0, handle_queue = {0xd88488, 0xd8bb78}, u = {fd = 0, reserved = {0x0,
0x0, 0x0, 0x0}}, next_closing = 0x0, flags = 8196, write_queue_size = 0, alloc_cb = 0x0, read_cb = 0x0, connect_req = 0x0, shutdown_req = 0x0, io_watcher = {
cb = 0x7ffff7de2170 <uv__stream_io>, pending_queue = {0xd8b4a8, 0xd8b4a8}, watcher_queue = {0xd8b4b8, 0xd8b4b8}, pevents = 0, events = 0, fd = -1}, write_queue = {0xd8b4d8,
0xd8b4d8}, write_completed_queue = {0xd8b4e8, 0xd8b4e8}, connection_cb = 0x0, delayed_error = 0, accepted_fd = -1, queued_fds = 0x0}
My question here is: do I need to setup something or check something prior starting pipe->read()
? Is it a bug in libuv? uvw?
Thank you very much for any pointer.