flow-ipc / ipc_core Goto Github PK

Just happened to notice an oversight in this code:

template<typename Persistent_mq_handle>
Blob_stream_mq_receiver_impl<Persistent_mq_handle>::~Blob_stream_mq_receiver_impl()
{
  // This is near-identical to Blob_stream_mq_sender_impl ctor.  Keeping comments light.  @todo Code reuse.

  FLOW_LOG_INFO("Blob_stream_mq_receiver [" << *this << "]: Shutting down.  Async-wait worker thread and timer "
                "thread will shut down/be joined shortly; but we must interrupt any currently-running async-wait to "
                "ensure async-wait worker thread actually exits.");

  if (m_mq)
  {
    m_mq->interrupt_receives();
  }
}

It does not need to do this, if (compile-time) Mq::S_HAS_NATIVE_HANDLE is false. The background thread is not necessary then and is never created or started, as we can just poll the MQ directly via poll. So there's nothing to interrupt.

It's harmless to call this but clearly a bug, as the sender dtor is:

template<typename Persistent_mq_handle>
Blob_stream_mq_sender_impl<Persistent_mq_handle>::~Blob_stream_mq_sender_impl()
{
  if constexpr(!Mq::S_HAS_NATIVE_HANDLE)
  {
    FLOW_LOG_INFO("Blob_stream_mq_sender [" << *this << "]: Shutting down.  "
                  "Async-wait worker thread and timer thread will shut down/be joined shortly; but "
                  "we must interrupt any currently-running async-wait to ensure "
                  "async-wait worker thread actually exits.");

    /* No-op if no send is pending; but it'd be boring worrying about checking m_pending_payloads_q.empty() just
     * to avoid a no-op.  It's harmless (not *exactly* a no-op, but m_mq is not long for this world -- so who cares
     * if it dies while in interrupted-sends mode or otherwise?). */

    if (m_mq)
    {
      m_mq->interrupt_sends();
    }

    /* That's it: If there was a (blocking) task on m_blocking_worker, then it has or will have stopped soon;
     * the m_timer_worker thread, at worst, might harmlessly fire auto-ping timer and signal that pipe -- which
     * we won't check or do anything about.  Now m_blocking_worker and m_timer_worker dtors will run in opposite
     * order, ending/joining those threads, and then this dtor will return.  Oh, and then m_mq is safely destroyed
     * too. */
  } // if constexpr(!Mq::S_HAS_NATIVE_HANDLE)
} // Blob_stream_mq_sender_impl::~Blob_stream_mq_sender_impl()

Note the if constexpr().

Trivial fix albeit low-impact functionally.

Filed by @ygoldfeld pre-open-source:

This issue does not (technically at least) violate any concept (namely Blob_sender/receiver), and there are likely plenty of mitigating facts that make it not matter super-much, but nevertheless it is a "miss":

Consider a Native_socket_stream (Unix domain socket-based bidirectional pipe). The way to gracefully end a connection is to [async_*]end_sending() on one side; the other side will receive a special error indicating this happened, once it has received all the preceding messages. This is the equivalent of a socket EOF (recv() in Berkeley sockets returns 0). Normally users should use this and/or their own user-space end-of-convo message. However, if the user simply destroys a peer N_s_s, the other N_s_s will sooner or later (investigate when exactly, I always forget these exact semantics without shutdown()) emit an error; whether it's EOF or broken-pipe.

By contrast, unless I am forgetting something, if one destroys Blob_stream_mq_sender – without *end_sending() – Blob_stream_mq_receiver will not be informed. Internally this is simply because the underlying MQ (whether POSIX or bipc) continues to exist, even though that dtor removes the named MQ from file system. The handle is still valid, so the MQ remains.

We should somehow signal this event. As to specifics:

• It could be a new CONTROL message type. Could even just send the CONTROL message of type END_SENDING (the FIN-like guy).
• Try to mimic what Unix domain sockets do if one just close()s a side, particularly if the socket is at that time write-blocked (buffer full) and also not write-blocked but with non-empty buffer; does the close() trigger some kind of immediate RST-ish thing? Or does it simply mimic a FIN-type thing (graceful close)? Might require a bit of investigation. Anyway do something similar in corner cases, to the extent possible.
• On the other hand don't worry too much about those; this should be a best-effort solution all in all.

A main mitigation is that user protocols should be written with graceful closing built in, and we provide full support for that already ([async_]end_sending() and the associated error on the receiver). Other mitigations include the fact that the underlying session in many cases will end too, so user would end all related channels if properly coding; the session uses a UDS for the session master channel, so even an abrupt ending will be signaled without a problem. Plus if the Channel (if any) with the blob-stream-MQ pipe(s) has a UDS pipe too, that one will signal the end of the channel.

None of those is 100%, so we should give a decent effort for this.