Queued messages should not get lost on exception during sending about graphyte

[benhoyt]

Thanks for the note. This is a worthwhile idea, but it could get tricky quickly, because I think you'd want to count how many retries had been performed for a particular batch so that you don't retry an arbitrary number of times if the network goes down (and putting them back on the queue would then potentially overflow the queue, preventing new messages from being sent). Then you have to worry about how many retries, backoff, etc.

I'd probably lean towards having your monitoring system monitor the error/exception logs in send_socket and alerting you on failure so you can fix the underlying issue.

However, if you did want to do this, I'd suggest subclassing Sender and overriding send_socket in your own code. You could either just retry a couple of times in a loop if there's an exception, or push the message back onto the queue to be picked up in the next interval.

[krzysbaranski]

Also library is swallowing almost all exceptions, so even in synchronous mode you cannot retry

[benhoyt]

@krzysbaranski I presume you're referring to the except Exception in send_socket? True, though that's definitely by design. The thinking is the caller shouldn't have to worry about errors; so graphyte logs them (assuming you're doing something with your logs).

If you want to do retries or more complex behavior, you can subclass Sender and override send_socket (which is used in both sync or async mode), adding your own exception handling, retries, etc. In your override, you'd call send_message -- which doesn't have a catch-all except clause -- to do the low-level sending operation.

I'm going to close this issue per my earlier message.