Consider mechanism for a watchdog of sorts about poshbot

[RamblingCookieMonster]

Here's some log output now that we're running at a higher logging level!

DataTime             Class           Method         Severity LogLevel Message                                   Data
--------             -----           ------         -------- -------- -------                                   ----                  
2017-08-16 20:35:52Z SlackBackend    ReceiveMessage Normal   Debug    Message type is [reconnect_url]. Ignoring     
2017-08-16 21:43:06Z Bot             Disconnect     Normal   Verbose  Disconnecting from backend chat network       
2017-08-16 21:43:06Z SlackConnection Disconnect     Normal   Info     Closing websocket                             
2017-08-16 21:43:06Z SlackConnection Disconnect     Normal   Info     Stopping receive job [1]                      
2017-08-16 21:43:06Z Bot             Disconnect     Normal   Verbose  Disconnecting from backend chat network       
2017-08-16 21:43:06Z SlackConnection Disconnect     Normal   Info     Closing websocket                             

So, ~ 1 hour goes by and it just... disconnects. The time seems arbitrary. Some times it will be days.
I don't know if it's activity/idle time related though, given that this was the middle of the day. Haven't traced through the code to see where other debug logging might be helpful, should have time over the next few weeks

Cheers!

[devblackops]

Thanks @RamblingCookieMonster. I haven't seen this issue with the backend disconnecting randomly but I did add some reconnection logic in commit f070972. I tested this by:

1. Starting the bot and verifying it is connected and processing messages/commands
2. Disabling the NIC
3. Watching the logs to see the disconnect happen and the reconnection logic trigger repeatedly (every 5 seconds)
4. Re-enabling the NIC
5. Observe the backend immediately logging into Slack again and begin processes messages.

Mind given this version a go?

[devblackops]

@RamblingCookieMonster Are the connection drops still an issue after the reconnection logic f070972 was put in?

[RamblingCookieMonster]

Nope, that worked perfectly, closing this out - thanks!