The Videbridge needs a long time to fix connection problems to the XMPP server. about jitsi-videobridge

[damencho]

What is the value of net.ipv4.tcp_retries2 in sysconf?

[freym]

$ sysctl net.ipv4.tcp_retries2
net.ipv4.tcp_retries2 = 15

[damencho]

This means that the kernel will detect broken TCP connections after the 13-15 min mark.
You can try setting it to 7, this will give you roughly 25 to 50 seconds.

[bgrozev]

@freym Can you clarify steps to reproduce? I get a "closed on error" log immediately after restarting prosody (and then a successful reconnect).

[freym]

It happened when we restarted Prosody once, but we now think the problem is related to our network setup and Kubernetes. However, reducing net.ipv4.tcp_retries2 to 7 has alleviated the problem somewhat.

[jbg]

In k8s it could also happen if the prosody deployment is restarted and the new pod(s) take a long time to go Ready (eg due to readiness or startup probes); if you're using a service DNS name to reach them from JVB then it won't resolve to the new pod IP(s) until they're Ready.

Can also be related to a too-long TTL being served on those service A records (configured in CoreDNS config on most k8s deployments) so JVB still has the old IP for a while.

[bgrozev]

The "Ping failed, the XMPP connection needs to reconnect." message you see early on should trigger a reconnect. I'm surprised that in your case it doesn't log anything about trying to reconnect.

[bgrozev]

I just tested the case where the server stops responding. As expected the XMPP pings timeout in 30-60 seconds and they trigger a re-connect. I suspect it has something to do with your setup (feel free to reopen if you have more info or more questions).