Timeout when waiting for a temination of a subprocess is hardcoded about tbot

[Rahix]

We can make this timeout configurable, but I think a better solution would be to actively shutdown the VM. This "kill all lingering subprocesses" code is mostly a safeguard and not really meant to be used as part of normal machine teardown.

Maybe you can post a rough sketch of what your machine config looks like for this VM machine?

[yanivy-nr]

Hello,

Thank you for the prompt response.

After further investigation, it looks like the issue is due to ssh process used by the SSHConnector is not always being terminated during close() in tbot/machine/channel/subprocess.py.
The SSH process is a child process of bash. During the call to close() in subprocess.py, a TERM signal is sent to bash which is the parent process, but on some occasaions it is not propagated to the child processes.

I have written a code that will loop through the child processes and send TERM to each one. In case the child is still running after a few seconds, it will be sent a KILL signal. This code fixed the issue for us.

Will send a PR in the next few days (I would like to thoroughly test it in our CI prior sending the PR).

thanks,
Yaniv