Thread handle leaks/process handle issue about retdec-idaplugin

[s3rvac]

Thank you for the report. @PeterMatula, can you please verify?

[GregoryMorse]

I have submitted a PR for this if you prefer. Please refer to #42

Update: both current and child process thread handle leaks I verified (the latter being Windows only). The process handle only leaks on a forcible process termination since IDAs term_process API must be used here.

[GregoryMorse]

I would also point out that this bug exists in all of the qthread_create's documented as needed for a crash issue with relaunching the code viewer (4 times in code_viewer.cpp and once in idaplugin.cpp).

I have tested removing the threads and just launching it from the main thread and it works completely fine. I have no idea what environment or situation led to thinking there is a crash. This workaround is bizarre as to do it cleanly requires tracking thread handles and even having a qthread_join, and also to avoid deadlock on shutdown, MFF_NOWAIT would need to be used, the request id kept for cancel_exec_request on shutdown, probably needing a mutex and semaphore for synchronization, etc.

My point is mainly that the current situation is a hack - it should all be doable from the main thread. At least on Windows 10 with IDA 7.2 this seems straight forward and working. Otherwise, it would be easier to debug and solve than to over code a fancy solution to do a clean workaround.

Leaking threads and having potential for unclean shutdown is an easy workaround but something that should be fixed in the long run.

My suspicion is originally a thread was designed to not block the main thread for the primary decompilation. This thread made calls like get_func_name or get_func_cmt which would sometimes cause crashes - because sometimes they access netnodes and are not thread safe API. If IDA does not mark something as thread safe then it must be executed in the main thread. But the only justifiable thread in this plugin is the one for launching and waiting for the decompilation process. The UI stuff seems like a relic from a time when things were not structured as they are now. Additionally, MFF_FAST is wrong anyway as the get_ calls require MFF_READ per docs.

So I recommend just removing the 5 creations of that thread and calling directly and resolving that whole set of leaks/unclean termination issues.

However, the apparent purpose of showDecompiledCode for the use singularly in decompiler.cpp - that one is certainly necessary or it will crash. With the previous fixes discussed the thread handle will not leak. Ideally the shutdown should go smoothly if it occurs exactly when the code viewer request came in. So it should likely be called with MFF_NOWAIT and the request id tracked so it can be cancelled on termination. The ShowOutput class would need to be dynamically allocated with the new operator, as IDA in a cancellation case automatically calls delete - not really well documented. This makes yet something else to deal with - something to signal success so it can be freed. Better is to not exit the thread and wait on a semaphore signaled on run. But to do clean termination then further a mutex locking mechanism is needed. The headaches of multithreaded code :).

I will just include an elegant solution as a PR since this is more than enough developer commentary.

[PeterMatula]

Fixed by merging #42.