Thread Pool: no nice way to do a cleanup about threads.js

[andywer]

Yeah, the pool shutdown is a bit tricky, since usually the pool is not terminated right away after doing an initial chunk of work.

That kill vs on('exit') situation is interesting. Maybe it might be a good idea to change the .kill() behavior, like performing a child.exit() and setting some timeout to only call child.kill() if it didn't already shut down during this timeout?

[mijay]

I would argue that sendAll would be both simpler to implement and more logical api to have in Pool. Plus that can be easily implemented cross-platform. Of course, then everyone who want to support cleanup scenario like I've described, will have to do it themselves, but that sounded more like a FAQ topic.

[andywer]

Fair enough, that is a good argument.

Just out of curiosity... What kind of clean-up do you do?

[andywer]

If you need the sendAll urgently I'd say open a PR and get credited in the changelog. I might implement it as well, but it's not a high priority to me.

Anyway I can publish v0.7.3 containing the unhandled promise rejection handler today.

[mijay]

I ended up using hack 1 and it works for now, just not beautiful enough - so no urgency.

I am using a pool of runners to run bunch of selenium tests in parallel, so I need to shutdown the driver used by each thread properly when there are no more tests left.

[AslanGoldenhour]

Just adding to this… I'm having trouble releasing my database connection from within a worker when the worker fails prematurely. That's my primary use case for cleanup. That and global.gc() when running Node with the --expose-gc flag.

EDIT: I didn't see code before I first posted…

{
    job.destroy();                    // to prevent memory leak
    this.idleThreads.push(thread);
    this.emit('threadAvailable');

    if (this.idleThreads.length === this.threads.length) {
      // run deferred to give other job.on('done') handlers time to run first
      setTimeout(() => { this.emit('finished'); }, 0);
    }
}

What are your thoughts on adding something just before job.destroy() for manual cleanup processes? e.g. (but with ES6)
if (typeof job.cleanup === "function") await job.cleanup();

[andywer]

@ZenModeRy Thanks for sharing your thoughts. To make if (typeof job.cleanup === 'function') possible, we would need to extend the API of Job, so it's not exactly a "done in ten minutes" change.

But a proper way to clean-up after a job would be nice. I will put it on the "version 1.0 wishlist" #54 😉

[andywer]

I think this should be fixed in v1.0. Closing. Feel free to re-open if it still happens in 1.0.