Resource Leak Issue with Files.list in ai.djl.util.Utils about djl

[frankfliu]

@yhzdys

From API point of view, we should close the Stream, however, there is no resource leak in Files.list() case. The DirectoryStream object doesn't hold any resource.

[yhzdys]

😃Thank you for your response.

Files.list() relies on file descriptors to manage the underlying file system resources. However, while Java's Garbage Collection automatically handles the closure of DirectoryStream objects created by Files.list(), there's a potential risk of file descriptor exhaustion before GC is triggered.

This occurs because GC operates non-deterministically, meaning it's uncertain when exactly it will run to reclaim resources. Consequently, if file descriptors aren't properly managed and closed, there's a possibility of exhausting system resources, leading to performance issues or even system crashes.


[yhzdys]

😃Thank you for your response.  Files.list() relies on file descriptors to manage the underlying file system resources. However, while Java's Garbage Collection automatically handles the closure of DirectoryStream objects created by Files.list(), there's a potential risk of file descriptor exhaustion before GC is triggered.  This occurs because GC operates non-deterministically, meaning it's uncertain when exactly it will run to reclaim resources. Consequently, if file descriptors aren't properly managed and closed, there's a possibility of exhausting system resources, leading to performance issues or even system crashes.

Upon reviewing the discussion at Stack Overflow, I've realized my previous understanding was mistaken. The feedback there confirms that Java's GC, while eventually handling DirectoryStream objects, does not reliably or promptly close file descriptors. To avoid leaks and ensure efficient resource management, employing try-with-resources is the recommended approach.

In light of this, updating methods like ai.djl.util.Utils.getNestedModelDir() with try-with-resources would align with best practices, ensuring file descriptors are closed in a timely manner. My previous assertion lacked accuracy, and I concur with the necessity for these improvements.