Investigate using mmap to remap global executables to local ones. about spindle

[mcfadden8]

This is only for the executable file, not libraries.

The loader creates a mapping (via mmap()) to the text/data segments of the executable file in the globally available file (stored in a file system that is shared across all the nodes). The idea is to remap those mappings to the file in the spindle cache (file system local to the node). This way, when those segments are paged in, they will be read from the local file stored locally instead of the shared one.

The implementation of this already exists and is executed when you run with the --debug=yes command line argument. The intention of this ticket is to test this feature further to determine whether we can use it by default.

One of the benefits of doing this is that it will allow readsym() to /proc/<pid>/exe to continue to work for programs that wish to use it.

[gpaulsen]

Sorry if this is a silly question:

Won't this cause a lot more memory usage for shared libraries, shared across many processes on the same node? What's the advantage?

[mcfadden8]

@mplegendre and I discovered that we were able to mmap over the entire text and data segment and /proc/<pid>/exe was preserved. I need to test this again on a TOSS-2 system to see if this is something that may have been fixed with later versions of Linux. Just adding a note here so that I don't forget.