Improvements and clean up related to the JITServer AOT cache about openj9

[cjjdespres]

Attn @mpirvu, @AlexeyKhrabrov.

[AlexeyKhrabrov]

Currently the AOT cache can generate class records for the [L class, which isn't really a class name at all - it's the romClass->className of an array class, which is incorrect. These classes can't be stored in the SCC - there happens to be a workaround for this issue in SVM, but otherwise if the old AOT cache implementation ever tried to rememberClass() a class with an array class in its chain somewhere it would simply fail to store the chain in the SCC and the AOT compilation would fail. Fixing this could be as easy as recording the correct class name in the ClassSerializationRecord (I haven't tried this) but we also speculated in one of our weekly meetings that a new ArrayClassSerializationRecord could also be created, which would record the arity of the array class and the element class name.

I already have a working implementation of handling object array classes. I needed it for caching profiling data at JITServer, but I don't remember this being an issue for AOT cache. Storing the full array class name in the ClassSerializationRecord and extracting the element class name and the arity from the full name during deserialization works fine, there is no need for a separate ArrayClassSerializationRecord. But the ROMClass hash stored in the class record must be a combination of the hashes for the L[ and array element ROMClasses, and the arity. I'll contribute the implementation eventually.

[AlexeyKhrabrov]

Lambda class names (which have $$Lambda$<index>/<address> at the end) are less normalized without an SCC present. With an SCC, the <address> part is just NULL, and I think the <index> is also calculated differently (but I'm less sure about that). Performing the same transformation with an SCC not present (even if it's just at the point where a ClassSerializationRecord is created) would improve matching, as currently the "Failed to find class in class loader" deserialization error comes up more frequently in the new implementation if the original ClassSerializationRecord was created for a client that did not have a local SCC present.

This is more complicated than that, and doesn't have much to do with the local SCC (except for the <address> part being NULL or not). As far as I understand, lambda classes currently cannot be looked up by name at all using any existing APIs, they are anonymous by design. Any AOT method (for both local SCC and AOT cache) that refers to any lambda classes in relocation/validation records cannot be loaded in the current implementation.

The <index> part can vary from run to run because it depends on the order in which lambdas classes are loaded, which is generally non-deterministic. So we have to keep track of all lambda classes defined by the same host class, and disambiguate them based on the ROMClass hash instead of the index, and exclude non-deterministic parts of class name strings when computing those hashes. The same applies to other types of classes defined at runtime such as proxy classes and generated reflection accessors. Anyway, I also have a working implementation that handles all that, I'll contribute it at some point.

By the way, the same issue applies to hidden classes because they are ignored in class lookup by name. But this is a much easier fix, we simply need to add a flag to jitGetClassInClassloaderFromUTF8() and propagate it down to wherever the check for a hidden class happens.

[AlexeyKhrabrov]

I think there is an assert in ClientSessionData::getClassRecord(ClientSessionData::ClassInfo &classInfo, bool &missingLoaderInfo) that's firing in the new implementation. It indicates that there is a class chain offset being cached without a corresponding loader name. It's not fatal, really. Just odd that it's happening. I still have to track down the cause.

Which assertion?

[cjjdespres]

This assert:

         TR_ASSERT(TR_SharedCache::INVALID_CLASS_CHAIN_OFFSET != classInfo._classChainOffsetIdentifyingLoader,
                   "Valid class chain offset but missing class name identifying loader");

[AlexeyKhrabrov]

Class loader and class re-caching can still be implemented in the new deserializer, if we save the name of the class loader and the name and hash of the class when we cache the records initially.

This should be easier to do with the infrastructure I have for AOT prefetching where I store not-yet-valid records in the deserializer cache until corresponding classes get loaded. When a class gets unloaded, we can simply recreate and store its serialization record.

Another approach would be to invalidate known IDs for unloaded classes and loaders on the server side, which would be preferable in terms of client memory usage. But we would have to figure out how to synchronize things properly, maybe using the existing mechanism with critical compilation requests.