PCSX2SaveStateImporter: IllegalArgumentException: newPosition > limit about ghidra-emotionengine-reloaded

[chaoticgd]

I could have a better look at this later, although for now I would consider just extracting the eeMemory.bin file and importing that on its own. The save state importer was inherited from ghidra-emotionengine and I'm not sure there's too much use in it. The additional memory blocks it imports aren't usually relevant since they just contain scratch data from whatever point the save state was taken at.

[chaoticgd]

Thought of something: New versions of PCSX2 use zstd compression for savestates. It's possible that's causing problems. Haven't tested this though. You could try adding SavestateZstdCompression=disabled to your PCSX2_vm.ini file in the EmuCore section.

[chaoticgd]

Do tell me if my idea works.

[yutamago]

Sorry, will try it later today

[yutamago]

Importing only the extracted eeMemory.bin results in following error:

Cannot invoke "ghidra.formats.gfilesystem.GFileSystem.lookup(String)" because "gfs" is null
java.lang.NullPointerException: Cannot invoke "ghidra.formats.gfilesystem.GFileSystem.lookup(String)" because "gfs" is null
	at PCSX2SaveStateImporter.getBuffer(PCSX2SaveStateImporter.java:51)
	at PCSX2SaveStateImporter.loadMainMemory(PCSX2SaveStateImporter.java:59)
	at PCSX2SaveStateImporter.run(PCSX2SaveStateImporter.java:36)
	at ghidra.app.script.GhidraScript.executeNormal(GhidraScript.java:397)
	at ghidra.app.script.GhidraScript.doExecute(GhidraScript.java:252)
	at ghidra.app.script.GhidraScript.execute(GhidraScript.java:230)
	at ghidra.app.plugin.core.script.RunScriptTask.run(RunScriptTask.java:47)
	at ghidra.util.task.Task.monitoredRun(Task.java:134)
	at ghidra.util.task.TaskRunner.lambda$startTaskThread$0(TaskRunner.java:106)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)

---------------------------------------------------
Build Date: 2023-May-10 1508 EDT
Ghidra Version: 10.3
Java Home: C:\Program Files\Java\jdk-17.0.2
JVM Version: Oracle Corporation 17.0.2
OS: Windows 11 10.0 amd64
Workstation: **********

And I'm using PCSX 1.6.0 (the latest stable), which doesn't seem to have zstd compression yet.
But I tried the config anyway, opened up the game again (Final Fantasy X) loaded and re-saved.
Importing that file and I got the same error as in my first message.

This is my savestate if it helps:
SLUS-20312 (BB3D833A).00.zip

[chaoticgd]

If you're just importing the eeMemory.bin file the PCSX2 save state import script isn't necessary at all. You could just drag the eeMemory.bin file (or the boot ELF) into ghidra's project browser and select the R5900 processor. As I said that script really isn't very useful.

[chaoticgd]

Having trouble reproducing this. Are you running the script from a file setup to use the R5900 processor (probably should've asked that first actually)?

[yutamago]

I'm new to Ghidra, so I'm not ruling out mistakes on my side.
The "About " option indicates that I'm using the "MIPS-R5900" processor.

To be honest, I didn't really know what to expect from the script. I thought the whole deal with it was, that it filled out missing parts in the PS2 game data that I'm looking at. Not open another file in a different window.

So what I did, is this:

• open the SLUS_203.12 file in CodeBrowser (Final Fantasy X (USA))
• run the plugin

In the meantime I tried asking GPT-4 to explain and fix the issue and I'm testing the proposed changes right now. At least I'm not getting an error anymore.

Here the explanation and the fix that it came up with for the loadMainMemory function:

GPT-4: There seems to be an assumption in the method that the maxAddress will always be less than or equal to the size of the ByteBuffer (i.e., the buf.limit()). But this assumption may not hold true, as the maxAddress is based on the memory block offsets and not the size of the data in the ByteBuffer. If the maxAddress exceeds the size of the ByteBuffer, you get the "newPosition > limit" error.

To fix this, you could revise the method to compute the maxAddress based on the size of the data in the ByteBuffer (i.e., the buf.limit()), rather than the end offsets of the memory blocks:

private void loadMainMemory(GFileSystem gfs) throws Exception {
    ByteBuffer buf = getBuffer(gfs, MAIN_MEMORY);
    MemoryBlock[] blocks = getMemoryBlocks();
    monitor.initialize(blocks.length);
    monitor.setMessage("Loading Main Memory...");
    for (MemoryBlock block : blocks) {
        monitor.checkCanceled();
        if (block.isWrite() && !block.isExecute()) {
            // only load and replace writable, non-executable memory blocks
            int offset = (int) block.getStart().getOffset();
            if (offset < buf.limit()) {
                buf.position(offset);
                replaceBlock(block, buf);
            }
        }
        monitor.incrementProgress(1);
    }
    long maxAddress = buf.limit();  // compute maxAddress based on ByteBuffer limit
    Address otherAddress = toAddr(maxAddress);
    buf.position((int) maxAddress);
    if (buf.hasRemaining()) {
        byte[] bytes = new byte[buf.remaining()];
        buf.get(bytes);
        MemoryBlock block = getMemoryBlock(OTHER_BLOCK);
        if (block == null) {
            block = createMemoryBlock(OTHER_BLOCK, otherAddress, bytes, false);
            block.setRead(true);
            block.setWrite(true);
        }
    }
}

[chaoticgd]

Sorry about the lack of instructions, I should probably update the readme with a "Getting Started" section or something. But basically, you don't need the PCSX2 save state import script at all. If you've already imported a file and selected the R5900 processor and run auto analysis you're all set, you don't need to do anything else.

[yutamago]

Cool, thanks! And sorry for my lack of experience. :)

[chaoticgd]

Actually I think I might've been misunderstanding the purpose of that script a bit. As I said I just inherited it from the original project. I take back my comments that it wasn't useful. Just I haven't used it much personally.

[chaoticgd]

I was initially confused as to how maxAddress could possibly be higher than 32MB, but I noticed the MAX_MEMORY variable in the script is actually set to 256MB, not 32MB. Sorry about this. When you import an ELF file without running the script, what are the memory blocks that get created (in the Memory Map window)?

[chaoticgd]

@yutamago Alright, I think I've figured it out: There were some ELF sections that were being imported as overlay sections by Ghidra's ELF importer, which lead to their start/end addresses being junk. So the correct fix seems to be to just skip over those sections.

[chaoticgd]

Alright, the script should work now. An unstable build with the fix should now be automatically generated, I'll wait a bit to put out a stable release since I have some other fixes I want to do first.

[yutamago]

Sorry if I'm asking stupid questions, but as a beginner in Ghidra I'm a bit confused what kind of purpose the script has.
You're saying that it does indeed have a different purpose than just looking at a eeMemory.bin file?

[chaoticgd]

From looking over the script it does three things:

1. Replaces the contents of existing non-executable blocks with the relevant data from the save state.
2. Creates a new block with all the data between the end of the last existing block and the end of EE memory. Hence it should import most of the contents of EE memory (if not all) into the file.
3. Loads additional sections of memory such as the scratchpad, VU0 memory and VU1 memory.

I hadn't previously noticed that it did (2), hence why I didn't think it had much use. (1) could be useful, but would probably only change the state of global variables to their runtime state, and (3) would probably not be very useful as those memory areas are only used for storing temporary data.

Hopefully that clears things up a bit. Don't apologize for asking "stupid" questions, I've been using Ghidra for years and have discovered some of its quirks for the first time while fixing this issue.

[chaoticgd]

I'm mostly interested in the R&C games. The script isn't very useful for those games as-is because they store the vast majority of their code in overlays in the level files, which it wouldn't handle correctly. It should be possible to come up with a script that works better for those games by importing the overlays first, but the simple solution of just importing the eeMemory.bin file works fine hence so far I haven't bothered.