This is my personal mod base. I created it because I got tired of having to manually copy all my new changes to every repo I had. By having a mod base, I can more easily maintain all my mods.
As I said, this is my personal mod base. Feel free to use it, but expect limited/no support. This mod base doesn't have any fancy stuff that would make things easier for beginners. It is designed to have as little bloat as possible.
The mod base contains a level called lltest2. You may find some examples of the mod base's features implemented in it. Please do not change anything about the level, otherwise you will run into merge conflicts. In the future I might add more and more examples into it, to show people what are currently possible.
The c++ side now has enough "space" to allow us to load all SBK files at once. The first time the game wants to load a level sound bank (SBK file), it will load all the SBKs instead. Default functionality of the level soundbank loading is removed/disabled. I emptied the sound-banks property of all level-load-info objects, because the changes I made make that property redundant/useless.
This change is currently only implemented for TPL. It wouldn't make too much sense for Jak II, because that game has sounds with same name but different content in different SBKs, so having all SBKs loaded could mess with what you hear ingame. There are no such conflicts in TPL, so it is safe there.
I borrowed the custom music system from 'The Forgotten Lands' mod, and made some changes to it. As far as I know, the base of that whole system was initially created by Zed, and then it was greatly expanded by Hat-Kid. Please let me know if this is inaccurate and I will adjust this section. Later the other modbase found a worthy replacement for SFML, called 'miniaudio', and they let me look at their changes. I implemented these on our side.
I implemented a way to use custom music for music ambients. To use it, just use the custom-music lump instead of the music lump in your music ambient. Also expanded the 3D debug text that is drawn when you are displaying music ambients.
I made some other minor changes to the custom music system for now, the GOAL side is quite different now but the c++ side is mostly unchanged. For the future I want to make its namings more generic/neutral (so they don't refer to The Forgotten Lands in the name). I also want to implement a way to have music variants/flavors, and I already have a clear vision for it, but I will have to rewrite a lot of code on the c++ side. And while there, I will probably do a general cleanup of that code as well. Also, for now, the TFL hints system is not reimplemented, since we switched to miniaudio. It is planned though.
I have extended the functionality of the load boundaries, to allow restricting them by tasks. You can do this with the new :tasks thing I added. If it is not added, things will behave like before. But when it is there, all the listed tasks will need to be completed before the load boundary works. Due to my lack of skill, you have to pass the tasks as strings. But hopefully it can be improved in the future.
See here for an example:
(static-load-boundary :flags (player )
:top 322236.6562 :bot -524288.0000
:points ( -140670.9062 -353851.7500 6646.9482 -375187.6875 -4327.1040 -433914.7187 )
:fwd (load village1 firecanyon)
:tasks ("training-gimmie" "training-door")
)
I made changes that allow placing custom navmesh into Jak 1 levels. This will hopefully become useless one day, if proper navmesh support is ever added to OpenGOAL.
The navmesh system in Jak II is more advanced, I haven't managed to figure it out yet.
Please keep in mind that you are expected to be familiar with custom levels and GOAL. Still, I tried to make things as understandable as possible.
I would recommend copying an existing navmesh as a start. You can use the inspect method I made. The actor whose navmesh you want to copy must be loaded. Example:
(inspect (-> (the-as entity-actor (entity-by-name "snow-bunny-55")) nav-mesh))
You should change the origin and bounds, depending on where you want to place your navmesh.
I usually just remove the nodes, because I do not understand it and things seem fine without it. But keep in mind that every navmesh that is in the game has at least one node.
We do not understand route, but it is needed - otherwise game will crash. If you copy an existing navmesh, the route data is copied correctly. But since we don't understand it, for fully custom navmesh we can never have proper route data. Correct route data is essential if you want to take advantage of gap triangles (where enemies jump).
You can make multiple enemies use the same navmesh. To do this, create the navmesh through code for the first actor, like in the example. And for the other actors, add a lump that tells the game to use another actor's navmesh. Reference is by aid. Example: "nav-mesh-actor": ["uint32", 40000]. Tip: You can do the same thing with paths, using the path-actor lump.
If the game crashes when you approach a custom navmesh, make sure you added :custom-hacky? #t to your custom navmesh definition. If that is there, then check if the actor has a path. It needs a path.
If something is still unclear, please look at the code. I added a lot of comments.
I am not an expert at decompiling, so my methods were not the most efficient. But with a lot of time, I managed to figure things out. There are probably people who could do this a lot better than me. Hopefully it will happen.
Also, I know my inspect method is not perfect. But it is very tedious to write such a thing, so I just included what we really need. And I think the nodes part could use a cleanup.
I am happy if anyone finds this useful. But I have a request: If you learn more about navmeshes, especially things that would benefit other modders as well, please let me know. And maybe we will add it to this branch.
- The 'PS2 Actor Vis' setting is force-disabled in TPL. This is because we have no way to control the actor birth distance, and just using this made things very convinient.
- Added a custom level actor vis forcing to the code, but it is probably currently unused due to the above change.
- I have "fixed" the index of
test-zone- it should be unique, but it wasn't. This will make it much easier to add and understand new levels, believe me. Always set the index correctly for any new levels you add. - Use the
BIG_COLLIDE_CACHE_SIZEby default. This should make the game able to handle much more collision ingame. - Ambient music marks will now try to display the music and flava data of the marks.
- The code now should refer to the
(game-task max)everywhere it is supposed to. No more hardcoded numbers. This makes it safe to add new tasks. - Moved the definition of
*game-counts*fromprogress-static.gctogame-info.gc- this needed to be done because we want to be able to add new game count entries, and that can only be done through code for now. Also added a commented out example of how to add new entries. It is at the beginning of the method below it. I don't want to add a live/working example, as that would bloat the mod base. To avoid merge conflicts, you should probably leave the example comment alone, and just add your stuff below it. - Added two new properties to the
level-load-infotype:custom-level?(you should set this to true for your custom levels), andcustom-music(if your level uses custom music). Keep in mind that both of these are of typesymbol. The default value of that type is 0. So if you want to check these in conditions, you need to be aware that they won't actually be#fin the levels where they weren't defined... they will be 0, which is actually considered "not false" in GOAL, so "true". I am considering defining these properties with value#fin the existing levels... - Commented out some annoying "Failed to find texture at..." console logs, because they were being spammed. I don't think they are really useful nowadays anyway.
- Commented out an assert (
ASSERT(diff < 7);), which was potentially making it more difficult to build custom levels that had ugly collision geometry. I am not sure how much it helps though, but hopefully it helps somewhat. - Added a hack that allows us to define the volumes for the water volumes. This hack was invented by Hat-Kid, and I still don't understand why it works, but it does.
- Borrowed a trick from the other mod base (the one by barg and Zed). As you may know, the project dir is found by looking for
jak-projectin the directory path. But if your repository doesn't have that name, it won't be found. They rewrote the function that finds the project directory. But I think it is not working on all platforms yet. I will keep observing their modbase and update this if necessary. What we borrowed for now is working fine on Windows. - Improved the debug Nav Mesh display. If you enable 'Nav Mesh Extras', you will see the IDs of the vertexes and the triangles.
- When you press L2+UP, the HUD now shows the true maximum obtainable number of orbs, flies, and cells instead of the hardcoded values of 2000, 112, and 101.
- Implement flavor/variant support for custom music.
- Make the TFL hints system work again.
- Clean up GitHub Actions.
- Understand more about the navmesh system (node, route).
- Show how to add new
battleactors, also show the part that needs to be done through code. I have already touched the battle system in the past. I should clear things up and implement an example. - Rewrite the warp gate system, to make it more extendable. I imagine a lot of custom level mods will want to use the warp gate. But its code is ugly and not easy to extend.
- Re-allow defining a water-actor for pontoons. It has valid use cases.
While you may think that forking this mod base is preferable, keep in mind that a fork of a public repo can not be private. For this reason, you might want to consider duplicating this repo, and adding this mod base as a remote so you can update from it.
Here is a guide for duplicating the repo: https://docs.github.com/en/repositories/creating-and-managing-repositories/duplicating-a-repository
Once your repository exists, you should add this mod base as a remote. This will allow you to stay up to date:
git remote add ll-modbase https://github.com/LuminarLight/LL-OpenGOAL-ModBase
The command above only needs to be ran once per repository. Afterwards, here is how you can update from the mod base any time:
git fetch ll-modbase
git merge ll-modbase/master --allow-unrelated-histories --no-commit
The --no-commit flag will prevent the command from automatically doing a commit. This will allow you to review all the changes. I think this is very preferable, so please keep using this flag. Also, if you remove the flag, things will still only be instantly commited if there are no conflicts.
This is how I update the mod base to stay up to date with OpenGOAL. You don't need to do this (I am supposed to do this to keep the mod base up to date), it is only here so I don't forget (first line only needs to be ran once per repo):
git remote add vanilla https://github.com/open-goal/jak-project
git fetch vanilla
git merge vanilla/master --allow-unrelated-histories --no-commit
As I said, this is my personal mod base. Support is limited/nonexistent. Still, I imagine there may be people who will find it useful. I am happy if that happens. And you can still reach out to me if you want, but I won't be able to help with basic stuff.
I tried to mention if a change was developed by someone else. If I missed anyone, please let me know and I will correct it. On the other hand, if you use this mod base or parts of it, please be respectful regarding crediting the people behind the features/changes.
~~Luminar Light
Important
Our repositories on GitHub are primarily for development of the project and tracking active issues. Most of the information you will find here pertains to setting up the project for development purposes and is not relevant to the end-user.
For a setup guide on how to install and play the game there is the following video that you can check out: https://youtu.be/K84UUMnkJc4
For questions or additional information pertaining to the project, we have a Discord for discussion here: https://discord.gg/VZbXMHXzWv
Additionally, you can find further documentation and answers to frequently asked questions on the project's main website: https://opengoal.dev
Warning
Do not use this decompilation project without providing your own legally purchased copy of the game. We do not distribute any assets from the game - you must use your own legitimately obtained PS2 copy of the game. We support every retail PAL, NTSC, and NTSC-J build, including Greatest Hits copies.
- Project Description
- Setting up a Development Environment
- Technical Project Overview
This project is to port the original Jak and Daxter and Jak II to PC. Over 98% of the games are written in GOAL, a custom Lisp language developed by Naughty Dog. Our strategy is:
- decompile the original game code into human-readable GOAL code
- develop our own compiler for GOAL and recompile game code for x86-64
- create a tool to extract game assets into formats that can be easily viewed or modified
- create tools to repack game assets into a format that our port uses.
Our objectives are:
- make the port a "native application" on x86-64, with high performance. It shouldn't be emulated, interpreted, or transpiled.
- Our GOAL compiler's performance should be around the same as unoptimized C.
- try to match things from the original game and development as possible. For example, the original GOAL compiler supported live modification of code while the game is running, so we do the same, even though it's not required for just porting the game.
- support modifications. It should be possible to make edits to the code without everything else breaking.
We support both Linux and Windows on x86-64.
We do not support, or plan to support the ARM architecture. This means that this will not run on devices such as an M1 Mac or a mobile device.
Jak 1 is largely playable from start to finish with a handful of bugs that are continually being ironed out. Jak 2 is in development.
YouTube playlist: https://www.youtube.com/playlist?list=PLWx9T30aAT50cLnCTY1SAbt2TtWQzKfXX
To help with decompiling, we've built a decompiler that can process GOAL code and unpack game assets. We manually specify function types and locations where we believe the original code had type casts (or where they feel appropriate) until the decompiler succeeds, then we clean up the output of the decompiled code by adding comments and adjusting formatting, then save it in goal_src.
Our decompiler is designed specifically for processing the output of the original GOAL compiler. As a result, when given correct casts, it often produces code that can be directly fed into a compiler and works perfectly. This is continually tested as part of our unit tests.
The remainder of this README is catered towards people interested in building the project from source, typically with the intention on contributing as a developer.
If this does not sound like you and you just want to play the game, refer to the above section Quick Start
All three Linux systems are supported using Docker.
Pick your supported prefered flavour of linux and build your chosen image
docker build -f docker/(Arch|Fedora|Ubuntu)/Dockerfile -t jak .
This will create an image with all required dependencies and already built.
docker run -v "$(pwd)"/build:/home/jak/jak-project/build -it jak bash
Note: If you change the content of the build/ directory you'll need to rerun the build command. Alternatively you can get the build via docker cp.
This will link your build/ folder to the images so can validate your build or test it on an external device.
Docker images can be linked into your IDE (e.g. CLion) to help with codesniffing, static analysis, run tests and continuous build.
Unfortunately you'll still need task runner on your local machine to run the game or instead, manually run the game via the commands found in Taskfile.yml.
Install packages and init repository:
sudo apt install gcc make cmake build-essential g++ nasm clang-format libxrandr-dev libxinerama-dev libxcursor-dev libpulse-dev libxi-dev python libgl1-mesa-dev libssl-dev
sudo sh -c "$(curl --location https://taskfile.dev/install.sh)" -- -d -b /usr/local/binCompile:
cmake -B build && cmake --build build -j 8Run tests:
./test.shNote: we have found that clang and lld are significantly faster to compile and link than gcc, generate faster code, and have better warning messages. To install these:
sudo apt install lld clangand run cmake (in a fresh build directory) with:
cmake -DCMAKE_SHARED_LINKER_FLAGS="-fuse-ld=lld" -DCMAKE_EXE_LINKER_FLAGS="-fuse-ld=lld" -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ ..Install packages and init repository:
sudo pacman -S cmake libpulse base-devel nasm python libx11 libxrandr libxinerama libxcursor libxi
yay -S go-taskFor Arch only, replace task with go-task in the rest of the instructions.
Compile:
cmake -B build && cmake --build build -j 8Run tests:
./test.shInstall packages and init repository:
sudo dnf install cmake python lld clang nasm libX11-devel libXrandr-devel libXinerama-devel libXcursor-devel libXi-devel pulseaudio-libs-devel mesa-libGL-devel
sudo sh -c "$(curl --location https://taskfile.dev/install.sh)" -- -d -b /usr/local/binCompile with clang:
cmake -DCMAKE_SHARED_LINKER_FLAGS="-fuse-ld=lld" -DCMAKE_EXE_LINKER_FLAGS="-fuse-ld=lld" -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -B build
cmake --build build -j$(nproc)Run tests:
./test.shWe primarily use Visual Studio on Windows for C++ development. Download the latest community edition from here. At the time of writing this is Visual Studio 2022.
You will require the Desktop development with C++ workload. This can be selected during the installation, or after via the Visual Studio Installer, modifying the Visual Studio Installation.
On Windows, it's recommended to use a package manager, we use Scoop. Follow the steps on the bottom of the homepage here to get it.
Once Scoop is installed, run the following commands:
scoop install git llvm nasm python taskClone the repository by running the following command in your folder of choice.
git clone https://github.com/open-goal/jak-project.gitThis will create a jak-project folder, open the project as a CMake project via Visual Studio.
Then build the entire project as Windows Release (clang). You can also press Ctrl+Shift+B as a hotkey for Build All. We currently prefer clang on Windows as opposed to msvc, though it should work as well!
NOTE: At this time you can only run the game on macOS if you have an Intel processor.
Ensure that you have Xcode command line tools installed (this installs things like Apple Clang). If you don't, you can run the following command:
xcode-select --installbrew install go-task/tap/go-task
brew install cmake nasm ninja go-task clang-format
cmake -B build --preset=Release-macos-clang
cmake --build build --parallel $((`sysctl -n hw.logicalcpu`))Not Supported at This Time
brew install go-task/tap/go-task
brew install cmake ninja go-task clang-format
cmake -B build --preset=Release-macos-clang
cmake --build build --parallel $((`sysctl -n hw.logicalcpu`))You may have to add the MacOS SDK to your LIBRARY_PATH:
export LIBRARY_PATH="$LIBRARY_PATH:/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib"
If you either don't want to or cannot use Visual Studio for working with the C++ project, VSCode is a good alternatively.
The clangd extension is recommended and requires clangd to be on your $PATH. If you can run clangd in a terminal successfully then you should be good to go.
Once you generate your CMake for the first time the clangd LSP should be able to index the project and give you intellisense.
TODO - Consider Contributing Documentation :)
Getting a running game involves 4 steps:
- Build C++ tools (follow Getting Started steps above for your platform)
- Extract assets from the game
- Build the game
- Run the game
First, setup your settings so the following scripts know which game you are using, and which version. For the black label version of the game, run the following in a terminal:
task set-game-jak1
task set-decomp-ntscv1For other versions of the game, you will need to use a different -set-decomp-<VERSION> command. An example for the PAL version:
task set-game-jak1
task set-decomp-palRun
task --listto see the other available options
At the time of writing, only Jak 1 is expected to work end-to-end!
The first step is to extract your ISO file contents into the iso_data/<game-name> folder. In the case of Jak 1 this is iso_data/jak1.
Once this is done, open a terminal in the jak-project folder and run the following:
task extractThe next step is to build the game itself. To do so, in the same terminal run the following:
task replYou will be greeted with a prompt like so:
_____ _____ _____ _____ __
| |___ ___ ___| __| | _ | |
| | | . | -_| | | | | | | |__
|_____| _|___|_|_|_____|_____|__|__|_____|
|_|
Welcome to OpenGOAL 0.8!
Run (repl-help) for help with common commands and REPL usage.
Run (lt) to connect to the local target.
g >Run the following to build the game:
g > (mi)IMPORTANT NOTE! If you're not using the non-default version of the game, you may hit issues trying to run
(mi)in this step. An example error might include something like:
Input file iso_data/jak1/MUS/TWEAKVAL.MUS does not exist.This is because the decompiler inputs/outputs using the
gameNameJSON field in the decompiler config. For example if you are using Jak 1 PAL, it will assumeiso_data/jak1_palanddecompiler_out/jak1_pal. Therefore, you can inform the REPL/compiler of this via thegameVersionFolderconfig field described here
Finally the game can be ran. Open a second terminal from the jak-project directory and run the following:
task boot-gameThe game should boot automatically if everything was done correctly.
Connecting the REPL to the game allows you to inspect and modify code or data while the game is running.
To do so, in the REPL after a successful (mi), run the following:
g > (lt)If successful, your prompt should change to:
gc>For example, running the following will print out some basic information about Jak:
gc> *target*You can also start up the game without booting. To do so run the following in one terminal
task run-gameAnd then in your REPL run the following (after a successful (mi)):
g > (lt)
[Listener] Socket connected established! (took 0 tries). Waiting for version...
Got version 0.8 OK!
[Debugger] Context: valid = true, s7 = 0x147d24, base = 0x2123000000, tid = 2438049
gc> (lg)
10836466 #xa559f2 0.0000 ("game" "kernel")
gc> (test-play)
(play :use-vis #t :init-game #f) has been called!
0 #x0 0.0000 0
gc>In the graphics window, you can use the period key to bring up the debug menu. Controllers also work, using the same mapping as the original game.
Check out the pc_debug, examples and pc folders under goal_src for some examples of GOAL code we wrote. The debug files that are not loaded automatically by the engine have instructions for how to run them.
There are four main components to the project.
goalc- the GOAL compiler for x86-64decompiler- our decompilergoal_src/- the folder containing all OpenGOAL / GOOS codegame- aka the runtime written in C++
Let's break down each component.
Our implementation of GOAL is called OpenGOAL.
All of the compiler source code is in goalc/. The compiler is controlled through a prompt which can be used to enter commands to compile, connect to a running GOAL program for interaction, run the OpenGOAL debugger, or, if you are connected to a running GOAL program, can be used as a REPL to run code interactively. In addition to compiling code files, the compiler has features to pack and build data files.
Environment Agnostic
If you have installed task as recommended above, you can run the compiler with task repl
Linux
To run the compiler on Linux, there is a script scripts/shell/gc.sh.
Windows
On Windows, there is a scripts/batch/gc.bat scripts and a scripts/batch/gc-no-lt.bat script, the latter of which will not attempt to automatically attach to a running target.
The second component to the project is the decompiler.
The decompiler will output code and other data intended to be inspected by humans in the decompiler_out folder. Files in this folder will not be used by the compiler.
You must have a copy of the PS2 game and place all files from the DVD inside a folder corresponding to the game within iso_data folder (jak1 for Jak 1 Black Label, etc.), as seen in this picture:
The decompiler will extract assets to the assets folder. These assets will be used by the compiler when building the port, and you may want to turn asset extraction off after running it once.
Environment Agnostic
If you have installed task as recommended above, you can run the compiler with task decomp
Linux
To run, you can use scripts/shell/decomp.sh to run the decompiler
Windows
To run, you can use scripts/shell/decomp-jak1.bat to run the decompiler
The game source code, written in OpenGOAL, is located in goal_src. All GOAL and GOOS code should be in this folder.
The final component is the "runtime", located in game. This is the part of the game that's written in C++.
In the port, that includes:
- The "C Kernel", which contains the GOAL linker and some low-level GOAL language features. GOAL has a completely custom dynamically linked object file format so in order to load the first GOAL code, you need a linker written in C++. Some low-level functions for memory allocation, communicating with the I/O Processor, symbol table, strings, and the type system are also implemented in C, as these are required for the linker. It also listens for incoming messages from the compiler and passes them to the running game. This also initializes the game, by initializing the PS2 hardware, allocating the GOAL heaps, loading the GOAL kernel off of the DVD, and executing the kernel dispatcher function. This is in the
game/kernelfolder. This should be as close as possible to the game, and all differences should be noted with a comment. - Implementation of Sony's standard library. GOAL code can call C library functions, and Naughty Dog used some Sony library functions to access files, memory cards, controllers, and communicate with the separate I/O Processor. The library functions are in
game/sce. Implementations of library features specific to the PC port are located ingame/system. - The I/O Processor driver, OVERLORD. The PS2 had a separate CPU called the I/O Processor (IOP) that was directly connected to the DVD drive hardware and the sound hardware. Naughty Dog created a custom driver for the IOP that handled streaming data off of the DVD. It is much more complicated than I first expected. It's located in
game/overlord. Like the C kernel, we try to keep this as close as possible to the actual game. - Sound code. Naughty Dog used a third party library for sound called
989SND. Code for the library and an interface for it is located ingame/sound. - PC specific graphics code. We have a functional OpenGL renderer and context that can create a game window and display graphics on it. The specific renderers used by the game however are mostly implemented. Aside from post-processing effects, everything in the game is rendered. This is located in
game/graphics. While many liberties will be taken to make this work, the end result should very closely match the actual game. - Extra assets used by the port in some fashion, located in
game/assets. These include extra text files, icons, etc.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "github-actions[bot]")
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent