xboxdev / nxdk

The cross-platform, open-source SDK to develop for original Xbox: *new* xdk

nxdk - the new open source xdk

nxdk is a software development kit for the original Xbox. nxdk is a revitalization of OpenXDK. It is maintained by the XboxDev community.

Notable features:

• Portable toolchain that works on modern versions of Windows, macOS and Linux.
• No complicated cross-compiling or big library dependencies! Builds with make and just needs standard tools and llvm.
• Modern C / C++ standards and compiler features.
• Supports popular APIs like Windows API and BSD sockets.
• SDL2 support for input, audio and 2D graphics.
• Custom API for 3D graphics using NVIDIA-designed shader-languages (with additional Xbox extensions).
• Open-Source drivers which can be modified to get the most out of the hardware.
• Modifiable startup code, for as much system control as necessary.
• Supported by an active community that can help with problems and responds to bug reports.

Build Status

Status

While nxdk still is in early stages of development, it can already be used in many projects. Take a look at the list of projects that are build on top of nxdk. Additionally, the provided samples show how to use common features.

Getting Started

Prerequisites

You will need the following tools:

• GNU make
• clang
• GNU bison and flex
• lld
• Git
• CMake

OS-specific instructions for installing these prerequisites can be found in the Wiki

Download nxdk

git clone --recursive https://github.com/XboxDev/nxdk.git

Build Samples

Samples are easily built by running the activation script bin/activate, which will spawn a shell that is fully set up to start using nxdk, and then running the Makefile in one of the sample directories. Details can be found in the Wiki. nxdk also supports automatic creation of ISO files.

Next Steps

Copy one of the sample directories to get started. You can copy it anywhere you like. Run nxdk's activation script bin/activate, then, in the directory of your program, you can simply run make.

Credits

• OpenXDK is the inspiration for nxdk, and large parts of it have been reused. (License: MIT)
• Large parts of pbkit, by openxdkman, are included, with modifications. (License: MIT)
• A network stack is included based on lwIP (License: Modified BSD)
• A libc is included based on PDCLib (License: CC0)
• Large parts of the runtime library are derived from LLVM's compiler-rt library (License: MIT)
• vp20compiler is based on nvvertparse.c from Mesa (License: MIT)
• fp20compiler is based on nvparse from the NVIDIA SDK 9.52.
• The NVIDIA Cg compiler is bundled.
• extract-xiso developed by in et al. (License: BSD)

Code Overview

• lib/hal - Barebones Hardware Abstraction Layer for the Xbox, from OpenXDK.
• lib/net - Network stack for the Xbox based on lwIP.
• lib/pdclib - Xbox port of PDCLib, a CC0-licensed C standard library.
• lib/pbkit - A low level library for interfacing with the Xbox GPU.
• lib/sdl - Xbox ports of SDL2 and SDL_ttf.
• lib/usb - USB support from OpenXDK. Hacked together parts of an old Linux OHCI stack.
• lib/winapi - Xbox specific implementations of common useful WinAPI-functions.
• lib/xboxkrnl - Header and import library for interfacing with the Xbox kernel.
• lib/xboxrt - Miscellaneous functionality for debugging etc.
• tools/cxbe - Simple converter for PE executables to the Xbox executable format, from OpenXDK.
• tools/fp20compiler - Translates register combiner descriptions to Xbox pushbuffer commands.
• tools/vp20compiler - Translates vertex program assembly to Xbox microcode.
• tools/extract-xiso - Generates and extracts ISO images compatible with the Xbox (and XQEMU).
• samples/ - Sample applications to get started.

nxdk's Issues

pbkit license (AFL) is not compatible with GPL

This was recently brought to my attention - I was not aware.

This is indeed an issue we should solve (Although I honestly don't care too much as the authors probably wouldn't care). We should still reach out and try to figure something out.

openxdkman is the original author and also goes by the name ps2devman in the PS2 scene.
They also worked on xenkit for the 360.

Unfortunately, xbox-scene and ps2dev are offline, so we can't reach them there anymore.

---

@Maturion @caustik-divx / @caustik / @tmbinc any idea how to reach openxdkman?

I've also asked elsewhere, this reddit lists some things I've tried: https://www.reddit.com/r/originalxbox/comments/6mnr67/looking_for_openxdkman_author_of_pbkit/ and ps2dev IRC already.

clang might include host files

See #60 (comment).

I speculated that stdarg.h (where this problem was first noticed) might not be an actual file in the filesystem, but a virtual file (part of the compiler). @thrimbor responded:

We still might want to disable that behavior though, since clang assumes we compile for win32 and who knows what kind of crazy stuff could hide in those builtins

And I agree.

So we probably want to add -nobuiltininc, but we didn't verify that it's actually necessary.

Provide High-Level GPU abstraction

I think we should implement some form of freeglut + subset of OpenGL 1.x (with NV specific extensions).

I did consider using MESA but I think it might be better to keep things minimalistic, as the primary use-case will be unit-tests for Xbox emulation. This means an OpenGL implementation should be rather flexible and easy to understand.

We probably also don't care too much about error-checking or 100% correct behaviour of OpenGL either.

I've started prototyping some (insanely ugly) OpenGL here JayFoxRox#3
Most of our existing code is also close to OpenGL already and could easily be hacked to have simple applications running.

nxdk has no proper libc

• printf is absolutely horrible and doesn't respect most format specifiers (not Xbox specific).
  A better minimalistic alternative is even present in cromwell
• fopen and friends are missing (Xbox specific).
• I'd appreciate C11 thread compatibility too (Xbox specific).

I'd like to hear about peoples opinion which libc we should use or if we should implement our own.
I have never ported a libc and I'm not sure how C++ compatibility would play into this in the future.

I'm also not sure how a libc would handle the platform specific parts in contrast to the non-platform specific parts (such as the printf stuff)

However, this is an important issue which causes a lot of trouble in any Xbox related homebrew I have.

Incorrect AV pack constants lead to crash on PAL-Xbox

I found this while investigating why all the nxdk-samples make my Xbox (PAL, 1.0 I think since it used to have that GPU-fan) lock up.

I fired up my Debug-BIOS, which showed that somewhere in the code, MmAllocateContiguousMemoryEx was called with a size-parameter of zero, which triggers an assertion in the kernel. I managed to track down this allocation to line 3489 of pbkit.c. Further investigation showed that no videomode was set, and neither pb_init() nor XboxCRT() handle that case. I dug in the video-code and found out that no mode was set because all were rejected based on my AV pack - the constants go up to 5, my Xbox reports a 6. Removing that check (not a clean solution) resolved that for me, and the httpd-sample stopped crashing and worked nicely (didn't test others yet).

So the AV-pack constants that nxdk has seem to be incomplete. I managed to get my hands on a list of constants the kernel uses for them, which is the following:

#define AV_PACK_NONE                      0x00000000
#define AV_PACK_STANDARD                  0x00000001
#define AV_PACK_RFU                       0x00000002
#define AV_PACK_SCART                     0x00000003
#define AV_PACK_HDTV                      0x00000004
#define AV_PACK_VGA                       0x00000005
#define AV_PACK_SVIDEO                    0x00000006
#define AV_PACK_MAX                       0x00000007
#define AV_PACK_MASK                      0x000000FF

These differ a bit from the ones nxdk uses. Interestingly, these constants claim that my AV pack is SVIDEO, but it's a standard composite-one.

To resolve this, I'd suggest that pb_init() gets a check for a set video-mode. However, I'm not sure about what exactly to do about the constant-issue, which is the core-problem.

Audio API discussion

Consider this a room for discussion as we were starting to derail #13 with audio talk.
(Feel free to close this once the discussion has ended)

Currently, nxdk supports XAudio which was originally from OpenXDK.

However, we don't want that. Trust me.

You'll want to use the APU for audio and not just ACI (AC97). I'm currently documenting the APU VP and adding emulation to XQEMU.

XAudio uses AC97 only. The ACI / AC97 is very limited: You allocate a buffer, you put 16 bit signed PCM audio in it and tell the AC97 to play it back at 48kHz. You can't modify the data format, you can't control the samplearate and the CPU has to do lots of work.

The APU on the other hand, manages up to 256 audio voices and you can control the pitch [~ samplerate] and the data format (8 bit, 16 bit, 24 bit and ADPCM compressed audio). It not only supports buffered audio, but also streamed audio, hardware-audio filters, envelopes etc.
- it outputs that audio as (I think) 24 bit, always at 48kHz, 32 samples grouped at a time to the GP - no CPU intervention.
The audio is then processed by the GP which is a programmable DSP (See #12 ) and then forwarded to the EP (also programmable DSP) by programming a FIFO. The EP then has a FIFO to a single page of memory which is used by the ACI / AC97 as playback source.
This way there is no CPU intervention anywhere + we gain a ton of features and avoid stuff like software-resampling or filters. We only need some code for the GP and the EP. If we don't want dolby digital, we can just pass-through the EP. If we want dolby digital we'd have to write our own AC3 encoder or dump the Microsoft / Dolby one (which is easy, but we couldn't legaly include it with nxdk).

Writing a pass-through GP / EP driver is easy.
Programming the VP takes some work, but is considerably easier than writing a software mixer and more CPU friendly than using an existing software mixer.

---

@thrimbor suggested an OpenAL-implementation. To me, writing these kind of libs is not a priority to be honest, but be my guest.
The truth is that most users of nxdk don't intend to write / port much homebrew.
Focus is very much on having something to write smaller utilities or unit tests for the emulators.
However, I personally chose to embrace nxdk-rdt which is a homebrew app compiled using nxdk. It's similar to the XDK xbdm.dll concept and allows remote control of the Xbox.
This makes it far easier to write unit tests remotely, but also means all my drivers have moved away from C code, to being python scripts ran on a PC (and only memory access happens on Xbox).
So I'll write an APU VP driver in python for the unit tests anyway. I'm not really motivated to write another, more complicated driver in C afterwards probably, especially as I would have little use for it: OpenAL is too high level for further unit tests and I don't intend to write other hombrew at this point in time.

SDL: cant find matching render device.

Is sdl 2 not properly implemented, or is it my setup thats wrong?

SHADER_OBJS deps on mingw64

Compiling the 0ldskoo1 sample on mingw64 by simply typing make fails, as the vs.inl and ps.inl files are not compiled properly. Compiling them directly works. The deps are incorrect for the shaders.

Removing the nxdk Makefile's "/debug" flag seems to also fix compilation of other samples. Not sure this is related or not.

Fix networking packet size issue

Seems like packets of size greater than 128 bytes cause issues, making the network stack almost useless. It's probably not too complicated, just needs investigation.

XReboot() should not be part of the samples

@JayFoxRox suggested that XReboot() should not be a part of the nxdk samples, returning from main should be enough.
Changing the default behavior on returning from main might be necessary though.

SDL requires `_exit` function

SDL requires an _exit function.
This is a bit odd, and it looks like a calling convention mismatch for exit.

I don't know wether it should be an handler that handles exits, or, if it's a function that's being called to force an exit.

Our use in sample code is inconsistent, too:

In the sdl sample, it's used like an atexit handler, that also forces an exit through reboot itself:

nxdk/samples/sdl/main.c

Lines 21 to 28 in 192c770

	int _exit(int x)
	{
	debugPrint("_exit called\n");
	debugPrint( " SDL_Error: %s\n", SDL_GetError() );
	XSleep(5000);
	XReboot();
	return 0;
	}

(Also called by application code)

In the sdl_ttf sample, it's used like a dummy and no "exit" of any kind is performed:

nxdk/samples/sdl_ttf/main.c

Lines 12 to 14 in 192c770

	// SDL requires this symbol to exist
	void _exit() {
	}

(Unused by application code)

---

Someone should investigate, fix this behaviour (probably shouldn't require hacks in application code) and make the sample code consistent.

Missed items with 'make clean'

make clean doesn't remove .d, .lib, or .iso files as shown below.

mike@MJD1043:~/projects/nxdk/samples/hello$ ls
bin  hello.iso  main.c  main.c.d  main.exe  main.lib  main.obj  Makefile
mike@MJD1043:~/projects/nxdk/samples/hello$ make clean
mike@MJD1043:~/projects/nxdk/samples/hello$ ls
bin  hello.iso  main.c  main.c.d  main.lib  Makefile

Merging back into OpenXDK?

This is not really an issue, but I don't know any better way to contact you guys. Out of archival purposes, I've created a basic website for OpenXDK over at http://openxdk.sourceforge.net. I stumbled upon nxdk and added a link to this project.
If you're interested in merging your code back into the main OpenXDK code branch, I'd be glad to help you.

pbkit does not kill itself when application quits

Once pbkit is initialized, it's the users sole responsibility to unload it eventually.
If this does not happen, the GPU is left in a broken state.

I consider this a critical bug, because it might break tools like nxdk-rdt or dashboards made in nxdk.
Basically when you leave an nxdk app uncleanly, it will corrupt the state of the Xbox until a cold-reboot.
So this will not trigger problems in nxdk apps themselves, but applications ran much later.

---

I've prototyped a small change in nxdk-rdt to test this. I hope someone else implements it properly.

Provide this callback and notification routine:

static HAL_SHUTDOWN_REGISTRATION pb_shutdown_notification;

static NTAPI VOID pb_shutdown_notification_routine(PHAL_SHUTDOWN_REGISTRATION shutdown_notifiction) {
  pb_show_debug_screen();
  pb_kill();
}

(Maybe we want to do less or more there. I don't know)

Do this when pb_init is done, to register the above routine (D3D also uses priority 0):

pb_shutdown_notification.NotificationRoutine = pb_shutdown_notification_routine;
pb_shutdown_notification.Priority = 0;
HalRegisterShutdownNotification(&pb_shutdown_notification, TRUE);

We also have to unregister the notification if the user manually kills pbkit. I have not implemented this step in the prototype.

This also needs an update in our xboxkrnl.h to fix the calling convention of PHAL_SHUTDOWN_NOTIFICATION which should probably be an stdcall (handled by NTAPI macro).

DSP assembler / compiler

Should we include an assembler / compiler for DSP code?

I'm currently using a56 which can be found at http://www.zdomain.com/a56.html (GPL)
However, it's for the DSP56001 and not DSP563xx (which is presumably what's in the Xbox).
I'm too dumb + lazy to figure out the exact differences between those architectures. But I noticed that the inc d instruction does not exist in a56. The (mostly simple) a56 code I've written seems to work on Xbox.

There is also a commercial gcc port by Motorola which can be found here: http://chameleon.atlantes.org/
I'm not sure about the legality of this though, it's outdated and I haven't tried it myself.

I've also come accross some binary assemblers (legality unclear) for DOS which supposedly run in dosbox.

https://www.rockbox.org/wiki/MotorolaDSP56k#dsp56k_45gcc also has an outdated overview of related tools.

---

Personally, I'm in favor of hosting a56 on the xqemu organization account and submoduling it for nxdk.
We can then consider writing our own tools later or improve a56 for the Xbox DSPs (hopefully using the same instruction table as in the XQEMU DSP emulation).

I already have a small (dirty) DSP assembler example at https://github.com/JayFoxRox/xbox-tools/blob/master/python-scripts/dsp_homebrew.py - I also have some python code to run a56 and parse its output in https://github.com/JayFoxRox/xbox-tools/blob/master/python-scripts/xbox/dsp.py#L41
Adding a56 to NXDK would be beneficial as we could locate a56 by the NXDK_DIR environment variable.

Vertex-Shader c[..] constant addressing convention

Should constant indexing work from 0 to 191 or from -96 to +95?
D3D uses -96 to +95, whereas OpenGL assumes 0 to 191.

The reasoning behind the weird addressing is that there are 2 areas of registers.
One is used for programmable vertex shaders only, the other one is also re-used for the fixed function pipeline.
To my knowledge the GPU also has flags to turn on/off usage of the second bank for vertex shaders.

So in order to make a good decision, we have to figure out how registers are used and how they are addressed in different GPU modes. Another influence should be the NV vertex program GL extension probably.

I personally favored the 0 - 191 addressing, but I believe that would result in 0 - 95 being unusable in one GPU mode. It's probably confusing if constant registers start at index 96.
Manually flipping the banks in the address calculation is probably also not possible because of relative addressing (but that needs hardware testing).

This also affects other uses of c[..] constant addressing, such as readback of constants via RDI (which is why I originally wanted an index starting at 0; so I could use the same index in shaders and the RDI readback code)

Nxdk fails to link __chkstk when allocating a large number of local variables on the stack

When trying to create a large number of local variables on the stack in a function, nxdk will fail to link because it is trying to use the __chkstk symbol.

lld-link: error: /mnt/e/Windows/Programming/nxdk/my_progs/xbox_kernel_test_suite/rtl_tests.obj: undefined symbol: __chkstk
/mnt/e/Windows/Programming/nxdk/my_progs/xbox_kernel_test_suite/../../Makefile:94: recipe for target 'main.exe' failed
make: *** [main.exe] Error 1

Code that causes this:

void foo() {
char buffer[2000];
}

Please enable the wiki

The GitHub wiki is useful for sharing development tips, notes, and guides specific to working with nxdk.

cxbe is incompatible with default installations on debug Xboxes

VerifierX (in dxt/vx.dxt) is incompatible with applications generated by cxbe.

The issue is that there's no libraries listed in our XBEs and this trips VerifierX.

I've sent a PR to https://github.com/OpenXDK/cxbe/pull/1 some time ago, but we don't even use that version of cxbe.

Someone should clean up my patch and apply it to nxdk.

Setup CI build system

Something like Travis CI would be great to have to test the build on PR. Maybe AppVeyor for Windows nightly build downloads.

`strdup` not implemented

I consider strdup a rather basic libc function, as it's present in standard header "string.h" for POSIX code. It's also part of later ISO-C++: https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/strdup-wcsdup-mbsdup?view=vs-2019

What's the best way to get access to these functions?

• Get them into pdclib?
• Wrap pdclib so we can add our own functionality?
• Replace pdclib?

It's getting very frustrating to keep a fixes.h around to implement this missing standard functionality.

Copy over additions to OpenXDK (as done in a Cxbx-Reloaded import)

In Cxbx-Reloaded, a lot of additions were made to the import of OpenXDK.

These additions were never 'upstreamed' back to any OpenXDK repository, so only Cxbx-Reloaded is currently benefiting from this work.

For this issue, determine which additions are usefull for nxdk and copy those over.

Here a list of commits that touch the OpenXDK import in Cxbx-Reloaded :
https://github.com/Cxbx-Reloaded/Cxbx-Reloaded/commits/develop/import/OpenXDK

(Note, that Cxbx-Reloaded specific commits date back to Aug 18, 2016. Anything older than that, seems to originate from a OpenXDK repository imported into Cxbx-Reloaded, or it's Cxbx-predecessor.)

Test Issue - Ignore

Update kernel imports

The kernel imports are not complete. This should be changed.
Unfortunately it's also not possible to easily generate a new libxboxkrnl.lib (I believe I ran into some issues with the existing one a while back, but I don't remember the specifics currently)

Regarding the headers / signatures and datatypes, @PatrickvL said:

Almost all kernel signatures have been added to the OpenXDK copy in Cxbx-Reloaded, here : https://github.com/Cxbx-Reloaded/Cxbx-Reloaded/tree/master/import/OpenXDK/include/xboxkrnl

I don't have the time to contribute to either OpenXDK nor NXDK, but you people shouldn't let a resource like this go to waste! ;)

However, I looked into this after naively moving the code to nxdk:

• There is some incompatibility as to where certain types are declared (but this is easily fixed).
• There is also an issue with the datatypes (structs) only being correctly put in memory for MSVC. clang / gcc use a different packing, so explicit padding fields and explicit packing has to be used. Even mingw does not pack some structures correctly I believe.
• Some signatures and data types are very obviously copied from copyrighted code. There are legal means to figure out these signatures, so we should use them, instead of just copying them around. While I think it's perfectly fine to copy the function signatures, at least whitespaces and comments should be removed / changed.

Eitherway, we should update our signatures and datatypes.
Contributions welcome.

SDL submodule has moved, update URL

make[1]: lib.exe: file or folder not found

Duplicate/Reopening of #38.
When trying to build through MSYS2 on Windows 10, I get a linker related error stating that lib.exe cannot be found.

Installed packages:

Click to expand

llvm-related:

Click to expand

local/clang-svn 60106.1d5b05f-1
local/lld-svn 4595.3511ec1-1
local/llvm-svn 124592.2aebced-1

gcc-related:

Click to expand

local/gcc 7.3.0-3 (msys2-devel)
local/gcc-libs 7.3.0-3 (base)
local/mingw-w64-x86_64-gcc 8.2.0-1 (mingw-w64-x86_64-toolchain)
local/mingw-w64-x86_64-gcc-ada 8.2.0-1 (mingw-w64-x86_64-toolchain)
local/mingw-w64-x86_64-gcc-fortran 8.2.0-1 (mingw-w64-x86_64-toolchain)
local/mingw-w64-x86_64-gcc-libgfortran 8.2.0-1 (mingw-w64-x86_64-toolchain)
local/mingw-w64-x86_64-gcc-libs 8.2.0-1 (mingw-w64-x86_64-toolchain)
local/mingw-w64-x86_64-gcc-objc 8.2.0-1 (mingw-w64-x86_64-toolchain)

flex-related:

Click to expand

local/flex 2.6.4-1 (base base-devel)
local/libpipeline 1.5.0-1 (libraries)

bison-related:

Click to expand

local/bison 3.0.5-1 (base-devel)

git-related:

Click to expand

local/git 2.19.0-1 (VCS)
local/mingw-w64-x86_64-crt-git 6.0.0.5225.fb06a4bf-1 (mingw-w64-x86_64-toolchain)
local/mingw-w64-x86_64-headers-git 6.0.0.5225.fb06a4bf-1 (mingw-w64-x86_64-toolchain)
local/mingw-w64-x86_64-libmangle-git 6.0.0.5173.49ca7cfe-1 (mingw-w64-x86_64-toolchain)
local/mingw-w64-x86_64-libtre-git r128.6fb7206-2
local/mingw-w64-x86_64-libwinpthread-git 6.0.0.5203.9afaf1c2-1 (mingw-w64-x86_64-toolchain)
local/mingw-w64-x86_64-tools-git 6.0.0.5173.49ca7cfe-1 (mingw-w64-x86_64-toolchain)
local/mingw-w64-x86_64-winpthreads-git 6.0.0.5203.9afaf1c2-1 (mingw-w64-x86_64-toolchain)
local/mingw-w64-x86_64-winstorecompat-git 6.0.0.5173.49ca7cfe-1 (mingw-w64-x86_64-toolchain)
local/msys2-launcher-git 0.3.32.56c2ba7-2 (base)
local/pactoys-git r2.07ca37f-1 (base base-devel)
local/pax-git 20161104.2-1 (base)

Lack of wprintf in xlibc makes unicode string debug diffcult

I am unsure if this should go in the blanket xlibc issue or have its own. Let me know if this should go in the blanket issue.

I'm working on making the unicode string tests in the xbox_kernel_test_suite better and not being able to print unicode strings with printf makes the debug difficult. wprintf does not exist, and '%ls' is not a valid format string for printf. I did some research today to see how I could possibly implement this functionality, but glibc's implementation of printf is confusing so I didn't make much progress.

My workaround solution has been to print the hex value of each WCHAR character. It works but it isn't the most pretty.

pbkit surface addresses controlled using base

It's not necessarily a problem, but I'd like to have on record that the framebuffer is configured using the surface offset in D3D (base 0), but pbkit uses the base to address the surface (offset 0).

This caused issues in nv2a-trace fo me (which I'll address in the future). I'd recommend to switch to D3D behaviour - there might be a reason why this was done.

'triangle' sample gives GPU Error at current state

I don't know anything about programming the Xbox's GPU at the moment, but here is the error I am getting when running the output default.xbe:
(Keep in mind I had to hand copy this by hand as I do not have the proper means for debug log copy and paste here.)

GPU Error : object state invalid error!
Error binary flags : 08x
Channel ID : 2048 (0=3D)
Channel class : 0
Push buffer inner register target : 04x
Push buffer data (lo) or instance : 08x
Push buffer data (hi) or instance : 08x
Multi-purpose register A [0x1D8C] : 08x
Multi-purpose register A [0x1D90] : 08x

Report is accurate only if pb_trace_mode=1 (slower)
System halted

Not sure how to enable pb_trace_mode=1 though...

`setjmp` / `longjmp` are not provided

I've looked into this before using builtins, but could not make it work.

Having setjmp and longjmp would be useful for micropython and other projects which depend on things like coroutines.

Related to #13.

lwip: Use BSD socket API

The existing code uses the lwip netbuf interface, however, for compatibility with existing code and ease of use, we should rewrite the httpd example to use the BSD style socket API.

We should also make sure that the BSD style API works flawless.
nxdk-rdt has a lot of issues with the netbuf API, however, it is still unclear where exactly these issues stem from (be it nxdk-rdt, lwip or the Xbox specific network driver). With the BSD API we could test behaviour on other platforms and we will not have to worry about splitting or joining the network packets manually like we currently do.

Finding files using wildcards can lead to unexpected behaviour

It already came up during review of #60: we should stop using wildcards in our Makefiles. Both, for nxdk toolchain files, and in our sample code.

setjmp / longjmp does not preserve the FPU state

When merging #60, we weren't sure wether this is necessary or not (see last paragraphs in said PR). This was discussed on Discord, and we found different infomation:

@mborgerson said:

https://lists.freebsd.org/pipermail/freebsd-amd64/2008-June/011284.html
google gave me that
which says c99 says fpu should be preserved

@thrimbor said:

Quote from the C11 standard: "It does not include the state of the floating-point status flags, of open files, or of any other component of the abstract machine."

So we should probably run some tests wether software actually depends on this, and if other implementations do this.

Samples pb_init() is ugly

Most samples do a switch case for pb_init() similar to

switch(pb_init()) {
  case 0: break;
  default:
    XSleep(2000);
    XReboot();
    return -1;
}

This is not pleasing on the eyes.
@JayFoxRox suggested a more standard

int ret = pb_init();
if (ret != 0) {
  XSleep(2000);
  XReboot();
  return -1;
}

which looks nicer IMO. This is also the way most other libs I've worked with suggests initialization checking to be done.

lld-link: Command not found (Ubuntu 18.04)

Installed executable name is lld-link-6.0. We can:

• Change to the -6.0 name, or
• Detect which version to use, or
• Ask user to update Makefile or create a symlink

fp20compiler: Split by shader-magic ("!!") instead of comment

fp20compiler currently splits 2 shaders (Texture Shaders + Register Combiners) in the same file by looking for a comment (that was placed by cg):

nxdk/tools/fp20compiler/main.cpp

Line 44 in 82e4860

char* ts_end = strstr(ts_start, "// End of program");

Instead, we should be searching for each !! in the file, and then ensure it's either at offset 0 in the string (!! in first line), or it's following a \n (\n!!, so !! at start of any other line).

We should iterate over each segment started by a magic, so we can translate each shader and report errors for unknown types.

That way, we'd make it easier for users to write their own shaders by hand, without having to remember cryptic comments.

Note: I did not check if the OpenGL spec has a better suggestion for keeping different programs in the same string. That might impact this design.

lld is not included in msys package list

This is about the broken instructions in the readme for the Windows / msys build.

We also need the package "lld" (which aliases to "lld-svn"). Otherwise lld-link will not be found when running make.
(I'm also not sure if we still need the "llvm" package.)

However, even after installing "ldd" it will trigger issue #38 .
So possibly something else is (or already has been) messed up. @LukeUsher wrote the original documentation for Windows. So maybe he can shed some light on this?

I also wasn't sure which shell to use for running make: MSYS / MinGW 32 bit?

Add Dockerfile for easy builds

Using containers for builds will help standardize the toolchain and build process across platforms. Let's create a basic Dockerfile that can be used as a baseline ("it works on my machine").

No support for AV_PACK_HDTV in combination with VIDEO_REGION_PAL

While XDK code works fine with the component cable when the Xbox is configured to PAL, the screen remains black (or Xbox even crashes?) with the same combination for nxdk.

The code implementing this is here:

nxdk/lib/hal/video.c

Line 58 in d1c2ff3

VIDEO_MODE_SETTING vidModes[] =

The code does list modes for VIDEO_REGION_NTSCM and VIDEO_REGION_NTSCJ in combination with AV_PACK_HDTV. However, a mode for VIDEO_REGION_PAL is missing.
We should also review other adapters and add fallback modes if possible.

༼ つ ◕_◕ ༽つ GIVE BUILDS ༼ つ ◕_◕ ༽つ

fp20compiler: Review support for DX8 shaders

There's existing code in fp20compiler to support Ps1.0 and Ps1.1 shaders, but it seems incomplete. This is even prototyped in main.cpp.

It should be fixed or removed.

If it is implemented, this would have to be separated from the OpenGL functionality (2 tools, or a command line switch).

0ldskoo1 sample is confusing (red rectangle)

The red rectangle is intended in the original code, but it's confusing users. Especially as we recommend 0ldskoo1 as the first sample to try.

Instead, we should hide the red box by default in the nxdk version.
Ideally we'd also change the message for the nxdk release.

vp20compiler: Confirm constant c[..] writes work correctly (VSPs)

Confirm if index for constant writes in VSPs is correct.
Also make sure that constant writes don't work for non VSPs (unless this works in hardware).

We also have to test if relative indexing through A0.x works for the following cases:

• indexed write
• indexed read and indexed write in same instruction

If this could affect combined instructions, it should also be checked (but I'm not sure if vp20compiler even supports combining. If not, that might as well be a seperate issue for the future).

xboxrt: Legacy wcsncpy and strncpy functions do not fill exactly n bytes

Legacy wcsncpy (and strncpy) functions in xboxrt do not match the definition of wcsncpy. Specifically, we do not fill exactly n bytes of data. A similar issue exists with strncpy. See the man page for more information:

Exactly n wide characters are written at dest. If the length wcslen(src) is smaller than n, the remaining wide characters in the array pointed to by dest are filled with null wide characters.

From @JayFoxRox
We currently don't fill exactly n bytes (and neither could we, because our users don't respect that - see wcscpy which calls with SIZE_MAX)

Edited by JayFoxRox: wcspy should not be affected by most of this (except that it's using wcsncpy poorly); However, strncpy is also affected by the exact same issue (in a different part of the code).

lib.exe where?

unable to find lib.exe in PATH: : No such file or directory happens on everything I try to compile, and I am super confused and frustrated that this issue seems very unique in my case.

I am running Windows 10 x64 with MSYS2 32bit.

Avoid duplication of documentation

Currently, a lot of the infomation is duplicated between README.md and the Wiki.

I think we should simply link the wiki from the README, and keep our README more simple.

The wiki encourages documentation contributions, but we don't have to waste time reviewing it.
The README will also feel like a wall-of-text, whereas the wiki can be used to structure content.

Sample Makefiles don't contain GEN_XISO line

We'd probably want to add a GEN_XISO = $(XBE_TITLE).iso line to every sample Makefile as some have them and some don't - consistency is nice.

Remove reboot on triangle and mesh samples

I added it for convenience (so I wouldn't have to reach over and restart my Xbox), but most people seem to perceive it as a crash since they haven't read the source code--so it should probably just be removed.

Adding NXDK to Visual Studio?

Hey there! I want to test out some Homebrew Developing with NXDK, so how can i add it to Visual Studio? A fast answer would be great!

vp20compiler: Confirm R12 still works correctly

#16 changed some internals around which might have broken R12 (did it even work correctly before?)

R12 is a shadow of the o[POS] output register.
This is done by nvidia because output registers o[..] are not readable, but only writable.

• We should check if R12 is writeable in our implementation, wether it is writeable in hardware.
• We should check if R12 is readable in our implementation, wether it is readable in hardware.
• We should check if R12 can be read or written when using oPos as an output register, for all kind of instructions, including combined MAC / ILU operations (if that's possible)