nicoboss / nsz Goto Github PK

When doing the conversion from nsp to nsz, it sometimes tells that some A.nsp has the same title id with B.nsz which is not the case. By using option -i to see the title ids of these files, they have different title ids. Is this abnormal or I misunderstood something ?

Some block compressed games infinitely decompresses/verify.

How to reproduce: Block compress Noire_0100830004FB6800_[v131072] and then just decompress it. After the process bar finishes it will be just stuck while still doing something in the background what never seems to end. This issue was first reported on https://gbatemp.net/threads/ryjin-a-nsz-converter-mod.550174/page-7 by GnK23 using a 500 MB game and so definitely something else then Noire. Unlike GnK23 I was able to reproduce this issue in booth 2.0 and latest commit.

If you noticed any other game affected by this issue please let me know.

Nuitka builds crash on block compresson:

python.exe: error: unrecognized arguments: --multiprocessing-fork
nut exception: 2
Error when compressing file: C:\Users\Administrator\Documents\NSZ\nsz\nsz.dist\SNES_Online_01008D300C50C000_v0_original.nsp
Traceback (most recent call last):
  File "C:\Users\Administrator\Documents\NSZ\nsz\nsz.dist\nsz\BlockCompressor.py", line 85, in blockCompress
  File "C:\Users\Administrator\Documents\NSZ\nsz\nsz.dist\multiprocessing\context.py", line 56, in Manager
  File "C:\Users\Administrator\Documents\NSZ\nsz\nsz.dist\multiprocessing\managers.py", line 567, in start
  File "C:\Users\Administrator\Documents\NSZ\nsz\nsz.dist\multiprocessing\connection.py", line 250, in recv
  File "C:\Users\Administrator\Documents\NSZ\nsz\nsz.dist\multiprocessing\connection.py", line 306, in _recv_bytes

decompressedSize is not defined in this scope, what is this trying to access

Sometimes, the progress bar shows 100%, but still keeps processing in the background. I believe it has something to do with TQDM incorrectly reporting the size processed. Though I haven't looked into the code. Will provide video when I compress my next NSPs to show bug in action.

read returned empty 0x700a828fb
nut exception: read returned empty 0x700a828fb
Traceback (most recent call last):
File "D:\Descargas\SWITCH\nsz-master\compressor\nsz.py", line 78, in
nut.compress(filePath, 17 if args.level is None else args.level, args.output)
File "D:\Descargas\SWITCH\nsz-master\compressor\nut_init_.py", line 46, in compress
container.open(filePath, 'rb')
File "D:\Descargas\SWITCH\nsz-master\compressor\Fs\Nsp.py", line 205, in open
super(Nsp, self).open(path or self.path, mode, cryptoType, cryptoKey, cryptoCounter)
File "D:\Descargas\SWITCH\nsz-master\compressor\Fs\Pfs0.py", line 177, in open
self.files[i].open(None, None)
File "D:\Descargas\SWITCH\nsz-master\compressor\Fs\Nca.py", line 215, in open
self.partition(0x0, 0xC00, self.header, Fs.Type.Crypto.XTS, uhx(Keys.get('header_key')))
File "D:\Descargas\SWITCH\nsz-master\compressor\Fs\File.py", line 69, in partition
n.open(None, None, cryptoType, cryptoKey, cryptoCounter)
File "D:\Descargas\SWITCH\nsz-master\compressor\Fs\Nca.py", line 70, in open
self.signature1 = self.read(0x100)
File "D:\Descargas\SWITCH\nsz-master\compressor\Fs\File.py", line 325, in read
self._buffer = super(BufferedFile, self).read(pageReadSize)
File "D:\Descargas\SWITCH\nsz-master\compressor\Fs\File.py", line 86, in read
return self.f.read(size)
File "D:\Descargas\SWITCH\nsz-master\compressor\Fs\File.py", line 328, in read
raise IOError('read returned empty ' + hex(self.offset))
OSError: read returned empty 0x700a828fb

Unhandled exception at 0x00007FF965CA092A (python38.dll) in python.exe: 0xC0000005: Access violation reading location 0x000000000000000B. occurred

It seems as if it cannot find python38.pdb

I have installed Python 3.8.* and Visual studio

This is happening when installing nsz and also running the requirements.txt

Windows 10 - I do python coding and others

Like 'nsz' cli command.
Would be 10 times easier to install and use.
Or this script is not intended to be used like that?

For example

without giving -o switch

Block compressing (level 17) G:\Switch\nsz\Mercenaries Saga Chronicles\Mercenaries Saga Chronicles [0100273008FBC000][v0].nsp -> G:\Switch\nsz\Mercenaries Saga ChroniclesMercenaries Saga Chronicles [0100273008FBC000][v0].nsz

with -o "%cd%"
Block compressing (level 17) G:\Switch\nsz\Mercenaries Saga Chronicles\Mercenaries Saga Chronicles [0100273008FBC800][v65536].nsp -> G:\Switch\nsz\Mercenaries Saga ChroniclesMercenaries Saga Chronicles [0100273008FBC800][v65536].nsz

Could be broken due to recent fix in the output path

Readme says that -t or --threads can be used as an optional arguments but I get the error that they are unrecognitzed.

There is also no parser.add_argument('-t','--thread', ............) line in nsz.py or any multithread argument in nut.compress

Cheers and thanks for the awesome work.

When compressing some files ( I found two such files ) nsz.py gives an exception..

While I'm not sure why this happens ( partNr is increased but there is nothing left to read? ) this causes compressing an entire folder to stop. I made some dirty modifications to stop raising this error and delete corrupted destination file so it will continue to process the folder, but you could find the root cause of this problem I think.

Here is the exception:

Adding file 7dfaaf90125b66824e14151d5744462f.cnmt.nca 4096 bytes to NSP
Adding file 5243655b68ed240487ab2977f3464d1b.ncz 252821504 bytes to NSP
  0%|                                                                                                                                                                         | 53.3k/253M [00:00<00:00, 568MB/s/s]
list index out of range
nut exception: list index out of range
Traceback (most recent call last):
  File "nsz.py", line 100, in <module>
    nut.compress(filePath, 17 if args.level is None else args.level, args.block, args.bs, args.output, args.threads, args.overwrite)
  File "/usr/home/media/mydev/nsz/compressor/nut/__init__.py", line 11, in compress
    SolidCompressor.solidCompress(filePath, compressionLevel, outputDir, threads, overwrite)
  File "/usr/home/media/mydev/nsz/compressor/nut/SolidCompressor.py", line 101, in solidCompress
    buffer = partitions[partNr].read(CHUNK_SZ)
IndexError: list index out of range```

Decompress.py does not run correctly, reporting that import is not a command on LM 19.3 cinnamon with all dependencies satisfied.

Latest DLCs with key_area_key_application_0a fails to compress with the latest release (not sure about master) of nsz.
If you need a DLC with key_area_key_application_0a, contact me on Discord. (knightslayer#9443)

Hi,

When decompressing an nsz, the nsp result file is different from the source nsp file.
Also these result files are not compressed by nsz.py, it produces an nsz file but about the same size as the nsp. I've noticed it's only listing nca in the output and do not execute any kind of compression and producing ncz.

file.nsp ---compress---> compressed_file.nsz ---decompress---> file_not_equal_to_src_nsp.nsp ---compress---> not_compressed_file.nsz

What kind of information could be lost in the compression or maybe decompression?
It's quite an issue to produce nsp files that are not equivalent to the source nsp for data preservation.

On last master commit, also tried with tag 1.0.

Thanks.

im trying to get my gdrive to mount with tinfoil. so far i can only open my gdrive with nutfs.
when adding gdrive to file browser, its empty. i have no path set.
when using nutfs. i added the nutfs path of my roms (nutfs://192.111.1.11:9000/gdrive/mydrive/roms)
inside the roms folder is a snes folder and inside the snes folder is roms, stored in zips.
do i have to extract all the rom zips to get tinfoil to scan them in?

Hi,

I was successful decompressing/compressing a few files, but one of them is giving me a weird error.

Here's the command I use:

./nsz.py -C "file.nsp" -l 22

Please find the output below:

compressing (level 22) path/nsz/compressor/Valfaris2 [010089700F30C000].nsp -> path/nsz/compressor/file.nsz
Adding file 010089700f30c0000000000000000008.cert 1792 bytes to NSP
Adding file 010089700f30c0000000000000000008.tik 704 bytes to NSP
Adding file 522e1cc2378cf300222c17d3cdfc7fa6.ncz 843907072 bytes to NSP
list index out of range
nut exception: list index out of range
0%| | 0.00/844M [00:00<?, ?B/s/s] 0%| | 51.7k/844M [00:00<00:03, 253MB/s/s]
Traceback (most recent call last):
File "./nsz.py", line 98, in
nut.compress(filePath, 17 if args.level is None else args.level, args.solid, args.bs, args.output, args.threads)
File "path/nsz/compressor/nut/init.py", line 162, in compress
buffer = partitions[partNr].read(CHUNK_SZ)
IndexError: list index out of range

The same command seems to work with my other nsps, but it fails here.
Thanks.

I feel that it would be useful to add support to decompress.py for decompressing directly from nsz to nsp

I'm using master brach 5b3afac and I have hash misssmatch on some files after being compressed and verified with the following command:

python nsz.py -C -l 22 -V -w -o "dir" "file.nsp"

On some files I get:

Solid compressing (level 22)
[VERIFY NSZ] [010085700abc8800][v65536].nsz
[EXISTS] 758d1bc445e70398d5c44fa7abf4a1a7.cnmt.xml
[CORRUPTED] 233c4b75c9b4192394b3d833cfa704de.ncz
nut exception: Verification detected hash missmatch

[VERIFY NSZ] [0100EA80032EA000][v0].nsz
[CORRUPTED] dc10021f535d07b3daf50908c462d7d5.nca
nut exception: Verification detected hash missmatch!

And I have to comment line 46 on nsz/__init__.py: raise
because the program hangs waiting for something and never ends, I had to press Ctrl+C which is bad on scripts.

Why is happenning the hash missmatch and how to fix this?

ERROR: No matching distribution found for kivy_deps.sdl2==0.1.22 (from -r requirements.txt (line 4))

I'm getting this in my distro (Clear Linux, which admittedly doesn't have the best repo), is there any way to get around this dep?
Trying on 3.1.0 btw.

When I try to pass in the name of an NCZ file (after separately extracting all files within the NSZ with a different tool), the app finishes instantly and does nothing. I assume that decompressing individual NCZ files is not implemented, is that correct?

If this is intended behavior (or an oversight), then consider this a feature request. I really want this functionality, and it seems like it would be really quick to implement.

NSZ works just fine if I want to extract and repack the entire .nsz file, but doing it like this would require some significant changes to the custom tools and code I'm already using. Decompressing a single NCZ, however, was a relatively simple thing to insert into my existing toolchain, and I'd hoped that your tool would do it.

If this is already possible, please let me know how. Currently, passing in an NCZ file with the following command line does nothing:
./nsz -D --output 'extracted\Nsz\Dir' 'extracted\Nsz\ Dir\fc255efbda053ecf0ccc1f071a4f9e86.ncz'
This is the only output, and no file:

       NSZ v2.1.1   ,;:;;,
                   ;;;;;
           .=',    ;:;;:,
          /_', "=. ';:;:;
          @=:__,  \,;:;:'
            _(\.=  ;:;;'
           `"_(  _/="`
            `"'

Done!

Overzealous verify check may be flagging up normal NCA files, or it may just be that some files always show up as corrupted. Used the commit from about 2 days ago.

Hey guys.

Thx a lot again for adding xcz support.
Would like to compress my backups and got the following error:

master_key_09 missing from keys.txt
Traceback (most recent call last):
File "nsz\nut\Keys.py", line 107, in load
OSError: master_key_09 missing from keys.txt

master_key_09 missing from keys.txt
Compressing 13%|████████ | 377/2953 MiB [00:42<04:47, > 8.97 MiB/s]

But the compression is going on (percentage is increasing)?

Do i have to get my keys.txt again?
Any hints?

thx a lot

#58 suggests that the pip package is supposed to be CLI only with the GUI requiring nszgui package to be installed. But trying to install nsz from pip still attempts to install the GUI also.

Let’s implement a function returning string(titleid) and int(version) from any NSP/NSZ file path.
If that data can be found in the filename it will be obtained using regex, if not the files metadata will be phrased.

Inside nsz.py:
Let's create a dictionary of all filenames in the destination folder.
Let's also create a dictionary containing dict(titleid, dict(version, filename))

Inside FileExistingChecks.py:

1. Let's obtain titled and version from the current file we want to check
2. If --rm-old-version is enabled let's delete all older versions of it and remove the deleted files from booth dictionaries
3. let's check if a file with the same version and titleid already exists.
   • If --overwrite is enabled let's delete it and remove the deleted files from booth dictionaries
4. If there's still a file with the exact same filename inside the filelist dictionary
   • If --overwrite is enabled let's delete it and remove the deleted files from booth dictionaries
   • Otherwise let's skip the compression and notify the user.

Extract TitleID/Version: Donkey Kong Country Tropical Freeze [0100C1F0051B6000][v0].nsp => 0100C1F0051B6000 0
Error when decompressing file: D:\Switch\Games\Donkey Kong Country Tropical Freeze [0100C1F0051B6000]\Donkey Kong Country Tropical Freeze [0100C1F0051B6000][v0].nsz
Traceback (most recent call last):
  File "D:\Switch\nsz-master\nsz\__init__.py", line 156, in main
    if not FileExistingChecks.AllowedToWriteOutfile(filePath, ".nsp", targetDict, args.rm_old_version, args.overwrite, args.parseCnmt):
  File "D:\Switch\nsz-master\nsz\FileExistingChecks.py", line 109, in AllowedToWriteOutfile
    if versionEntry < titleIDEntry:
TypeError: '<' not supported between instances of 'int' and 'dict'
Errors:
Error when processing D:\Switch\Games\Donkey Kong Country Tropical Freeze [0100C1F0051B6000]\Donkey Kong Country Tropical Freeze [0100C1F0051B6000][v0].nsz
Traceback (most recent call last):
  File "D:\Switch\nsz-master\nsz\__init__.py", line 156, in main
    if not FileExistingChecks.AllowedToWriteOutfile(filePath, ".nsp", targetDict, args.rm_old_version, args.overwrite, args.parseCnmt):
  File "D:\Switch\nsz-master\nsz\FileExistingChecks.py", line 109, in AllowedToWriteOutfile
    if versionEntry < titleIDEntry:
TypeError: '<' not supported between instances of 'int' and 'dict'

Let me know what other information I can give. Using the master tree from after the 2.1 release was made.

             NSZ v3.1   ,;:;;,
                       ;;;;;
               .=',    ;:;;:,
              /_', "=. ';:;:;
              @=:__,  \,;:;:'
                _(\.=  ;:;;'
               `"_(  _/="`
                `"'

Solid compressing (level 18) D:\downloads\switch\01006F8002326000.nsp -> D:\SWITCHTOOLS\nsz\01006F8002326000.nsz
[ADDING]     83d23343779ecd95e7355c39e931e24a.cnmt.nca 3584 bytes to NSP
not packed!
[ADDING]     80cbc793d37b6f493ba53561106bb5c6.nca 6692945920 bytes to NSP
Traceback (most recent call last):
  File "nsz\SolidCompressor.py", line 128, in solidCompressNsp
  File "nsz\SolidCompressor.py", line 114, in processContainer
  File "nsz\Fs\File.py", line 331, in read
OSError: read returned empty 0xffee5010

Done!

then the console hangs at:
Compressing 0%| | 0/100 MiB [00:07<?, 0.00 MiB/s]

i have:

• nsz-2.1.1 unrared as a folder in c:\nsw\NSZ
• prod key in the right place
• latest python installed,
• running from a windows 10 cmd (admin) console
• the console is in e.g. C:\NSW\NSZ

but "pip install nsz" gives LOTs of red lines: eg

Collecting nsz
  Using cached https://files.pythonhosted.org/packages/14/23/0eb200d2052203e94ffc311829b4df82db43e6660cb0f4ca7fedbcdba4d0/nsz-2.1.1-py3-none-any.whl
Collecting colorama
  Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl
Collecting pycryptodome==3.9.0
  Using cached https://files.pythonhosted.org/packages/b6/41/d3749c0caa860041eb6b0832c7087253d59cb5af8bb303a8d4b6daa74014/pycryptodome-3.9.0.tar.gz
    ERROR: Command errored out with exit status 1:

it also says "or just use the windows portable build."
havent a clue where this is... is it an exe?

sorry to use this as a forum to help, discord was a waste of time...

Windows 10,
tested build
nsz_v2.1.1_win64_portable.zip
nsz_nuitka_ab76c8e_portable.zip

parameters:
block compressing, default size, level 18, -w -V --rm-source

solid compressing with level 22 seems does the same thing

Problem with files:
[01003BC0000A0800][v2031616].nsp
[0100152000022800][v589824].nsp

Working fine 2.1.1
Broken at @ b3ad221

Error when compressing file: L:\AASwitch\Some rom . U 测试\a game rom. 测试_[v0].nsp
Traceback (most recent call last):
File "D:\git\nsz\nsz_init_.py", line 120, in main
compress(filePath, outfolder, args)
File "D:\git\nsz\nsz_init_.py", line 31, in compress
outFile = solidCompress(filePath, compressionLevel, outputDir, threadsToUse)
File "D:\git\nsz\nsz\SolidCompressor.py", line 15, in solidCompress
if filePath.endswith('.nsp'):
AttributeError: 'WindowsPath' object has no attribute 'endswith'

Summary of errors which occurred while processing files:
Error when processing L:\AASwitch\Some rom . U 测试\a game rom. 测试_[v0].nsp
Traceback (most recent call last):
File "D:\git\nsz\nsz_init_.py", line 120, in main
compress(filePath, outfolder, args)
File "D:\git\nsz\nsz_init_.py", line 31, in compress
outFile = solidCompress(filePath, compressionLevel, outputDir, threadsToUse)
File "D:\git\nsz\nsz\SolidCompressor.py", line 15, in solidCompress
if filePath.endswith('.nsp'):
AttributeError: 'WindowsPath' object has no attribute 'endswith'

Done!

I see that tqdm gets installed when running requirements.txt, so I was wondering if I should be seeing a progress bar when starting and during the compression stage. Or is it there in an If needed basis by piping it when running the command? I have no problem waiting for it to finish, but a progress bar would be nice to see.

Hey.

Me again :-)
When i'll add two XCI files from icon view, the second is displayed wrong, wee picture.
But when i do some action on it (comp. / decomp.) the seems to be processed correctly.
Seems its just an cosmetic bug.

macOS Catalina
Python environment 3.7.3 via pyenv
Installed nsz via pip container (current version 2.1)

"nsz -C -l 22 -t 3 Game.nsp"

I've tried this with multiple NSP files. One thing to denote that these NSP files were once XCI dumps converted using 4nxci. They all work as they are currently installed on my Switch SD card.

2.1.1

For example, if there's exception during processing:

Traceback (most recent call last):
File "D:\nsz\BlockCompressor.py", line 122, in blockCompress
Exception: NCA can't be decrypted. Outdated keys.txt?
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:\nsz_init_.py", line 137, in main
File "D:\nsz_init_.py", line 36, in compress
File "D:\nsz\BlockCompressor.py", line 228, in blockCompress
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process:

There will be a 52KB .nsz file created but not removed.
I think if there's exception during the process, the result should be removed to preventing confusing.

Also when the cause of the exception solved, (updating the keys), even without -w overwrite directive, the programm will still overwrite the bad 52KB nsz file.
Maybe here we should not overwrite it .

Encountered this problem on many files, for example
[0100DCA00DA7E000][v0] with update [0100DCA00DA7E800][v65536]
[0100273008FBC000][v0] with update [0100273008FBC800][v65536]
nsp files installed and run fine.
Both games are relatively small so should be good example to test.

nsz Bock compressed with 2.1.1 will install fine with Awoo, but could not start the software.
the same nsz install via Tinfoil 7 will report of multiple unkown frame descriptor problem during install. Install will finish, but same problem could not start the software.

The key file used in nsz compressing/decompressing is generated from the switch (9.1+Atomsphere 0.10.1)

Solid compressed nsz using 2.1.1 installed fine and works fine.

So it seems the block compressing is broken on 2.1.1 as well as the newest a562957 source. At least for some files. Or the installer (Tinfoil 7, Awoo 1.2.2 has problem understanding the block compressed nsz files).

Hey guys.

Tried to compress several XCI backups and the following error pops up on almost every game i have tried. here is the error:

Failed to extract TitleID/Version from filename .. Use -p to extract from Cnmt.

And here the output when i try to decompress the xcz WHICH FAILED:

Failed to extract TitleID/Version from filename ... Use -p to extract > > from Cnmt.
Error when decompressing file: ...
Traceback (most recent call last):
File "nsz_init_.py", line 219, in main
File "nsz_init_.py", line 62, in decompress
TypeError: decompress() missing 1 required positional argument: 'pleaseNoPrint'
Failed to extract TitleID/Version from filename ... Use -p to extract > from Cnmt.
Error when decompressing file: ...
Traceback (most recent call last):
File "nsz_init_.py", line 219, in main
File "nsz_init_.py", line 62, in decompress
TypeError: decompress() missing 1 required positional argument: 'pleaseNoPrint'

Any hints what i am doing wrong? :-)

thx

1. "Output Folder" should go Settings. And set input folder as output folder by default.
2. Kivy settings add "font" function, right now nsz_gui cant display Non-English characters well.

Thanks!

Hi nicoboss,

Thanks for your great work. I am trying to decompress the 1.1.1 updates for ACNH. It contains ncz file and I cannot use nsz -D filename.ncz to decompress.

Summary of errors which occurred while processing files: Error while processing /Users/drmeerkat/Downloads/switch/games/Animal Crossing New Horizons[01006F8002326800][US][v131072][CR-18]/3119652241e80b83078c27a3abb79d2e.ncz Traceback (most recent call last): File "/Applications/anaconda3/lib/python3.7/site-packages/nsz/__init__.py", line 218, in main if not args.overwrite and outfile.is_file(): AttributeError: 'str' object has no attribute 'is_file'

Then I manually fix this by removing the file existing check from init.py. But now another bug comes,
Summary of errors which occurred while processing files: Error while processing /Users/drmeerkat/Downloads/switch/games/Animal Crossing New Horizons[01006F8002326800][US][v131072][CR-18]/3119652241e80b83078c27a3abb79d2e.ncz Traceback (most recent call last): File "/Applications/anaconda3/lib/python3.7/site-packages/nsz/__init__.py", line 222, in main decompress(filePath, outfolder) File "/Applications/anaconda3/lib/python3.7/site-packages/nsz/__init__.py", line 64, in decompress NszDecompress(filePath, outputDir, statusReportInfo) File "/Applications/anaconda3/lib/python3.7/site-packages/nsz/NszDecompressor.py", line 21, in decompress container.open(filePath, 'rb') File "/Applications/anaconda3/lib/python3.7/site-packages/nsz/Fs/File.py", line 222, in open raise IOError('Invalid file parameter') OSError: Invalid file parameter

Could you help with this?

Hey guys.

Tried now to compress my XCI backups and the following pops up a lo in the logs:

`
Traceback (most recent call last):
File "multiprocessing\connection.py", line 312, in _recv_bytes
BrokenPipeError: [WinError 109] Die Pipe wurde beendet

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "multiprocessing\process.py", line 297, in _bootstrap
File "multiprocessing\process.py", line 99, in run
File "nsz\BlockCompressor.py", line 18, in compressBlockTask
File "", line 2, in get
File "multiprocessing\managers.py", line 819, in _callmethod
Traceback (most recent call last):
Traceback (most recent call last):
Process Process-6:
File "multiprocessing\connection.py", line 250, in recv
Traceback (most recent call last):
File "multiprocessing\connection.py", line 312, in _recv_bytes
File "multiprocessing\connection.py", line 312, in _recv_bytes
File "multiprocessing\connection.py", line 321, in _recv_bytes
File "multiprocessing\connection.py", line 312, in _recv_bytes
BrokenPipeError: [WinError 109] Die Pipe wurde beendet
BrokenPipeError: [WinError 109] Die Pipe wurde beendet
EOFError
BrokenPipeError: [WinError 109] Die Pipe wurde beendet
`
Is this normal/known behaviour?
Because the tool generates XCZ files (which are smaller than the XCI).

Hey guys.

tried to compress the game with title id "bbb-h-aacca".
A little curious is that the compress takes just a few seconds and game shrunk from 1,75 GB to 1,38 GB. But there was no progressbar shown during compression,
When i decompress it, the resulting XCI is the same size as the shrunken XCZ.
I have compared them with an binary compare program and it says the are binary the same!

Failed to extract TitleID/Version from filename "1 2 SWitch - bbb-h-aacca.xcz". Use -p to extract > > from Cnmt.
Decompressing ....xcz -> ....xci
[ADDING] secure 0 bytes to NSP
[ADDING] fbc46f4b672d686bcc618ec3c0c37753.nca 1480868352 bytes to NSP
[VERIFIED] fbc46f4b672d686bcc618ec3c0c37753.nca
[ADDING] ce6606d93e67c89b7c75bb83c89e7507.nca 184320 bytes to NSP
[VERIFIED] ce6606d93e67c89b7c75bb83c89e7507.nca
[ADDING] 55c76a09e5bdc9a2bfd0a9ac74be3a33.nca 278528 bytes to NSP
[VERIFIED] 55c76a09e5bdc9a2bfd0a9ac74be3a33.nca
[ADDING] c7ce40219ce34c22c388e915a856bd69.cnmt.nca 4096 bytes to NSP

Thats the log on decompressing.

PS.: The File name has spaces in it, maybe this is the issue?

nsz.py --level 22 -C "L A Noire[0100830004FB6800][US][v131072].nsp"
,;:;;,
;;;;;
.=', ;:;;:,
/_', "=. ';:;:;
@=:__, ,;:;:'
_(.= ;:;;'
"_( _/="
`"'
bktr reloc exception: Too many entries
bktr reloc exception: Too many entries
bktr reloc exception: Too many entries
compressing (level 22) \L A Noire[0100830004FB6800][US][v131072].nsp -> \L A Noire[0100830004FB6800][US][v131072].nsz
Adding file 0100830004fb68000000000000000003.cert 1792 bytes to NSP
Adding file 0100830004fb68000000000000000003.tik 704 bytes to NSP
Adding file 4b0fe8e40b498d7a4556c3d1e8433a51.ncz 14134086144 bytes to NSP

When the file is finished compressing it won't install at all.

Edit: At 77% I get the following error on the same file:

nut exception:
Traceback (most recent call last):
File "\nsz.py", line 98, in
nut.compress(filePath, 17 if args.level is None else args.level, args.solid, args.bs, args.output, args.threads)
File "\nut_init_.py", line 165, in compress
buffer += partitions[partNr].read(CHUNK_SZ - len(buffer))
MemoryError

It is not clear from description: does sections covers all file (0x4000-EOF) or only describes parts of the file that should be reencrypted.

Also, how can I get uncompressed length? Should I get it from original nca header, or I can get it from last section offset+length?

No matter the source file size, if I try to use any more than 1 thread for compressing nsp to nsz, the script dies with a not enough memory error, I have 24GB of RAM, surely its a error, no?

I left a remark before expandfiles.. It's acting weird if you give one file in current directory and when you're executing nsz.py from its' path like this:

python /path/to/nsz/nsz.py -C -o /path/to/out file.nsp

While this should find out file.nsp's absolute path it appends /path/to/nsz before file.nsp and tries to find file in /path/to/nsz/ directory. Why? I didn't really get it myself :)

Reproduce:
nsz_v3.0.0_hotfix1_win64_portable.zip
add gen10 prod.keys
load two .nsz files via select/load nsz files
click verify
the Gui crashed and closed.

System Windows 10 64B, Python 3.8

win32file module is missing unless pywin32 module is installed, which causes the GUI to crash when selecting a file/folder for processing

Thank you for updating this project. There are a few reasons I'll be staying on 2.1.1, however.

3.1.1 won't work without the master_key_07. Not even on games/dlc that don't use that. I also acknowledge that this issue may cause some other ones listed here.

Freezes on doing any action (completion bar stays at 0% or 100%).

The GUI shuts down while the command prompt stays up.

GUI doesn't allow drag-and-drop.

GUI doesn't open a Windows-style prompt for directory/file information. More hassle to use 3.1.1 than 2.1.1 (I have a batch file for 2.1.1 with the directories already set, so I don't have to click through a dozen folders each time).

OS: Win10 x64

EDIT: Very sorry about wasting your time. I dumped my keys using the proper payload but didn't copy them from the correct folder (copied an old keys file from a different place). Keys fixed, program appears to work with files it didn't before.

I got this error while trying to decompress the update of Super Mario Party, but it happened with others updates. I compressed it with solid compression and level 22 and it went fine. I'm using the latest commit and here's the log i get
1.txt

Is XCZ support planned?

Same file as before and now it just stops anywhere between 68-73% and the output file disappears.

nsz.py --level 22 -C "L.A. Noire - Update v1.2.nsp"
,;:;;,
;;;;;
.=', ;:;;:,
/_', "=. ';:;:;
@=:__, ,;:;:'
_(.= ;:;;'
"_( _/="
`"'
compressing (level 22) L.A. Noire - Update v1.2.nsp -> L.A. Noire - Update v1.2.nsz
Adding file 0100830004fb68000000000000000003.cert 1792 bytes to NSP
Adding file 0100830004fb68000000000000000003.tik 704 bytes to NSP
Adding file 4b0fe8e40b498d7a4556c3d1e8433a51.ncz 14134086144 bytes to NSP
70%|██████████████████████████████████████████████████▊ | 9.83G/14.1G [1:03:37<27:49, 2.58MB/s/s]

fin

nsz.py --level 22 -C "L.A. Noire - Update v1.2.nsp"
,;:;;,
;;;;;
.=', ;:;;:,
/_', "=. ';:;:;
@=:__, ,;:;:'
_(.= ;:;;'
"_( _/="
`"'
compressing (level 22) L.A. Noire - Update v1.2.nsp -> L.A. Noire - Update v1.2.nsz
[ADDING ] 0100830004fb68000000000000000003.cert 1792 bytes to NSP
[ADDING ] 0100830004fb68000000000000000003.tik 704 bytes to NSP
[ADDING ] 4b0fe8e40b498d7a4556c3d1e8433a51.ncz 14134086144 bytes to NSP
69%|██████████████████████████████████████████████████▎ | 9.73G/14.1G [1:03:49<28:52, 2.54MB/s/s]

fin

Using the latest build of nsz.py. I have more than enough space free and grabbed the file from a different source, same outcome.

When trying to decompress or verify big games (4GB+ i think) with these parameters : -C -w -V -l 22 -B
At some point i run out of RAM (i only have 8GB) and it just crashes.
Here's the log for a 6.6GB game
1.txt
Here it crashed during verification but the same happens during decompression.