Bug

As initially reported by PatchIt! Version 1.1.1 RC tester RobExplorien:

I can make a patch for LOCO with LEGO Racers data, and install it.

And confirmed by PatchIt! Version 1.1.1 RC tester JimbobJeffers:

I noticed this too. It would be nice if you could run a simple check for the folders that are entered, even if it's just for the existence of 'GAMEDATA' and 'MENUDATA' folders.

Comments

This can be fixed, but I have delayed it to Version 1.1.2. Version 1.1.1 is already really large, and I would like to have a smaller release for a change. Futher more, as I will speak about in the next section, I'm still trying to find the best place to put such a check.

Possible Fix

As RobExplorien and JimbobJeffers agreed on, yes, a simple check for GAMEDATA and 'MENUDATA' and the LEGO LOCO folders will suffice, but there is some logic involved that makes it less straight-forward that one might think.

Research using pyqtdeploy for binary freezing once a GUI is implemented. This may be required if cx_Freeze does not handle PyQt5 well.

More modern logo to be released along side a GUI (#30), hopefully created by @rioforce. Has the possibility to not happen.

Bug

As reported by PatchIt! Version 1.1.0 RC tester JimbobJeffers:

When exporting a Patch to my desktop, it went through and deleted every single thumbnail in all the folders on my desktop, which meant going through all my LEGO Racers GAMEDATA and MENUDATA folders for all the various mods I have. This isn't preferable ;)

Comments

This issue is a semi-extension of Issue #2.
JimbobJeffers had chosen his Deskop (C:\Users\username\Desktop) for the files to be compressed. So, PatchIt! began doing what it is coded to do: run the delThumbs() process to go through all the folders in the root folder via os.walk() (in this case, the Desktop folder), finding and deleting all instances of Thumbs.db, so the Patch will not end in an error during installation. However, since he selected his Desktop folder, it deleted every last instance of Thumbs.db in every single folder, including subfolders, on his Desktop. He said he was pretty nervous, as he didn't know exactly what was doing on. And that is understandable.

Possible Fix

• The fix for this is to limit the number of folders os.walk() walks though. It walks until it reaches the very last subfolder. Ideally, it is limited to the max numbers of subfolders in either GAMEDATA or MENUDATA (if one has more subfolders than the other, limit it to the one with the max folders). GAMEDATA has 26 subfolders, while MENUDATA has 35 subfolders. So, the limit needs to be set at 35.
• http://bytes.com/topic/python/answers/34058-os-walk-maxdepth
  http://stackoverflow.com/a/2212698

Bug

Sigh, leave it to me to make the same dumb mistake twice.

One of the main reasons I rewrote the PiP File Format in V1.1.0 was to not limit the description to a single line. If it was longer than one line, stuff could/would break.

Also in V1.1.0, I rewrote the Settings system, and made the Racers settings check if it was a 1999 or 2001 release, and write it in Racers.cfg (per Documentation). I added this version check to aid in supporting both releases of LEGO Racers.

However, if the Racers installation it really long (see overly exaggerated example below), and does not fit on the one line, the Racers version can get pushed down, and I referenced that version in later code and it was in the wrong location, who knows what could happen. So I made the same mistake I did when I originally wrote the PiP File Format. *facedesk*

Overly Exaggerated Example

// PatchIt! V1.1.x LEGO Racers Settings
# Ensures the first-run process will be skipped next time
1
# Your LEGO Racers installation path
C:/Users/le717/Documents/LEGO/Racers/The Game/1/New folder/js/css/New folder/html/hahahahahah/TAHEGAME/Red-bearded 
Thunder/Okkakakoohh/MMMMMMMMMMOOOOOOOOOOOOOOOOOOOO
# Your version of LEGO Racers
1999

Fix

Redo the Racers.cfg file to move the version below the first-run check, and move the installation path to the end of the file (basically, the version check and path swap locations). So the overly exaggerated example would now look like:

// PatchIt! V1.1.x LEGO Racers Settings
# Ensures the first-run process will be skipped next time
1
# Your version of LEGO Racers
1999
# Your LEGO Racers installation path
C:/Users/le717/Documents/LEGO/Racers/The Game/1/New folder/js/css/New folder/html/hahahahahah/TAHEGAME/Red-bearded 
Thunder/Okkakakoohh/MMMMMMMMMMOOOOOOOOOOOOOOOOOOOO

This requires me to update the Patch Installation modules to look for the new path, update the Settings writing code to write the CFG in the proper layout, and it forces the users to reset their Racers settings yet again (the first time in 1.1.0). I should have never made this mistake in the first place. *facedesk*

With the new Patch Create module, some of the error messages are displayed using a Tkinter messagebox.showerror() box and others red text.

Documentation/CONTRIBUTING.md and Documentation/Settings.md need to be updated with the newest information.

Documentation/CONTRIBUTING.md

• New folder paths
• Running most at once by running setup.py in the root director
• Credit to wget

Done!

Documentation/Settings.md

• PatchIt.cfg now contains the build number. Thus, line 3 (and the example) now reads something like

1.1.2 Stable build 52

The Details header also needs to be updated to mention the build number.

Done!

Both extraction and recompression.

Bug

As reported by PatchIt! V1.1.0 RC Tester @Brickever:

When installing a patch, there is no confirmation for duplicate files.
It is possible for a malicious user to create a patch with fake important game files, making the game unusable on the computer.
Either don't allow files outside the GAMEDATA folder to be modify or ask confirmation for every duplicate files.

Comments

Hobino's point is understandable. While it seems he is referring to broken or fake game files ("malicious") causing the game to break, his report reveals a deeper issue.
If someone made a malicious patch, the user would unknowingly be installing a virus to their computer.

Possible Fix

• Files outside GAMEDATA or MENUDATA will not work, for then custom music could not be installed.
• Create a list of all files inside the ZIP archive before opening it but after user confirms the installation. Then create list of all files in Racers installation. If one of the file names match, confirm installation of file. If no, skip installing that file. If yes, delete the file in the Racers installation, then install the new file.

While this fix is technically possible, it would be a bit more difficult to implement, and may be postponed until after the release of V1.1.0. It is possible to list all files in an ZIP before installation, as well as in a series of folders. Comparing the two could be tricky, may require an internal list or temporary external file. Skipping the installation of a file... I have no idea if that is possible or not. Overwriting a file is possible, but in more of a round-about way. Windows will not allow the overwriting of a file; it must be deleted then installed. Fortunately, I already have code to do this, found in ICU ReDirect.

Bug

For some reason, my code for deleting Thumbs.db before creating a Patch is deleting the file in more folders than it should. Possible bug is located in the os.walk() line, because I am still trying to figure out how to properly use os.walk().

Fix

• Fix delThumbs() to delete Thumbs.db in the modded files ONLY
• Remove delThumbs() and replace it with code to exclude certain files from the ZIP archive. Ideal fix, but unknown if it is possible with shutil.make_archive. Is possible with tarfile, but do not want to switch to a TAR.
• Remove os.walk() and instead use the glob module. Use the code from ICU ReDirect as the base, or use it altogether.

How to reproduce:

• Coming Never, as this was fixed before this section was written

Feature

For a while now, I've wanted to add a way to insert new lines in the Patch Description. Right now, it is an absolute WYSIWYG deal: if a word is not completed by the end of the line, you just have to keep typing, and however the word is cut off is how it is displayed during Installation.

Comments

I would need to use some character to designate a new line, one that would not force character escaping, but is still easy to type. Further more, it should be one that can typed on any keyboard, laptop, desktop, or tablet, full-size or not.

The Implementation

I know exactly how I will implement this feature, and the character I will use. I'll use an horizontal bar (|) to denote a new line, as it can be typed on any keyboard, and many gaming utilities already use it to denote a new line, so it wouldn't be a special-case thing. As for the code, I just have to detect the presence of any bars, replace them with the new line symbol (\n) using the built-in replace function, then write the description to the PiP File. This does not edit the PiP File Format, as it it already designed and coded to exist on however many lines are left in the PiP.
The following code and it's interpretation shows exactly how it works.

desc = input("Description: ")
print(desc)

# Check for the presence of | in the input,
# and replace it with a new line
if "|" in desc:
    newdesc = desc.replace("|", "\n")

# Print the updated text
print(newdesc)

# Display iin a list to show a diagram of the new text
print([newdesc])

# Now, let's write the description
with open("MyPatch.txt", "wt") as f:
    f.write(desc)   

# ...And read it back
with open("MyPatch.txt", "rt") as f:
    lines = f.read()

# And display the reading   
print(lines)

# Again, display in a list so we can see the diagram of the text
print([lines])
# Tada! The new lines are preserved. :)

And the code, when run

Description: MyPatch is an example PatchIt! Patch whose description spans multiple lines.|It uses an horizontal bar to denote a new line. Before it is written, the bar is replaced with the new line|symbol so new lines are written||Hey, look! I am | three lines before the last paragraph!
"MyPatch is an example PatchIt! Patch whose description spans multiple lines.
It uses an horizontal bar to denote a new line. Before it is written, the bar is replaced with the new line
symbol so new lines are written"

Hey, look! I am 
 three lines before the last paragraph!"
['"MyPatch is an example PatchIt! Patch whose description spans multiple lines.\nIt uses an horzintal bar to denote a new line. Before it is written, the bar is replaced with the new line\nsymbol so new lines are written\n\nHey, look! I am \n three lines before the last paragraph!"']

Bug

By default, the Python input() box allows all characters to be entered, both strings, integers and floats. I do not think it has any internal character limits, not even on Windows. However, Windows has some internal character limits (see attached screenshot). Since Python does not seem to honor those limits, a very dangerous and serious bug is created, existent since V1.0 Beta 2, where a Patch could theoretically be created with an invalid character, and would be unable to delete it. I have had this happen to me once before. Ubuntu was duel-booted on a computer, and I took a screenshot. The screenshot had a colon in the file name, and colons are reserved for drive letters on Windows. Once I booted back into Windows, I was unable to edit, rename, move, copy, or delete the file, and Ubuntu had already been removed. It stayed there until the HDD was redone. This bug could create the situation, and that never needs to happen.

How to reproduce

Will not be written. As already stated, this is a dangerous bug to exploit, and will not tell how to do it and mess up someone's computer.

Fix

• Attach a character checker to the Name and Version input fields during Patch Creation. I already have code to do this, and is easy to implement.

Screenshot

Feature

As requested by PatchIt! Version 1.1.1 RC tester JimbobJeffers:

I selected my Rock Racers folder for creating a patch, and it took quite a while to complete. I wasn't sure how long it would take nor whether or not the program had crashed or something. I don't know how feasible it is, but some sort of progress update would be nice. As the program does take not of the files it's processing (it outputs to the log), perhaps you could run a preliminary scan on the number of files/folders to process, convert it to some usable percentage, and use print whenever a significant amount is reached? E.g. every 10%. It would help for large mods. 😄

Comments

This is completely doable. However, it will be postponed until Version 1.1.2 3. Version 1.1.1 2 is already really large, and I would like to have a smaller release for a change.

The Implementation

A percentage may not be possible, but some form of a progress bar is. Correct, the files it is sorting is not displayed, only logged. I did this because I didn't want a bunch of text flying onto the screen. However, since a GUI is not here yet (@rioforce has made one in QT4, but I have to learn OOP before I can use it), a bunch of text may be the only way. I can't do a bunch of dashes being printed horizontally across the screen, the command window is not capable of doing that. I could display the files as they are compressing, in similar fashion to how LUCA does it (I actually wrote the "progress bar" in that program).

Things that must be done before PatchIt! Version 1.1.1 will be released, listed in no particular order:

• Change Patch Archive extension to .PiA
• Update PiP Format V1.1 documentation with .PiA extension
• Add Credits Page to Website/Other Website Issues (see Issue #9)
• Finish updating PiP Format V1.1.md PiA Archive section
• Add TAR Archive extraction code to Patch/modernextract.pyw
• Add TAR Archive compression code to Patch/moderncompress.pyw
• Finish writing whitelist code for TAR Archive compression
• Any bugs from RC testers
• Any open Issues for this milestone
• Make sure Documentation is up-to-date
• Add file encoding check when reading game settings during installation process
• Possibly add new PatchIt! Settings
• Write compiling directions for Inno Setup Installer
• Final version for PiP Format
• Move old downloads from Readme.md to releases page
• Release! 😃

Update Documentation/Settings.md for new JSON based settings formats.

• Back up current Python configuration
• Download and install latest version of Python 3.4
• Install latest version of cx_Freeze
  • Requires custom build from http://www.lfd.uci.edu/~gohlke/pythonlibs/#cx_freeze until next release or binary reuploads (see https://bitbucket.org/anthony_tuininga/cx_freeze/issue/81/python-34-venv-importlib-attributeerror)
• Install latest Python 3.4 compatible PyQt5 build
• Install pywin32 extensions
• Update Python 3.3 version locked shebangs
• Perform through testing and freezing of code base, including all helper scripts
• Update all code as needed for full support

Seriously, it needs to be a singleton. I'm running the same processes with unchanged values each time I need it, as evidenced by the multiple first-run messages in an empty log when I've only launched it once.

An (semi-incomplete) list of everything that must be done before 1.1.2 is released.

• Break up a lot of the functions into smaller scripts
• Add code to add new lines in a Patch Description (see #13)
• Integrate RunAsAdmin for Administrator rights access
• Trim out Tkinter files to discover just what is need to run
• Improved 16x16 icon
• Update freezing documentation with new folder paths
  and wget (see #17)
• Freeze Uninstaller when freezing PatchIt! itself as part of process
• Update all setup.py scripts to use "GPLv3" for the license field instead of "GNU GPLv3"
• Make all scripts use import constants as const syntax (see #18)
• Improved installer sidebar graphic
• Any reported bugs
• Any open Issues for this milestone

As part of my new goal for PatchIt!, LEGO® LOCO no longer warrants a spot in a LEGO® Racers modding utility.

• Remove all LEGO® LOCO code
• Update PiP File Format with removal
• Update PiP installation to mention removal
• Update all documentation with removal
• Update website with removal

Issue

Somewhat an extension of #3, the Patch Creation Name and Version fields both allow invalid file names on Windows.

How to reproduce

This is a very dangerous bug, and I will not explain how exploit it and mess up someone's computer beyond repair.

Fix

The fix for this issue is rather simple, it is mainly a matter of implementing it. The names would be stored in a list, much like the invalid characters are. The code would check if the input matched the list exactly (normalizing the case first because Windows is case-insensitive). If it matched, then the user would have to use a different name. Now, which input check would come first (name or characters) has yet to be decided, but it will be.

Invalid Names (minus commas)

Aux, Com1, Com2, Com3, Com4, Con, Lpt1, Lpt2, Lpt3, Prn, Nul

Feature Prototype

_The following is a conversation I had with @Anonymooseable about the updater, and how it might work. I think it is the best way to explain the Updater concept._

I already added a CFG file that reports the version of the app being used,
and it was going to read that, compare it to a text file it downloaded containing the new version number,
If yes, read the rest of the file, get the download link, and download it. Simple as that.
The "delta update" so to speak contains only the changed bits. Once the download is complete, check if PatchIt is not running (if so, close it), and it will extract the archive (most likely a tar.gz) into the installation path, overwriting the other files.
It will get that path by asking for your installation upon first-run, and store it for later.
The Tk stuff would NOT be included in that delta update.
Ship the updated exe, library.zip, and whatever else is new.
All done.

Comments

I've wanted this for a while (sometime after the development of v1.0.1 did I have the idea), and I know have enough knowledge and skill in Python to do it. It won't be that hard to write as I've already had experience with this type of work from an unreleased utility experiment for a different project, and it can be used for v1.1.2, since I added a feature in v1.1.1 to support it. In addition, I figured out exactly what main files are changed when during freezing, so downloading the files or a tarball containing the files will do.

The Implementation

• I'll use wget to download the files, and add my own download error check (since wget does not do this yet).
• The file that contains the newest version and download will be hosted on the gh-pages branch, the same branch the official Website is hosted on. This way, an HTTP download can occur, instead of an HTTPS one a raw file download link uses.
• The Updater will read the downloaded file and PatchIt.cfg, compare the version numbers, and take the appropriate action depending on the outcome.
• If PatchIt.cfg does not exist, it automatically prompts to update.
• It will also make use of the build numbering system implemented in v1.1.2 to determine if the copy needs updating, with "error" handling for v1.1.1.
• It will automatically download and use the RunAsAdmin utility by @QuantumCD
  to update PatchIt!, and either delete it when done or put it in the proper location if the updated release needs it and it was not already present. The download location will be from the master branch.
• The Updater will be distributed through the Inno Setup installer, and a shortcut to it will be made in the Start Menu/Screen.
• Cross-platform support will most likely be added once PatchIt! itself is that way.

The original PiP Format has served it's duty, and now it is time for it to be removed.

• Delete Patch/legacyinstall.pyw
• Update Installation to detect original PiP file, and throw warning saying it is no longer supported.

• The 1.1.1 Change Log still needs to be uploaded
• An Credits page must be created to contain all credit text @le717 had in the old Readme.
• A bit of restructure is needed.
• Merge contents of Readme branch into the gh-pages` branch, have all changes publish directly

Bug

As reported by PatchIt! Version 1.1.0 RC tester RobExplorien:

PatchIt closes immediately when a validity check turns out false. Though when installing a patch, or creating one, and the .PiP file cannot be found, it shows the red error message and proceeds to go to the main menu. So would it be a problem to have PatchIt proceed to the main menu too, if a validity check turns out false?

...

I tampered with the .PiP file. If it is tampered with, can it show an error message telling that it is an invalid patch? Just like it is when you [a user] install a patch, and the file cannot be found, it shows an error message and proceeds to the main menu.
I don't know how the Python script works, and if it is possible to not crash, but show the message in this case and go to the main menu.

Comments

Yea... I forgot about this. I have a check on the validity line, but not on the PiP as a whole. I ran into this the other week, but ignored it.

Possible Fix

• Yes, this is possible to fix. I think I have an early version of the PiP Format V1.1 (and if not, I can make one), so I can try to install it and see what error I get. I'm thinking all I need to do is surround it in a try... except block. If that doesn't work, I'll come up with something.

Screenshot

Semi-complete list of all changes that must be done before the release of PatchIt! Version 1.1.3.

• Add code to serve as a progress bar until GUI is added (see #11)
• Restructure all scripts in OOP
  • Patch-related
  • Settings-related
• Drop all LEGO® LOCO code (see #20)
• Remove Legacy Patch Installation support (Patch/legacyinstall.pyw) (see #19)
• Further script renaming and project organization
• Full support of LEGO® Racers 2001 release
• Finish improving installer sidebar graphic
• Make as much code as possible pass flake8 and pep257
• Any reported bugs
• Any open Issues for this milestone