bl-sdk / pythonsdk Goto Github PK
View Code? Open in Web Editor NEWBorderlands 2 PythonSDK
Borderlands 2 PythonSDK
For some reason, .__setitem__
isn't working in Python for my custom TArray type.
When running obj dump
, arrays get truncated at 100 elements. It is occasionally useful to view more than this. Since we already create a console key and enable set commands, it would be nice if we also automatically removed this limit.
Patch (this one also works for TPS):
7E 05 B9 64 00 00 00 3B F9 0F 8D
to
75 05 B9 64 00 00 00 3B F9 0F 8D
If you'd also like to remove the message ... XY more elements
Patch this aswell:
0F 8C 7B 00 00 00 8B 8D 9C EE FF FF 83 C0 9D 50
to
0F 85 7B 00 00 00 8B 8D 9C EE FF FF 83 C0 9D 50
TPS Specific
To remove the message mentioned above - patch:
7C 7B 8B 8D 94 EE FF FF
to
75 7B 8B 8D 94 EE FF FF
The SDK supports calling function with out
params, by returning the changed parameters in a list.
ret, vehicle = PC.IsUsingVehicleEx(True)
As the title says, if the parameter is a struct or an array, it returns a garbage pointer, which is likely to cause a crash when you do anything else to it.
unrealsdk.Log(PC.Pawn.InvManager.GetItemList())
This is because these two are complex value types, which rely on memory in the params struct. This struct is zero'd and freed shortly after the function call, leaving us with garbage pointers. Fixing this simply requires working out how to make copies of these types, and how to allocate new memory for them.
Unreal allows for optional arguments to functions, these aren't dumped with the SDK generators. This functionality is needed for avoiding incredibly verbose functions.
The SDK codebase is a mess of code styles, since it's been passed between so many authors without proper standardization. We should define a standard, and format it to fix.
Tracking:
TArrays are more complicated than a simple struct. Simply malloc and freeing them causes issues, so we need to hook into the init from UE.
We have a logging module, but it's very simple and kind of confusing. We should use a dedicated library instead. Ideally:
[<module>] message
).unrealsdk.Log
in Python takes a module and integrates directly with logging system.stdout
in Python goes to file as in normal logging, but goes directly to console without special formatting.Trying to assign any FArray object into an array field crashes the game. This means rather unintuitively, the following example will cause a crash:
obj1 = bl2sdk.FindObject("MissionDefinition", "GD_Anemone_Side_VaughnPart2.M_Anemone_VaughnPart2")
obj2 = bl2sdk.FindObject("WeaponTypeDefinition", "GD_Weap_Pistol.A_Weapons.WeaponType_Jakobs_Pistol")
obj3 = bl2sdk.FindObject("SkillDefinition", "GD_Assassin_Skills.Bloodshed.Execute")
obj4 = bl2sdk.FindObject("ItemPoolDefinition", "GD_CustomItemPools_Lilac.Psycho.BlueBold")
# Each line here causes a crash
obj1.Dependencies = obj1.Dependencies
obj2.WeaponIdleAnimations = obj2.WeaponIdleAnimations
obj3.SkillConstraints = obj3.SkillConstraints
obj4.BalancedItems = obj4.BalancedItems
If you convert the FArray into a list beforehand it will correctly be set, and the game won't crash. Not all fields support being assigned from a list however.
obj1 = bl2sdk.FindObject("MissionDefinition", "GD_Anemone_Side_VaughnPart2.M_Anemone_VaughnPart2")
obj1.Dependencies = list(obj1.Dependencies)
Hi there. Been trying to play around with some BL2 mods to spice it up a bit more. Trying to achieve something simple before going in on other mods: use apple1417's SDK Autorun mod to load up UCP on launch. But I'm hitting the snag as described in the title. If ddraw.dll is there, the game briefly shows the splash screen before crashing completely. If it's not there, the game loads up perfectly fine with no issues. I've followed and checked all the various (now kind of outdated) guides and I've done everything right as far as I can see. The issue is purely down to ddraw. I figured maybe the Valorant anti-cheat thing might be preventing it, so I closed that, and nope it still crashes.
Sitting next to the python files in the Binaries/Win32 folder is this log file, which reads thus:
======= Borderlands 2 Python Loader =======
[AntiDebug] Hook added for NtSetInformationThread
[AntiDebug] Hook added for NtQueryInformationProcess
Found EXE name as 'Borderlands2.exe'
[Internal] GObjects = 0x019CAB20
[Internal] GNames = 0x01988744
[Internal] UObject::ProcessEvent() = 0x004F27F0
[Internal] UObject::CallFunction() = 0x004F2300
[Internal] FFrame::Step() = 0x004E31B0
[Internal] UObject::StaticConstructObject() = 0x00429D20
[Internal] UObject::LoadPackage() = 0x0042C8C0
[Internal] GMalloc = 0x0193A8F4
[Internal] FindOrCreateFName = 0x003F08D0
[Internal] GetDefaultObject = 0x003C1F20
Sigscan failed (Signature not found, Mask = FF 83 C4 0C 85 C0 75 1A 6A 01 8D)WINAPI Error when enabling 'SET' commands: 487
Plugin loaded: d
[AntiDebug] NtSetInformationThread called with ThreadHideFromDebugger
I've searched up and down for some kind of fix or alternative solution to this problem and the most I found was just another random mod that acknowledges ddraw.dll can prevent the game from starting up. Any ideas?
This code causes a consistent crash when the hooked function is run (shortly after loading into a map with vendors).
def Log(caller: unrealsdk.UObject, function: unrealsdk.UFunction, params: unrealsdk.FStruct) -> bool:
unrealsdk.Log(params)
unrealsdk.RunHook("WillowGame.PopulationFactoryVendingMachine.CreateInteractiveObject", "test", Log)
The function signature (and thus what the params struct should contain) is as follows:
protected function WillowInteractiveObject CreateInteractiveObject(PopulationMaster Master, Vector SpawnLocation, Rotator SpawnRotation, int GameStage, int AwesomeLevel, InteractiveObjectDefinition SelectedObjectDefinition) {
Hi, i am a player who use Simplified Chinese Language
There is a problem when adding pythonSDK to borderlands 2
Using Proton-7.1-GE-2, with launch option: "WINEDLLOVERRIDES="ddraw=n,b" bash -c 'exec "${@/Launcher.exe/Borderlands2.exe}"' -- %command%".
OS: Ubuntu 20.04.4 LTS
The game crashes when I open Borderlands 2 with the launch options mentioned.
FArray needs +=
, .append
, and .insert` support
FArray = FArray
and FStruct = FStruct
also needs to be added
When assigning strings to objects' properties using the SDK, the results are as expected, and the changes appear in game as desired:
namePart = bl2sdk.FindObject(
"ItemNamePartDefinition",
"GD_ClassMods.Prefix_Assassin.Prefix_Infiltrator_00_Infiltrator"
)
namePart.PartName = "Infilbooper"
However, when the same property is subsequently altered elsewhere in the engine (specifically via a set
console command), it triggers a crash:
bl2sdk.GetEngine().GamePlayers[0].Actor.ConsoleCommand(
"set GD_ClassMods.Prefix_Assassin.Prefix_Infiltrator_00_Infiltrator PartName Infiltrator",
0
)
We should setup automated builds so we can tell when errors are introduced. We can then also use it as part of branch protection, to prevent merging errors.
We should also just generally have branch protection requiring PRs/preventing force pushing on bl3
and master
.
Similar to dynamic objects, this will completely unlock the SDK to work in any version of BL2, past, present, or future.
System info: Arch Linux x86_64 5.10.13-zen1-2-zen
Proton Version: 5.0-10 Also tested with Proton-GE-5.21
Issue: When trying to edit mod settings and keybinds through in-built menu, clicking/hovering on the links does nothing
Latest Python-SDK is installed 0.7.8
Tested with only default mods installed no extras, enabled Borderlands 2 Map Reloader for keybinds and Backpack Manager for mod options. When hovering or clicking on the button error messages are output to python-sdk.log.
At:
Z:\home\michael.local\share\Steam\steamapps\common\Borderlands 2\Binaries\Win32\Mods\ModMenu\KeybindManager.py(231): _HandleSelectionChangeRollover
SystemError: <built-in method next of PyCapsule object at 0x16CEB320> returned NULL without setting an error
At:
Z:\home\michael.local\share\Steam\steamapps\common\Borderlands 2\Binaries\Win32\Mods\ModMenu\OptionManager.py(101): _WillowScrollingListOnClikEvent
SystemError: <built-in method next of PyCapsule object at 0x16CEB320> returned NULL without setting an error
We currently pull all our files in the top level of the zip, meaning users must manually navigate to <game>\Win32\Binaries
to extract them. We should add these two folders, so you can directly extract the zip over the game folder, similarly to modding a lot of other games.
Doing this is a very simple change of course, this issue exists more a reminder to update all the documentation and tutorials currently telling people to navigate.
I saw streamers using it, how can i make it work?
There's likely a flag that can be passed to the function that will fix this. Alternatively, we may be able to load just the needed objects from the package.
Title
Instead of printing out <UObject at 0x00000>
, it would be nice if it would instead call UObject.GetFullName()
and show that instead.
If you have a reference to an object in Python and the object gets garbage collected in game, then trying to access the object in Python again will sometimes cause crashes.
Here is some example code to cause the crash.
import bl2sdk
bl2sdk.LoadPackage("GD_Mercenary_Streaming_SF")
GCObj = bl2sdk.FindObject("SkillDefinition", "GD_Mercenary_Skills.Brawn.Incite")
# Use a hook so we can try reference the object later
def HookMainMenuInput(caller: bl2sdk.UObject, function: bl2sdk.UFunction, params: bl2sdk.FStruct) -> bool:
if params.ukey == "F1":
bl2sdk.Log(str(GCObj))
bl2sdk.Log(str(GCObj.SkillName))
return True
bl2sdk.RemoveHook("WillowGame.FrontendGFxMovie.SharedHandleInputKey", "GCDemo")
bl2sdk.RegisterHook("WillowGame.FrontendGFxMovie.SharedHandleInputKey", "GCDemo", HookMainMenuInput)
To test this code run it on the main menu, then wait for the next GC cycle and press F1. While waiting you can run getall SkillDefinition Name
in console to see what objects are loaded. Once the object has been removed pressing F1 might crash the game. The crash is not consistent, but still relatively common. If the game doesn't crash then it will never crash using that reference (so re-execute the file), and the log file will contain the following.
>>> pyexec GCDemo.py <<<
[HookManager] (EngineHooks) ERROR: Failed to remove hook "GCDemo" for "WillowGame.FrontendGFxMovie.SharedHandleInputKey"
SkillDefinition GD_Mercenary_Skills.Brawn.Incite
Incite
SkillDefinition GD_Mercenary_Skills.Brawn.Incite
Incite
SkillDefinition GD_Mercenary_Skills.Brawn.Incite
None
SkillDefinition GD_Mercenary_Skills.Brawn.Incite
None
>>> pyexec GCDemo.py <<<
SkillDefinition GD_Mercenary_Skills.Brawn.Incite
Incite
SkillDefinition GD_Mercenary_Skills.Brawn.Incite
Incite
<Game Crashes>
Note that the hook fires on both key press and release, hence the double logging.
Removing the second log line sometimes gives the following output instead. I have never seen it happen with both log calls, but as the crash is inconsistent anyway it might still happen.
>>> pyexec GCDemo.py <<<
[HookManager] (EngineHooks) ERROR: Failed to remove hook "GCDemo" for "WillowGame.FrontendGFxMovie.SharedHandleInputKey"
SkillDefinition GD_Mercenary_Skills.Brawn.Incite
SkillDefinition GD_Mercenary_Skills.Brawn.Incite
SkillDefinition UnknownName.ByteProperty.Incite
SkillDefinition UnknownName.ByteProperty.Incite
<Game Crashes>
I believe making UObject::GetProperty()
and UObject::SetProperty()
check if the object has been GCed will probably prevent this, though the inconsistent nature of the crash may mean the issue is hidden deeper somewhere.
On a related note, the SDK doesn't have a good way for mods to check if an object has been GCed. Even if this crash gets fixed, mods that are working with objects that may be GCed will need to know this to prevent themselves from causing exceptions.
The console has a very powerful syntax WeaponPartDefinition'GD_Orchid_BossWeapons.AssaultRifle.AR_Barrel_Bandit_Rapier'
, that will get said object of said class. Mimicking this functionality will be crucial, and if we can add an ability to load said object if it's unloaded, we'll unlock a ton of potential.
I believe that this will be done via StaticLoadObject
: http://api.unrealengine.com/INT/API/Runtime/CoreUObject/UObject/StaticLoadObject/index.html
Other important hooks: http://api.unrealengine.com/INT/API/Runtime/CoreUObject/UObject/StaticDuplicateObjectEx/index.html
http://api.unrealengine.com/INT/API/Runtime/CoreUObject/UObject/StaticFindObject/index.html
DynamicLoadObject doesn't seem to work as I would expect it to based on documentation.
https://wiki.beyondunreal.com/Legacy:DynamicLoadObject
Right now, script hooks get parameters via stack
, and rely on the user to correctly pop items off of the stack. This introduces a lot of risk for crashing. There is likely some way of making this safe and doing that work before passing it back to the user by analyzing the UFunction.
In this repo's current state, it only works for current patch. Older and future patches will not work, unless we start dynamically getting the class pointers.
pClassPointer = (UClass*)UObject::GObjObjects()->Data[2];
On init, we should be able to iterate through all objects in GObjObjects, run GetName() on them, and make a mapping of string names to indexes. This would let us dynamically get these class pointers.
We're currently using pybind 2.3.dev0, from mid-2019. We should upgrade. #68 is fixed by this, and there are likely other changes which make development better.
Having both of these adds needless complexity on the modder's end, for no extra functionality. These can be merged after #29, because we'll be able to dynamically pop all of the parameters off of the stack and put them into a struct for Python access via __getattr__
.
The function signature should be:
def process_hook(caller: UObject, function: UFunction, params: FStruct) -> bool:
Notice the missing stack
and result
, as manipulating the stack should not be part of Python functionality. result
(i believe) is just a pointer to the ReturnValue
inside of the params
struct, so that can be ignored as well.
As we remove stack
, we'll use the return value from the function in the case of hkCallFunction
calls, and automatically call SkipFunction
if need be, otherwise we will return the stack to the proper place before continuing execution.
autoexec stuff
This'll be implemented after #31 / #38 as this heavily relies on consistent file directories.
We'll want to save the user's force set keybinds for plugins.
In the same vein, it'll probably be best for us to set a user's option values (#40)
Mods\Settings.json
or we could give each mod their own Settings.json
file like: Mods\[mod]\Settings.json
.The same concept applies onto the Slider/Boolean/Spinner option types in the PLUGINS
submenu in the options menu.
Currently, the sigscans module must know the exact executable name in order to grab patterns. This makes using it on new games unnecessarily difficult, you have to recompile even if none of the patterns change.
Additionally, even within a single game we cannot guarantee executable name - briefly poking through the support server logs, I've seen Borderlands2.exe
, borderlands2.exe
and Launcher.exe
. We've also previously seen GetModuleFileName
returning corrupted names, though never been able to replicate.
Proposed solution is to store a list of patterns, and have each game store the best index. We can then iterate through all patterns if the game's unknown.
Whenever I launch the game with PythonSDK installed, the game will crash upon starting up on Windows 11. It's consistent with both BL2 and Pre-Sequel, without the modified binaries the game launches perfectly fine.
Currently the SDK requires the user to navigate a complex - and therefore error prone - assortment of files and folders for both installation and usage.
ddraw.dll
Plugins\
Plugins\python36.dll
Plugins\python36.zip
Plugins\pythonSDK.dll
Plugins\Python\
Plugins\Python\[mod].py
With even more complexity necessary to use mods that depend on additional files as resources:
Plugins\Python\[mod's library dependency].py
Plugins\Python\Textures\
Plugins\Python\Textures\[Texture file imported by mod].png
...etcetera.
A new approach is proposed in order to both simplify this for the user and allow developers freedom in bundling resources with their mods:
ddraw.dll
Mods\
Mods\[mod]\
Mods\[mod]\[mod resource]
In this proposed structure, ddraw.dll
includes the libraries and resources the SDK itself is dependent on. (It may possibly also perform loading of additional DLLs located in Plugins\
, if the user has any, to retain compatibility with existing plugins).
Besides installation of the SDK itself, this would also simply installation of mods, as the user would only be responsible for copying a single directory for each mod.
Mods will be adding new items, weapons, etc, and these are not currently persisted throughout saves. For these mods to be viable, we'll need to modify save files a bit.
My current proposal is to add a field that is only accessible in C++, that can make saves dependent on certain mods. Skyrim has some system whereby saves know which mods they were made with, and notifies the user that the save may not function properly without them.
This would need to be shown on the main menu when hovering over "Continue" and in the "Select Character" menu when hovering over a file.
"DLC" menu won't open without internet connection
We need a set of example/template mods showing how to do certain tasks, such as creating textures, creating weapons, creating hooks, modifying functions, and debugging.
As in the title, we should follow PEP8. Due to the difference in code style, this will also make it clearer what bits of code are referencing Python vs UnrealScript objects.
Additionally, we should ship a stubs file showing what we export.
https://answers.unrealengine.com/questions/23440/loading-textures-at-runtime.html
This might not be too hard?
StaticLoadObject(UTexture2D::StaticClass(), NULL, *(Path))
This means any mod trying to use them will not work in coop.
Basically all in the title. Issue comes from the selection change hook, it explicitly checks against the English name OPTIONS
.
if caller.MenuDisplayName == "OPTIONS":
selectedIndex = params.TheList.GetSelectedIndex()
Options.isMenuPluginMenu = (params.EventID == 0 and (selectedIndex == 5 or selectedIndex == 4))
https://github.com/bl-sdk/PythonSDK/blob/16a3eaf/Mods/OptionManager.py#L133
I've got a mod which I'm using to regenerate BLCMM data dumps, and the first step of that is looping through a ton of getall
statements to find object names (pythonsdk itself still has some deficiencies in that regard, so I can't do so "directly" via the sdk yet). It runs 500 getall
statements at a time, and then uses Tick
to wait a bit, so we don't run into the Runaway loop detected (over 1000000 iterations)
crash.
Anyway, on my machine, after about 18 of those chunks-of-500-getall
-statements, I get a crash-to-desktop, with this on the console:
Fatal Python error: deallocating None
Current thread 0x00000028 (most recent call first):
File "Z:\usr\local\games\Steam\SteamApps\common\Borderlands 2\Binaries\Win32\Mods\TooMuchConsole\__init__.py", line 71 in doGetall
File "Z:\usr\local\games\Steam\SteamApps\common\Borderlands 2\Binaries\Win32\Mods\TooMuchConsole\__init__.py", line 37 in staticDoApocTick
abnormal program termination
That's from a pared-down version to reproduce the error, which can be found here: https://gist.github.com/bcd3911f378da7fc90b8bfa54c5826c1
Line 71 is just:
pc.ConsoleCommand(command)
After loading the mod it'll start doing periodic dumps every 9 seconds or so; you can just remain on the main menu and let those happen. As I say, on my machine I get a crash about at the 18-iteration mark.
On my "real" app I've worked around it by just creating some files which I then execute with exec
, instead of running all the getall
s by hand, so to speak, so that works well enough for me for now, but it'd be nice to track this down.
Rather than defining python dirs and output dirs directly in the project, we should have a prebuild script setting up all needed defines, so that all user specific changes are in a single place.
Basically all in the title. Demo program:
import unrealsdk
unrealsdk.Log(__file__)
>>> pyexec test.py <<<
NameError: name '__file__' is not defined
At:
D:\SteamLibrary\steamapps\common\Borderlands 2\Binaries\Win32\Mods\\test.py(3): <module>
>>> py from Mods import test <<<
'D:\\SteamLibrary\\steamapps\\common\\Borderlands 2\\Binaries\\Win32\\Mods\\test.py'
Workaround: cause an exception and extract the file path from it. Put this code near the top of the file:
if __name__ == "__main__":
import sys
try:
raise NotImplementedError
except NotImplementedError:
__file__ = sys.exc_info()[-1].tb_frame.f_code.co_filename # type: ignore
If you do:
py import bl2sdk
py PC = bl2sdk.GetEngine().GamePlayers[0].Actor
py bl2sdk.Log(PC.Location)
It'll log out { }
But after that command you do:
py bl2sdk.Log(PC.Location.X)
It'll return PC.Location
's X property which while fine gets tedious when you need to write out all of the properties of a specific/unknown struct.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.