python tools not functional from install (prefix) location when building from source about wesnoth

[white-haired-uncle]

This was compiled from git, no packages involved. I can't say for sure, but I seem to remember that the tools don't come with package versions.

[soliton-]

@loonycyborg ^

[ProditorMagnus]

SyntaxWarning: invalid escape sequence '\*' indicates issues in file.

[white-haired-uncle]

There is no file in this case, but at least wmlindent is running as compared to the first case when it aborts due to missing module.

[Elvish-Hunter]

Since the Python tools appear to run correctly in previous versions, it could be related to the new Python version (I haven't updated yet so I can't tell for sure).
The SyntaxWarning: invalid escape sequence '\*' line certainly is specific to Python 3.12, and this is something that we'll have to fix because it's planned that a future Python version will convert that warning into a much harsher SyntaxError.

[white-haired-uncle]

Since the Python tools appear to run correctly in previous versions, it could be related to the new Python version (I haven't updated yet so I can't tell for sure).

This isn't new. I'm not positive it goes back to 1.16, but I'm over 50/50. Or if you mean python version I didn't pay attention, but the issue goes back a couple years at least.

The SyntaxWarning: invalid escape sequence '\*' line certainly is specific to Python 3.12, and this is something that we'll have to fix because it's planned that a future Python version will convert that warning into a much harsher SyntaxError.

Oh, that's a different issue I guess. I thought that was just because I ran it without any input, but I just ran it on /usr/local/share/wesnoth/data/campaigns/Secrets_of_the_Ancients/scenarios/11_Battleground.cfg.out and it does error. I was only trying to show that /usr/local/bin/wml* and /usr/local/share/wesnoth/data/tools/wml* worked differently.

Perhaps I just stumbled on this since I was working on u24.04 (python=3.12.3)? I haven't seen an issue with python=3.10.12 on an earlier 1.19 build.

[white-haired-uncle]

I tried to test this with cmake, but I can't find any way to build the WML tools other than scons. Perhaps that's documented along with the WML tools build instructions for scons.

On a different note, I did try setting, for example export PYTHONPATH=/usr/local/lib/python/site-packages and I still got an error, but a different one. I'm wondering if no one notices /usr/local/bin/wmlindent not working (for them) because they have their environment set correctly, whereas I am working from a clean slate. If so, I think it's reasonable for the user to expect at least some directions on what to set up (though just doing something like installing links in $prefix/bin to $prefix/share/wesnoth/data/tools/ seems like a better approach since that just works without the user having to do anything -- unless perhaps there's a reason to respect their environment?).

[CelticMinstrel]

I've added the Linux label since this is clearly dependent on Linux's practice of distributing an installed program and its associated files across various places.

I'd say the easiest fix is to just install a shell script that changes to the appropriate directory and runs the tool from there. There might be a better fix though.

But I am wondering if this is specifically a Wesnoth issue at all. Unless you installed the Flatpak version, this is probably something to raise with the Ubuntu package managers rather than with Wesnoth.

EDIT: Ah, it could still be a bug in scons install-pytools, which would indeed be on us.

[CelticMinstrel]

Still specific to Linux in any case.

[loonycyborg]

Running tools installed fails because it can't find the "wesnoth" module. The build script seems to be installing that module to site-packages, so this could be probably fixed by setting it to correct value for your environment. There's python_site_packages_dir setting in scons script for that. I'm not sure how python is looking for site packages now. I bet it changed a lot. You can still launch them from data/tools directly and it will find "wesnoth" module then because it's in same dir as the tool in question. Probably with python it's expected to use setuptools/poetry/etc to do proper system installs.