Update doesn't find git executable about base16-builder-python

[InspectorMustache]

Thanks for reporting, I have no way of running the builder on Windows but I'll look into how shutil.which works on that platform when I find the time. Let me know if you find out anything yourself.

[InspectorMustache]

Okay, I'm not sure how executable lookup works in the Win command shell but as a starting point could you check what paths os.getenv(PATH) produces and check whether or not the git executable is actually in one of those places?

I guess we could just use an EAFP approach with Popen but since there are a lot of threads running git in parallel, I feel it's safer to find out where git is and make sure it's present before running those threads.

[InspectorMustache]

@cafeclimber ping

[cafeclimber]

Hey sorry for the slow response. running os.getenv('PATH') shows C:\\Program Files\\Git\\cmd

Running where git shows the above path followed by git.exe

[Cerebus]

This is probably related to #11. updater.py calls Popen() with the env argument, which per subprocess.py:

If env is not None, it must be a mapping that defines the environment variables for the new process; these are used instead of the default behavior of inheriting the current process’ environment.

Unfortunately, updater.py was setting only one key in the environment passed to Popen(). #12 fixes this by copying the current environment and adding the key required by updater.py.

[InspectorMustache]

Merged #12 . Let me know if you're still having issues.

[InspectorMustache]

@cafeclimber could you let me know if you're still having issues? Otherwise, I'd like to push the new version to pip.

[cafeclimber]

Still the same issue :/ Git executable not found in path

[InspectorMustache]

Okay. So further investigation required. 😶 This might be a dumb question, but did you use the latest commit from master?

[cafeclimber]

Haha totally fair question. I did a fresh git clone and got the results I did. If that isn't the latest commit, then I'm a dingus

[InspectorMustache]

Aaaaah... sorry for the late reply. So here's some other ideas, could you run these in the interpreter?

os.getenv('PATHEXT')

platform.architecture()
platform.architecture('C:\\Program Files\\Git\\cmd\\git.exe')

[cafeclimber]

Sure thing. Here's the output:

>>> os.getenv('PATHEXT')
'.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC;'
>>> platform.architecture()
('32bit', 'WindowsPE')
>>> platform.architecture('C:\\Program Files\\Git\\cmd\\git.exe')
('32bit', '')

[InspectorMustache]

Okay, so your Python executable is 32bit. Is your version of Windows the same? Also, could you check the path you passed to platform.architecture()? The fact that the second element of the tuple is an empty string probably indicates that there is no file at that location.

[cafeclimber]

No it's 64 bit windows. I check on that path and it seems okay. I get that result for a number of different programs. I navigated to that path in file explorer, and the file is there. I tried navigating to that directory in cmd then running the interpreter and doing platform.architecture() and got the same result. I imported os and used abspath() to make sure the path was correct and it was. :/

This version of python was installed by virtualenv-wrapper so I'm not sure if that's why it's 32-bit?

[InspectorMustache]

Virtualenv shouldn't isolate your Python environment from system binaries so that's probably not the cause, though I can't be sure. Is it possible for you to install a standard 64bit Python distribution and check again?
If that doesn't change anything, I'm pretty much out of ideas. You could open an issue on the offical Python tracker and ask why shutil.which isn't able to see git. I'd be very interested in the reason for that as well.

[cafeclimber]

So I'm a total idiot and don't know why I didn't think of it before, but running shutil.which('git.exe') instead of shutil.which('git') succeeds

[InspectorMustache]

That shouldn't be an issue though because shutil.which is supposed to take in account the PATHEXT environment variable (as described in the docs. Your previous post shows that .EXE is part of that variable.

[InspectorMustache]

Okay, so this is what I think is happening here: You're using a 32bit version of Python but a 64bit version of git. 32bit Python automatically translates 'Program Files' to 'Program Files (x86)' and doesn't find the git executable there. At least that's what shutil.which seems to be doing, no idea why the program is visible to Popen. We could implement a workaround by appending '.exe' to the executable name in Windows but that seems very hacky to me. Also, until we make sure that this is unintended behaviour, I feel that if git isn't accessible by shutil.which it shouldn't be accessible to the builder as well.

[cafeclimber]

No no I uninstalled my 32 bit version of Python and installed a 64 bit and retried everything. I see that in the docs though and it's weird it doesn't find it. It does the same thing for any program I try...

I'll ask a question on SO and see if I can get anything

[InspectorMustache]

@cafeclimber Any news?

[InspectorMustache]

Closing for now.