Comments (10)
This is weird because pdoc really just imports.
Can you test if calling importlib.invalidate_caches()
inserted just above these lines maybe helps?
What is the output of python -m site
?
from pdoc.
"python -m site?"
Here it is:
(venv) C:\Users\jlaufersweiler\dev\project>python -m site
sys.path = [
'C:\\Users\\jlaufersweiler\\dev\\project',
'C:\\Users\\jlaufersweiler\\dev\\project\\venv\\Scripts\\python36.zip',
'C:\\Program Files\\Python36\\DLLs',
'C:\\Program Files\\Python36\\lib',
'C:\\Program Files\\Python36',
'C:\\Users\\jlaufersweiler\\dev\\project\\venv',
'C:\\Users\\jlaufersweiler\\dev\\project\\venv\\lib\\site-packages',
]
USER_BASE: 'C:\\Users\\jlaufersweiler\\AppData\\Roaming\\Python' (exists)
USER_SITE: 'C:\\Users\\jlaufersweiler\\AppData\\Roaming\\Python\\Python36\\site-packages' (exists)
ENABLE_USER_SITE: False
This correctly sees the venv layered atop the system-wide 3.6 install.
In contrast, running pdoc att the same prompt is and searching executing from my user-specific 3.7 instance beneath ~\AppData\Local...:
(venv) C:\Users\jlaufersweiler\dev\project>pdoc --html batconvert
Traceback (most recent call last):
File "C:\Users\jlaufersweiler\AppData\Local\Programs\Python\Python37\lib\site-packages\pdoc\__init__.py", line 540, in import_module
module.__loader__.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "C:\Users\jlaufersweiler\dev\project\batconvert\__init__.py",
(rest of exception snipped)
pdoc3 is installed in both the 3.6 and 3.7 site-packages.
from pdoc.
Hypothesizing that pdoc executable is linked to the wrong Python (3.7), does it maybe work if you run:
python -m pdoc --html batconvert
In that case, maybe also run pip uninstall pdoc
and python -m pip install pdoc
and then simply pdoc
might work.
I'm not familiar with the intricacies of your Windos installation. I'd appreciate your further help in debugging this. 🥇
from pdoc.
OK, I uninstalled pdoc3 from the 3.7 environment, and now invoking pdoc within the venv is looking at the system 3.6 instance upon which the venv is based. However, it's still not seeing any modules installed within the venv:
(venv) C:\Users\jlaufersweiler\dev\project>pdoc --html batconvert
Traceback (most recent call last):
File "c:\program files\python36\lib\site-packages\pdoc\__init__.py", line 540, in import_module
module.__loader__.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "C:\Users\jlaufersweiler\dev\project\batconvert\__init__.py", line 2, in <module>
import requests
ModuleNotFoundError: No module named 'requests'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Program Files\Python36\Scripts\pdoc-script.py", line 11, in <module>
load_entry_point('pdoc3==0.5.3', 'console_scripts', 'pdoc')()
File "c:\program files\python36\lib\site-packages\pdoc\cli.py", line 338, in main
for module in args.modules]
File "c:\program files\python36\lib\site-packages\pdoc\cli.py", line 338, in <listcomp>
for module in args.modules]
File "c:\program files\python36\lib\site-packages\pdoc\__init__.py", line 542, in import_module
raise ImportError('Error importing {!r}: {}'.format(filename, e))
ImportError: Error importing 'batconvert\\__init__.py': No module named 'requests'
I am able to successfully generate results by installing pdoc3 directly within the venv.
It seems that without pdoc3 installed within the venv, the shell looks for pdoc.exe first in %VIRTUAL_ENV%\Scripts, doesn't find it there, then works down the %PATH% until it finds it in c:\program files\python36\Scripts. (It was invoking with my user-level 3.7 install previously, as it is ahead of site-wide 3.6 in my %PATH%.)
Thus, pdoc.exe is invoking cli.py with the system python.exe from within the (venv) shell, even though %VIRTUAL_ENV% is set to C:\Users\jlaufersweiler\dev\project\venv within that shell.
It is still managing to find the module the that was passed as an argument thanks to this part of cli.py:
def main(_args=None):
""" Command-line entry point """
global args
args = _args or parser.parse_args()
if args.close_stdin:
sys.stdin.close()
if args.template_dir is not None:
if not path.isdir(args.template_dir):
print('Error: Template dir {!r} is not a directory'.format(args.template_dir),
file=sys.stderr)
sys.exit(1)
pdoc.tpl_lookup.directories.insert(0, args.template_dir)
# Support loading modules specified as python paths relative to cwd
sys.path.append(os.getcwd())
That last line in the excerpt is how it's loading .\batconvert\__init__.py from C:\Users\jlaufersweiler\dev\project\ despite being in the system context rather than the venv context, since it's adding the current directory to sys.path, but it's not looking in .\venv\Lib\site-packages.
So that's what's happening and why.
Hoping for a workaround without installing pydoc3 in the venv, I uninstalled it from the venv, deactivated, set include-system-site-packages = true in ./venv/pyenv.cfg, reactivated, and tried again.
Unfortunately, while pip list inside the venv shell does now show pdoc3, pdoc --html batconvert still fails to find the modules in .\venv\Lib\site-packages. I think I see why...
Here's python -m site after the changes:
(venv) C:\Users\jlaufersweiler\eclipse-workspace\xltflask>python -m site
sys.path = [
'C:\\Users\\jlaufersweiler\\eclipse-workspace\\xltflask',
'C:\\Users\\jlaufersweiler\\eclipse-workspace\\xltflask\\venv\\Scripts\\python36.zip',
'C:\\Program Files\\Python36\\DLLs',
'C:\\Program Files\\Python36\\lib',
'C:\\Program Files\\Python36',
'C:\\Users\\jlaufersweiler\\eclipse-workspace\\xltflask\\venv',
'C:\\Users\\jlaufersweiler\\eclipse-workspace\\xltflask\\venv\\lib\\site-packages',
'C:\\Users\\jlaufersweiler\\AppData\\Roaming\\Python\\Python36\\site-packages',
'C:\\Program Files\\Python36\\lib\\site-packages',
]
USER_BASE: 'C:\\Users\\jlaufersweiler\\AppData\\Roaming\\Python' (exists)
USER_SITE: 'C:\\Users\\jlaufersweiler\\AppData\\Roaming\\Python\\Python36\\site-packages' (exists)
ENABLE_USER_SITE: True
The venv shell can find the pdoc modules C:\Program Files\Python36\lib\site-packages now, but without pdoc.exe installed within the venv\Scripts, it can't invoke it with the venv python, since C:\Program Files\Python36\Scripts isn't in the venv's sys.path even when 'C:\Program Files\Python36\lib\site-packages' is.
PROPOSED SOLUTION: Add a routine in cli.py that checks to see if the VIRTUAL_ENV environment variable is set, and if so do something like...
sys.path.append( os.path.join( os.environ.get( 'VIRTUAL_ENV' ), 'lib', 'site-packages' ) )
just like it does the cwd.
IMPLEMENTATION QUESTION: Should this always be done, or should it be a switchable behavior? If the latter, should it default to on or off?
...
from pdoc.
IMPLEMENTATION QUESTION: How should one write tests for something like this? Unit test cases for the environment variable check and sys.path modification operations would be fairly straightforward, but is that enough? Or would proper testing need to actually mock up a venv, with some modules in it that aren't in the standard library, make the example package import them, and then spawn a sub-process shell to enter the venv context and try to run the pdoc cli?
from pdoc.
To sum up: when virtualenv exists, overlaying system site-packages where pdoc is installed, invoking pdoc doesn't "see" the virtualenv packages because pdoc console script contains a shebang pointing to the system python interpreter, so venv python interpreter is never run, and sys.path
is never amended with venv paths.
I think this is a Python issue.
PEP 486 and its implementation aimed to make Python VIRTUAL_ENV
-aware in some cases on Windos, but it seems to not work in our case of console scripts.
Would you consider raising this issue up on the CPython issue tracker?
from pdoc.
This isn't an instance of the py launcher case in that PEP, though they do touch on similar concepts, and the PR I submitted to fix this here uses a similar approach to the proposed solution for the launcher.
What this would really need to be solved at the python-wide level is another flag for venv that would parallel --system-site-packages, say "--system-scripts," that would put the system scripts directory behind the virtualenv's scripts directory in the path when activated, just like --system-site-packages does with site-packages.
That being said, venv is in the standard library. A change like that to such an important component would need to start on the dev mailing list, then go through the PEP process. While I agree that it might be worth pursuing at that level, I suggest accepting the PR in the meantime, as it could take months or even years for such a change to get into 3.future. The patch isn't invasive, and could easily be extracted in a future update should such a hypothetical change to venv come to pass.
from pdoc.
What this would really need to be solved at the python-wide level is another flag for venv that would parallel --system-site-packages, say "--system-scripts," that would put the system scripts directory behind the virtualenv's scripts directory in the path when activated
It's what virtualenv already does by default:
$ source venv/bin/activate
$ echo $PATH
/home/user/venv/bin:/home/user/.local/bin:/usr/local/bin:/usr/bin:/bin
It just doesn't invoke the correct (current venv's) Python interpreter when the bin script shebang points to another. I think it's a very similar issue to what PEP 486 was intended to fix. 🤔
I'm not strongly inclined to merging simple workarounds without any input from the parties the issue likely most concerns. I encourage you to show the problem on CPython issue tracker (they'll know what needs or needs not be done) and please reference it here. 💮
from pdoc.
Same Problem here no Solution in sight I guess...
from pdoc.
I re-read PEP 486 and have changed my opinion. 😳 I guess appending virtual env path, if it exists, to sys.path
is acceptable. 👍
from pdoc.
Related Issues (20)
- Skip-errors not recogniazed HOT 1
- Support inline math with dollar `$...$` delimiters when `--config latex_math=True`
- search field wont appear in generated html file HOT 1
- generating the whole project documentation with 1 command HOT 2
- Support search in docpage HOT 2
- Wrong search hyperlinks to page sections for inherited classes HOT 1
- Strike Through Markdown not working
- PEP224 style docstring not honoured for global variable of type Callable
- HTML files should end with a newline
- Upgrade code reference warnings to errors.
- --skip-errors flag does not work
- feature request: support for headings to organize functions
- Include html containing javascript
- Using a - in folder names prevents linking
- missing docstrings of methods HOT 2
- syntax error with match-case function
- Incorrectly rendered Args section HOT 1
- Add GitHub Flavor Markdown support HOT 2
- Remove indentation in text template for markdown files
- Deprecation warning for PEP224 class variable docstrings
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pdoc.