Comments (23)
markqiu
commented on July 17, 2024
3
ok. got it. thank you very much!
邱承
Mark
…____________________________
盈利模方，打造您的投资能力库
From: "David ***@***.***>
Date: Tue, Apr 16, 2024, 20:06
Subject: Re: [DavidVujic/python-polylith] can't develop under editable mode in rye (Issue #198)
To: ***@***.***>
Cc: ***@***.***>, ***@***.***>
That doesn't look good!
I actually could reproduce the problem with the test and setting the
1/0
back-and-forth. It seems like the code that is run is cached or directed into the virtual environment as you suggested before. I will investigate more on this and try to learn more about Rye (I am not using Rye on a daily basis).
—
Reply to this email directly, view it on GitHub<#198 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAISSHOQLLXTCN34MUP4MBLY5UH3FAVCNFSM6AAAAABGD3M4TSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJYHEZTCMRUGU>.
You are receiving this because you were mentioned.[image: https://github.com/notifications/beacon/AAISSHKFLITQOQ42SZ7GZZTY5UH3FA5CNFSM6AAAAABGD3M4TSWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTT2XDGC2.gif]Message ID: ***@***.***>
![https://github.com/notifications/beacon/AAISSHKFLITQOQ42SZ7GZZTY5UH3FA5CNFSM6AAAAABGD3M4TSWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTT2XDGC2.gif]Message](https://github.com/notifications/beacon/AAISSHKFLITQOQ42SZ7GZZTY5UH3FA5CNFSM6AAAAABGD3M4TSWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTT2XDGC2.gif]Message){kind=link}
from python-polylith.
markqiu
commented on July 17, 2024
1
Workspace summary
projects: 2
components: 8
bases: 2
development: 1
brick ability profitcube_cli development
────────────────────────────────────────────────────
abilities ✔ - ✔
backtest - - ✔
core ✔ - ✔
data ✔ - ✔
resources ✔ - ✔
services ✔ - ✔
utils ✔ - ✔
可视化 - - ✔
cli ✔ ✔ ✔
config ✔ - ✔
from python-polylith.
markqiu
commented on July 17, 2024
1
You are so awesome!
But what I can't figure out is how the following code kick in?
[tool.polylith.bricks]
"bases/profitcube/cli" = "profitcube/cli"
"bases/profitcube/config" = "profitcube/config"
"components/profitcube/core" = "profitcube/core"
"components/profitcube/abilities" = "profitcube/abilities"
"components/profitcube/backtest" = "profitcube/backtest"
"components/profitcube/resources" = "profitcube/resources"
"components/profitcube/utils" = "profitcube/utils"
"components/profitcube/data" = "profitcube/data"
"components/profitcube/可视化" = "profitcube/可视化"
"components/profitcube/services" = "profitcube/services"from python-polylith.
DavidVujic
commented on July 17, 2024
This was an interesting error indeed!
Could it be because of that Rye wasn't aware of the "components" and "bases" folders (maybe the ".pth" file in the .venv wasn't created by Rye)? Does it help when running rye sync?
from python-polylith.
markqiu
commented on July 17, 2024
This was an interesting error indeed!
Could it be because of that Rye wasn't aware of the "components" and "bases" folders (maybe the ".pth" file in the .venv wasn't created by Rye)? Does it help when running
rye sync?
No. running rye sync is useless.
My pyproject.toml is the same with #192 (comment)
from python-polylith.
DavidVujic
commented on July 17, 2024
rye sync should do the proper things, I at least thought so. Can you try with removing the environment and recreate it again?
rm -rf .venv
rye sync
from python-polylith.
markqiu
commented on July 17, 2024
I've do it a lot of times with different configs. No luck.
from python-polylith.
DavidVujic
commented on July 17, 2024
Also, can you check if there is a ".pth" file in .venv/lib/python<your python version here>/site-packages/?
It should contain paths with bases and components.
from python-polylith.
markqiu
commented on July 17, 2024
Yes, there is a file called _xxxx.pth contains paths with bases and components.
from python-polylith.
DavidVujic
commented on July 17, 2024
Can you share the output of the rye run poly info command here?
from python-polylith.
markqiu
commented on July 17, 2024
I noticed the .pth file's name has a '' prefix in rye's venv. But there is no '' under poetry's venv.
from python-polylith.
DavidVujic
commented on July 17, 2024
Yes, I have the prefix too but that shouldn't matter. This is a very strange error indeed.
If you have the time, could you clone the Polylith rye-example repo and test if you have the same issues there?
from python-polylith.
markqiu
commented on July 17, 2024
I tried to add some codes to consumer and found some strange things.
from example import kafka, log
logger = log.get_logger("Consumer-app-logger")
def parse_message(topic: str, key: str, value: str) -> None:
logger.info("Consumed message with topic=%s, key=%s, value=%s", topic, key, value)
a = 1/0
return a
def main() -> None:
topic = "message"
kafka.consumer.consume(topic, parse_message)I run rye run pytest before I added the codes and after that, the result is the same. All tests passed.
After I delete the .venv directory, and run rye sync and rye run pytest again. The error came out.
test/bases/example/consumer/test_core.py:6:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
topic = 'topic', key = 'key', value = 'value'
def parse_message(topic: str, key: str, value: str) -> None:
logger.info("Consumed message with topic=%s, key=%s, value=%s", topic, key, value)
> a = 1/0
E ZeroDivisionError: division by zero
.venv/lib/python3.12/site-packages/example/consumer/core.py:8: ZeroDivisionError
===================================================================================== short test summary info ======================================================================================
FAILED test/bases/example/consumer/test_core.py::test_sample - ZeroDivisionError: division by zero
=================================================================================== 1 failed, 5 passed in 1.89s ====================================================================================
Rye installed the example project as a package, I think maybe that the problem.
from python-polylith.
DavidVujic
commented on July 17, 2024
Thank you for doing such great research @markqiu!
Installed as a package ... is there something in your config that tells Rye not to do that maybe?
from python-polylith.
markqiu
commented on July 17, 2024
You mean it ok for you in the rye exampme project?
I don't know rye much. maybe I should try to reinstall rye?
or do you need to see my rye config?
邱承
Mark
____________________________
盈利模方，打造您的投资能力库
From: "David ***@***.***>
Date: Fri, Apr 12, 2024, 20:27
Subject: Re: [DavidVujic/python-polylith] can't develop under editable mode in rye (Issue #198)
To: ***@***.***>
Cc: ***@***.***>, ***@***.***>
rye sync should do the proper things, I at least thought so. Can you try with removing the environment and recreate it again?rm -rf .venv
rye sync
—
… Reply to this email directly, view it on GitHub<#198 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAISSHL5IRW3YBVPHTIO5TLY47HJJAVCNFSM6AAAAABGD3M4TSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJRGY3DIOJXG4>.
You are receiving this because you authored the thread.[image: https://github.com/notifications/beacon/AAISSHMMAKUVLLRC634ZYBDY47HJJA5CNFSM6AAAAABGD3M4TSWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTT2JHWFC.gif]Message ID: ***@***.***>
![https://github.com/notifications/beacon/AAISSHMMAKUVLLRC634ZYBDY47HJJA5CNFSM6AAAAABGD3M4TSWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTT2JHWFC.gif]Message](https://github.com/notifications/beacon/AAISSHMMAKUVLLRC634ZYBDY47HJJA5CNFSM6AAAAABGD3M4TSWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTT2JHWFC.gif]Message){kind=link}
from python-polylith.
DavidVujic
commented on July 17, 2024
Good idea!
Just now, I installed Rye on a fresh Linux install (to investigate if it might be something with the OS, I use Mac OS) and ran the polylith example repo without any issues. I also added the dependencies from the project file you shared before and the polylith commands, and the rye build commands work well.
I hope reinstalling will solve it for you! 🙏
from python-polylith.
DavidVujic
commented on July 17, 2024
Another possible thing: I couldn't find pytest in your dev dependencies (the other issue #192 ) and wonder if it could be a possible cause, if you have a "global" pytest somewhere with a different type of config?
from python-polylith.
markqiu
commented on July 17, 2024
I tried in a linux machine with a new installed rye. The problem is the same.
the following is my pyproject.toml
[project]
name = "profitcube"
version = "0.1.0"
description = "jinniuai's investment power store"
authors = [
{ name = "markqiu", email = "[email protected]" }
]
dependencies = [
"jinja2>=3.1.3",
"typer>=0.12.2",
"rich>=13.7.1",
"numpy>=1.26.4",
"pydantic>=2.6.4",
"loguru>=0.7.2",
"pandas>=2.2.1",
"pandera>=0.18.3",
"dolphindb>=3.0.0.0",
"mootdx>=0.11.4",
"openpyxl>=3.1.2",
"mytt>=2.9.3",
"python-dotenv>=1.0.1",
"simple-toml-configurator>=1.2.2",
"croniter>=2.0.3",
"statsmodels>=0.14.1",
"pendulum>=3.0.0",
"pyfinance>=1.3.0",
"pydantic-extra-types>=2.6.0",
"backtrader @ https://pypi.jinniuai.com/packages/backtrader/backtrader-1.9.76.123.2.tar.gz",
"tomli-w>=1.0.0",
"dynaconf>=3.2.5",
]
readme = "README.md"
requires-python = ">= 3.11"
[build-system]
requires = ["hatchling", "hatch-polylith-bricks"]
build-backend = "hatchling.build"
[tool.rye]
managed = true
dev-dependencies = [
"polylith-cli>1.5.0",
"pytest>=8.1.1",
"ruff>=0.3.7",
"mypy>=1.9.0",
]
virtual = false
[tool.hatch.build]
dev-mode-dirs = ["components", "bases", "development", "."]
[tool.hatch.build.hooks.polylith-bricks]
[tool.polylith.bricks]
"bases/profitcube/cli" = "profitcube/cli"
"bases/profitcube/config" = "profitcube/config"
"components/profitcube/core" = "profitcube/core"
"components/profitcube/abilities" = "profitcube/abilities"
"components/profitcube/backtest" = "profitcube/backtest"
"components/profitcube/resources" = "profitcube/resources"
"components/profitcube/utils" = "profitcube/utils"
"components/profitcube/data" = "profitcube/data"
"components/profitcube/可视化" = "profitcube/可视化"
"components/profitcube/services" = "profitcube/services"
[tool.hatch.metadata]
allow-direct-references = true
[tool.ruff]
exclude = [
".git",
".github",
"__pycache__",
".mypy_cache",
".ruff_cache",
"dist",
".venv",
"./development/*.py",
]
line-length = 120
[tool.ruff.lint]
select = ["E4", "E7", "E9", "F", "I"]
from python-polylith.
DavidVujic
commented on July 17, 2024
Will it make any difference if you remove the virtual = false?
(and/or setting it to true?)
from python-polylith.
markqiu
commented on July 17, 2024
Will it make any difference if you remove the
virtual = false?(and/or setting it to
true?)
If I remove the line, It didn't work.
If I change to true, the following error reported:
raceback (most recent call last):
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/_pytest/config/__init__.py", line 856, in import_plugin
__import__(importspec)
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/_pytest/assertion/rewrite.py", line 178, in exec_module
exec(co, module.__dict__)
File "/home/markqiu/work/principle/test/components/profitcube/fixtures/events.py", line 2, in <module>
from profitcube.core.event import Event
ModuleNotFoundError: No module named 'profitcube'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/markqiu/work/principle/.venv/bin/pytest", line 8, in <module>
sys.exit(console_main())
^^^^^^^^^^^^^^
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/_pytest/config/__init__.py", line 197, in console_main
code = main()
^^^^^^
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/_pytest/config/__init__.py", line 155, in main
config = _prepareconfig(args, plugins)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/_pytest/config/__init__.py", line 337, in _prepareconfig
config = pluginmanager.hook.pytest_cmdline_parse(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 501, in __call__
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 119, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 138, in _multicall
raise exception.with_traceback(exception.__traceback__)
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
teardown.throw(exception) # type: ignore[union-attr]
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/_pytest/helpconfig.py", line 105, in pytest_cmdline_parse
config = yield
^^^^^
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 102, in _multicall
res = hook_impl.function(*args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1143, in pytest_cmdline_parse
self.parse(args)
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1492, in parse
self._preparse(args, addopts=addopts)
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1396, in _preparse
self.hook.pytest_load_initial_conftests(
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 501, in __call__
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 119, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 138, in _multicall
raise exception.with_traceback(exception.__traceback__)
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
teardown.throw(exception) # type: ignore[union-attr]
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/_pytest/warnings.py", line 150, in pytest_load_initial_conftests
return (yield)
^^^^^
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
teardown.throw(exception) # type: ignore[union-attr]
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/_pytest/capture.py", line 153, in pytest_load_initial_conftests
yield
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 102, in _multicall
res = hook_impl.function(*args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1221, in pytest_load_initial_conftests
self.pluginmanager._set_initial_conftests(
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/_pytest/config/__init__.py", line 579, in _set_initial_conftests
self._try_load_conftest(
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/_pytest/config/__init__.py", line 617, in _try_load_conftest
self._loadconftestmodules(
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/_pytest/config/__init__.py", line 657, in _loadconftestmodules
mod = self._importconftest(
^^^^^^^^^^^^^^^^^^^^^
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/_pytest/config/__init__.py", line 733, in _importconftest
self.consider_conftest(mod, registration_name=conftestpath_plugin_name)
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/_pytest/config/__init__.py", line 814, in consider_conftest
self.register(conftestmodule, name=registration_name)
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/_pytest/config/__init__.py", line 510, in register
self.consider_module(plugin)
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/_pytest/config/__init__.py", line 822, in consider_module
self._import_plugin_specs(getattr(mod, "pytest_plugins", []))
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/_pytest/config/__init__.py", line 829, in _import_plugin_specs
self.import_plugin(import_spec)
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/_pytest/config/__init__.py", line 858, in import_plugin
raise ImportError(
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/_pytest/config/__init__.py", line 856, in import_plugin
__import__(importspec)
File "/home/markqiu/work/principle/.venv/lib/python3.11/site-packages/_pytest/assertion/rewrite.py", line 178, in exec_module
exec(co, module.__dict__)
File "/home/markqiu/work/principle/test/components/profitcube/fixtures/events.py", line 2, in <module>
from profitcube.core.event import Event
ImportError: Error importing plugin "test.components.profitcube.fixtures.events": No module named 'profitcube'from python-polylith.
DavidVujic
commented on July 17, 2024
That doesn't look good!
I actually could reproduce the problem with the test and setting the 1/0 back-and-forth. It seems like the code that is run is cached or directed into the virtual environment as you suggested before. I will investigate more on this and try to learn more about Rye (I am not using Rye on a daily basis).
from python-polylith.
DavidVujic
commented on July 17, 2024
@markqiu Mystery solved!
I have wrongfully added the hatch hook into the Workspace pyproject.toml in the setup guide, and in the example repo.
I must have confused it with PDM, that needs a special workspace hook.
Without it, things should work as expected. I have verified in the example repo. Here's what I changed in the pyproject.toml at the root/workspace. The hook should only be in the projects.
DavidVujic/python-polylith-example-rye@ab029e9
from python-polylith.
DavidVujic
commented on July 17, 2024
That part is used by the poly commands.
But the projects also need it to build wheels from, and that is where the hatch-build-hook is used too: it parses the bricks section and appends them to the things about to be built.
from python-polylith.
Related Issues (20)
- poly delete HOT 1
- `tdd` theme doesn't work with `pdm` HOT 5
- adding a poly deps command
- poetry scripts break when building with --with-top-namespace HOT 8
- poetry poly create base will allow names with dashes HOT 4
- Poly sync not working properly for projects in sub-directories HOT 1
- there are some errors using rye HOT 10
- v1.5.0 release is broken for Python < 3.12 HOT 6
- error in rye build --whell HOT 1
- Improve functionality of "components" (specification of external dependencies & wheel builds)
- Aliases configuration file HOT 12
- [Guidance] A way to specify SAAS provided dependencies HOT 10
- "poetry poly sync" returns "'Key "tool" does not exist.| HOT 13
- Add a namespace for the bases and component HOT 1
- Issue with autogenerated tests and implicit namespace folders for `tdd` theme HOT 4
- Discrepency between `poly diff` and `poly create` for `tdd` theme HOT 2
- Getting in touch! HOT 2
- (poetry plugin) Third-party import lookup fails for some libraries (example: python-jose).
- poly libs and poly check: optionally return error for library version mismatch
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from python-polylith.