Comments (9)
In the documentation of SQLAlchemy here, they explain that using 'select' for an async relationship is attempting to use implicit IO and is subsequently not allowed.
Word for word from their example:
Accessing the A.bs collection on newly loaded instances of A when eager loading is not in use will normally use lazy loading, which in order to succeed will usually emit IO to the database, which will fail under asyncio as no implicit IO is allowed.
The link also provides their solution to this which is to use the AsyncAttrs mixin.
When you use the 'joined' loading mechanism, the relationship is pre-loaded with the result instead of on an as-needed basis that comes with the 'select' mechanism.
from sqlmodel.
In https://sqlalche.me/e/20/xd2s the following is also mentioned:
When using the ORM this is nearly always due to the use of lazy loading, which is not directly supported under asyncio without additional steps and/or alternate loader patterns in order to use successfully.
I did not check the code yet, but if we implement an additional loader in SQLModel, this could work?
For completes sake, here is an example (mainly from the docs) to reproduce the issue:
import asyncio
from typing import Optional, List
from sqlalchemy.ext.asyncio import create_async_engine
from sqlmodel import Field, Relationship, SQLModel, select
from sqlmodel.ext.asyncio.session import AsyncSession
class Team(SQLModel, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
name: str = Field(index=True)
headquarters: str
heroes: List["Hero"] = Relationship(back_populates="team")
class Hero(SQLModel, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
name: str = Field(index=True)
secret_name: str
age: Optional[int] = Field(default=None, index=True)
team_id: Optional[int] = Field(default=None, foreign_key="team.id")
team: Optional[Team] = Relationship(back_populates="heroes")
async def main() -> None:
engine = create_async_engine("...")
async with engine.begin() as conn:
await conn.run_sync(SQLModel.metadata.create_all)
async with AsyncSession(engine) as session:
session.add(Team(name="Some Team", headquarters="Somewhere"))
await session.commit()
session.add(Hero(name="Spider-Boy", secret_name="Pedro Parqueador", team_id=1))
await session.commit()
async with AsyncSession(engine) as session:
statement = select(Hero).where(Hero.name == "Spider-Boy")
result = await session.exec(statement)
hero_spider_boy = result.one()
print("Spider-Boy's team again:", hero_spider_boy.team)
asyncio.run(main())
logs:
$ python -m async_relations
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/a/path/async_relations.py", line 47, in <module>
asyncio.run(main())
File "~/.pyenv/versions/3.11.7/lib/python3.11/asyncio/runners.py", line 190, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "~/.pyenv/versions/3.11.7/lib/python3.11/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "~/.pyenv/versions/3.11.7/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/a/path/async_relations.py", line 44, in main
print("Spider-Boy's team again:", hero_spider_boy.team)
^^^^^^^^^^^^^^^^^^^^
File "/a/path/.venv/lib/python3.11/site-packages/sqlalchemy/orm/attributes.py", line 566, in __get__
return self.impl.get(state, dict_) # type: ignore[no-any-return]
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/a/path/.venv/lib/python3.11/site-packages/sqlalchemy/orm/attributes.py", line 1086, in get
value = self._fire_loader_callables(state, key, passive)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/a/path/.venv/lib/python3.11/site-packages/sqlalchemy/orm/attributes.py", line 1121, in _fire_loader_callables
return self.callable_(state, passive)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/a/path/.venv/lib/python3.11/site-packages/sqlalchemy/orm/strategies.py", line 967, in _load_for_state
return self._emit_lazyload(
^^^^^^^^^^^^^^^^^^^^
File "/a/path/.venv/lib/python3.11/site-packages/sqlalchemy/orm/strategies.py", line 1068, in _emit_lazyload
return loading.load_on_pk_identity(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/a/path/.venv/lib/python3.11/site-packages/sqlalchemy/orm/loading.py", line 692, in load_on_pk_identity
session.execute(
File "/a/path/.venv/lib/python3.11/site-packages/sqlmodel/orm/session.py", line 129, in execute
return super().execute(
^^^^^^^^^^^^^^^^
File "/a/path/.venv/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 2308, in execute
return self._execute_internal(
^^^^^^^^^^^^^^^^^^^^^^^
File "/a/path/.venv/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 2190, in _execute_internal
result: Result[Any] = compile_state_cls.orm_execute_statement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/a/path/.venv/lib/python3.11/site-packages/sqlalchemy/orm/context.py", line 293, in orm_execute_statement
result = conn.execute(
^^^^^^^^^^^^^
File "/a/path/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1416, in execute
return meth(
^^^^^
File "/a/path/.venv/lib/python3.11/site-packages/sqlalchemy/sql/elements.py", line 517, in _execute_on_connection
return connection._execute_clauseelement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/a/path/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1639, in _execute_clauseelement
ret = self._execute_context(
^^^^^^^^^^^^^^^^^^^^^^
File "/a/path/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1848, in _execute_context
return self._exec_single_context(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/a/path/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1988, in _exec_single_context
self._handle_dbapi_exception(
File "/a/path/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 2347, in _handle_dbapi_exception
raise exc_info[1].with_traceback(exc_info[2])
File "/a/path/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1969, in _exec_single_context
self.dialect.do_execute(
File "/a/path/.venv/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 922, in do_execute
cursor.execute(statement, parameters)
File "/a/path/.venv/lib/python3.11/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 580, in execute
self._adapt_connection.await_(
File "/a/path/.venv/lib/python3.11/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 121, in await_only
raise exc.MissingGreenlet(
sqlalchemy.exc.MissingGreenlet: greenlet_spawn has not been called; can't call await_only() here. Was IO attempted in an unexpected place? (Background on this error at: https://sqlalche.me/e/20/xd2s)
And an example using joins like mentioned by @Trevypants :
# ...
async with AsyncSession(engine) as session:
statement = select(Hero, Team).join(Team).where(Hero.name == "Spider-Boy")
result = await session.exec(statement)
hero_spider_boy, team = result.one()
print("Spider-Boy's:", hero_spider_boy)
print("Spider-Boy's team:", hero_spider_boy.team)
print("Spider-Boy's team again:", team)
results in:
Spider-Boy's: id=2 name='Spider-Boy' team_id=1 age=None secret_name='Pedro Parqueador'
Spider-Boy's team: headquarters='Somewhere' id=1 name='Some Team'
Spider-Boy's team again: headquarters='Somewhere' id=1 name='Some Team'
```
from sqlmodel.
I've got the same error on Many to Many and I've solved it with defining relationship like this:
class HeroTeamLink(SQLModel, table=True):
team_id: int | None = Field(default=None, foreign_key="team.id", primary_key=True)
hero_id: int | None = Field(default=None, foreign_key="hero.id", primary_key=True)
class Team(SQLModel, table=True):
id: int | None = Field(default=None, primary_key=True)
name: str = Field(index=True)
headquarters: str
heroes: list["Hero"] = Relationship(
back_populates="teams",
link_model=HeroTeamLink,
sa_relationship_kwargs={"lazy": "selectin"},
)
class Hero(SQLModel, table=True):
id: int | None = Field(default=None, primary_key=True)
name: str = Field(index=True)
secret_name: str
age: int | None = Field(default=None, index=True)
teams: list[Team] = Relationship(
back_populates="heroes",
link_model=HeroTeamLink,
sa_relationship_kwargs={"lazy": "selectin"},
)
from sqlmodel.
@a410202049 could you include code that can reproduce this issue? I would like to take a look at it
from sqlmodel.
async with AsyncSession(engine) as session: statement = select(Hero, Team).join(Team).where(Hero.name == "Spider-Boy") result = await session.exec(statement) hero_spider_boy, team = result.one()
this approach works for many-to-one side when querying a hero with only one team. However, it becomes more complex for one-to-many side when querying a team with many heroes, and even more so for querying many teams with many heroes, since result.all()
returns a list where each element is a tuple of one team and one hero. we need extra effort to loop over and format the result.
from sqlmodel.
from sqlalchemy.ext.asyncio import AsyncAttrs
...
async with AsyncSession(engine) as session:
statement = select(Hero).where(Hero.name == "Spider-Boy")
result = await session.exec(statement)
hero_spider_boy = result.one()
team = await hero_spider_boy.awaitable_attrs.team
When creating a new model class, adding AsyncAttrs to the inherited class can achieve this function to a certain extent. However, when obtaining this attribute, the database will be accessed again, so the performance is not very good. I donβt know if there is an operation such as prefetch that can be done in one step.
from sqlmodel.
I believe what you're seeking about prefetch is eager loading, which can be achieved by selectinload, you can find demos on google. And I'm looking forward to learning how to use AsyncAttrs with SQLModel and lazy loading.
from sqlmodel.
I believe what you're seeking about prefetch is eager loading, which can be achieved by selectinload, you can find demos on google. And I'm looking forward to learning how to use AsyncAttrs with SQLModel and lazy loading.
thanks for reply, I've found this method. This method is very convenient and efficient when obtaining the associated information of a set of data.Instead of accessing the database every time you access an attribute of one item, you only need to access the database once to obtain the associated information for this set of data.
from sqlmodel.
I believe what you're seeking about prefetch is eager loading, which can be achieved by selectinload, you can find demos on google. And I'm looking forward to learning how to use AsyncAttrs with SQLModel and lazy loading.
Thanks.
It seems work.
This is my code:
# select
statement=select(MHero).options(selectinload(MHero.mteam_links)).where(MHero.id==hero_id)
res=await session.exec(statement)
test_hero:MHero=res.one()
print('test_hero',test_hero.mteam_links)
# get
db_hero = await session.get(MHero, hero_id,options=[selectinload(MHero.mteam_links)])
from sqlmodel.
Related Issues (20)
- There is no unique constraint matching given keys (one-to-many, connecting with many-to-many tables) HOT 2
- Preparing for Pydantic v2 release HOT 13
- Data Integrity: Raise error on attempt to delete an object required via a Relationship HOT 8
- [Querying] negating `Model.boolean` in `where()` HOT 2
- [M2M] Query dependent incl. `link_model` fields HOT 3
- Could not refresh instance HOT 9
- SQLModel doesn't recognize Relationship between models HOT 1
- Get select with options (selectinload) using response schema HOT 2
- Dose there any better way to write timezone aware datetime field without using the SQLAlchemy ? HOT 3
- Obtaining `TypeError: Cannot pickle 'module' object` on models with many-to-many relationships HOT 2
- Order of columns in the table created does not have 'id' first, despite the order in the SQLModel. Looks like it's prioritising fields with sa_column HOT 3
- Erro ao executar uvicorn.run(...) HOT 1
- Many to many relationship between a table and itself HOT 6
- How to add current date time by default on a table declaration? HOT 13
- Add documentation about how to use the async tools (session, etc) HOT 5
- π Roadmap HOT 28
- Internal link failed at create-db-and-table.md
- Field cannot autocompletion when its a SQLModel HOT 6
- Add an overload to the `exec` method with `_Executable` statement for update and delete statements HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sqlmodel.