Bug

I cannot add new field to model during the iterations of my application.

At the first iteration of the development of my application, I define the model of User, which has four fields : appid, openid, and nickname, avator. But three month later, we need add a new field "unionid" to every user of our application, including new ones and old ones. But I cannot make it unless I mannually update all users' data to add a new field with MongoDB command.

Current Behavior

When query data from MongoDB, odmantic will try to parse every field defined in the model and required all of them existed no matter whether each of them is required, and raise an exception when encounter an unrequired and missing field rather than provide a default value according to the model.

Expected behavior

when encounter an an unrequired and missing field, odmantic should provide a default value according to the definition of model

Environment

• ODMantic version: 0.2.1
• MongoDB version: ...
• Pydantic infos (output of python -c "import pydantic.utils; print(pydantic.utils.version_info())):

...

• Version of additional modules (if relevant):
  • ...

Additional context

Add any other context about the problem here.

How to check connection to mongoDB using odmantic?

Feature request

Context

Since using odmantic the purpose is to to async calls to the db, it makes sense to be able to validate some field regarding some data already in the DB.

Use Case Example:

Registering a new user, we want to make sure that the user doesn't exist already on the db.
Since we are probably using await engine.find_one, and we are using pydantic validators that do not support async, it is impossible to do such validation, and warn our user that the username already exists.

Solution

Add async validators on top of odmantic.

Alternative solutions

I don't have an alternative at the moment, if someone can point one out, I would be thankful.

Links in the contributing.md file with :target=blank_ displays the target=blank_ part and does not open in a new tab. For example:

If you want to contribute (thanks a lot ! 😃), you can open an issue{:target=blank_}.

I'm not sure of a way to make the {"target=blank_"} part work correctly so it seems like the best solution would be to remove it? If you would like I could open a PR removing the {:target-blank_} part.

from datetime import datetime

from odmantic import Model
from pydantic import EmailStr


class Subscriber(Model):
    email: EmailStr
    subscribed_at: datetime = datetime.utcnow()

Trying to make a timestamp field, but I get the error: Value not declarable with JSON Schema

When trying to call the Model.copy method, a NotImplementedError occurs.

So, it is hard to partially update models data.

Workaround:

obj = await engine.find_one(ObjModel, ObjModel.id == some_object_id)
data = {**obj.dict(), **other.dict(exclude_unset=True, exclude_none=True)}
obj = ObjModel(**data)

Here is simple example:

import asyncio
from abc import ABC
from typing import Optional

from motor.core import AgnosticCollection
from motor.motor_asyncio import AsyncIOMotorClient
from odmantic import Model, EmbeddedModel, Field, AIOEngine


class PersonalInfo(EmbeddedModel, ABC):
    first_name: Optional[str] = None
    last_name: Optional[str] = None


class User(Model, ABC):
    user_id: int
    personal_info: PersonalInfo = Field(default_factory=PersonalInfo)

    class Config:
        collection = 'test_collection'


motor_client = AsyncIOMotorClient('mongodb://localhost:27017/?readPreference=primary&ssl=false')
collection: AgnosticCollection = motor_client['test_db']['test_collection']
db_engine = AIOEngine(motor_client, database='test_db')


async def test():
    user = User(user_id=1)
    await db_engine.save(user)

    u1 = await db_engine.find_one(User, User.user_id == 1)
    u1.personal_info.first_name = 'Benyamin'
    await db_engine.save(u1)

    u2 = await db_engine.find_one(User, User.user_id == 1)

    print(u1)
    print(u2)


loop = asyncio.get_event_loop()
loop.run_until_complete(test())

And output:

id=ObjectId('5fcc19de04932eff2f00480b') user_id=1 personal_info=PersonalInfo(first_name='Benyamin', last_name=None)
id=ObjectId('5fcc19de04932eff2f00480b') user_id=1 personal_info=PersonalInfo(first_name=None, last_name=None)

I have a code like this:

from odmantic import Field, Model, EmbeddedModel, Reference
from typing import List

class Revision(EmbeddedModel):
    modified_by: "User" = Reference()

class User(Model):
    revisions: List[Revision]

It yields this error:

Traceback (most recent call last):
  File "..../api/m_example.py", line 4, in <module>
    class Revision(EmbeddedModel):
  File ".../lib/python3.8/site-packages/odmantic/model.py", line 443, in __new__
    mcs.__validate_cls_namespace__(name, namespace)
  File .../lib/python3.8/site-packages/odmantic/model.py", line 290, in __validate_cls_namespace__
    parse_obj_as(field_type, value)
  File "pydantic/tools.py", line 35, in pydantic.tools.parse_obj_as
  File "pydantic/main.py", line 360, in pydantic.main.BaseModel.__init__
  File "pydantic/main.py", line 955, in pydantic.main.validate_model
pydantic.errors.ConfigError: field "__root__" not yet prepared so type is still a ForwardRef, you might need to call ParsingModel[ForwardRef('User')].update_forward_refs().

Protection queries are a particular type of MongoDB queries where you can specify fields you want to get in the output.

Another example where the projection parameter is used:
db.writers.find( { "author": "Gaurav Mandes" }, { _id:0, author:1, title:1 } )
In the example above, the _id field is excluded, which automatically gets added, and the title and author fields are displayed.

Hello, I'm thinking to enable the aliases on the fields. It would enable a cleaner attribute validation to avoid conflicts between builtin methods and the fields for example (as already done by Pydantic).

The current behavior I'm thinking of:

• If a field has an alias and no key_name, the resulting document key will be the alias
• It's still possible to customize the key_name with an alias already set by specifying the key_name kwarg

Do you think some more customization/options would be useful ?

There's this Python Bitwise operator warning found in Documentation API Reference > odmantic.query which, in my humble opinion, should be added to Filtering > Logical Operators' section as well.

This will be a useful reminder to those who forget to add parenthesis when using bitwise operator comparisons and may overlook the API Reference section which explains why it's necessary to do so.

Description

Trying to implement odmantic (0.3.1), but get python hints:

Class Job must implement all abstract methods

My code

from odmantic import Model

class Job(Model):
    created: datetime
    state: str

Screenshots

Alse when I'm trying to add ABC methods I receive this one:

Summary

If this action is not nesessary, please remove abstract method mark.
If this action is nesessary, please, add it to examples

@art049 Great library looks very promising, but there is 2 missing features I think which are:

1. Model inheratance (maybe available but not documented)
2. Mongodb Migration
   And I can contibute if needed.

waiting model inherance... :)

Feature request

Context

Now the id field is printed in documents with information about the used database:
MongoDB ObjectId string

I'm not sure if showing this information is a good idea.
What kind of database and what kind of identifier generation are used to store the data is not a client's business.

Solution

I suggest removing the description of the id field. This will hide unnecessary information from the user.

Alternative solutions

It will also be great if odmantic will be able to specify a custom description and parameters for id field

Additional context

It would also be great to make the example look more similar.
I.e. replace ffffffffffffffffffffffff with 5f85f36d6dfecacc68428a46

Bug

Creating a model instance with datetime fields fail if microseconds is between 999500 and 999999.
This seems to be a rounding error.

Current Behavior

import datetime
from odmantic import Model

class MyModel(Model):
    date_time: datetime.datetime


instance = MyModel(date_time=datetime.datetime.now().replace(microsecond=999500))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "services/registry/.venv/lib/python3.8/site-packages/odmantic/model.py", line 481, in __init__
    super().__init__(**data)
  File "pydantic/main.py", line 362, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for MyModel
date_time
  microsecond must be in 0..999999 (type=value_error)

Expected behavior

No validation error should be raised.

I'm not sure if datetimes with microseconds > 999500 should be rounded up to the next second or not. Most implementations do not round, but just crop the last three digits of microseconds like these:

• now.isoformat(sep=' ', timespec='milliseconds')
• now.strftime("%H:%M:%S.%f")[:-3]

We could use the following expression to crop:

now = now.replace(microsecond=now.microsecond-now.microsecond % 1000)

Environment

• ODMantic version: 0.3.2
• MongoDB version: 4.4
• Pydantic: 1.7.:
• python: 3.8.5

print(pydantic.utils.version_info())
             pydantic version: 1.7.3
            pydantic compiled: True
                 install path: services/registry/.venv/lib/python3.8/site-packages/pydantic
               python version: 3.8.5 (default, Sep  7 2020, 19:43:22)  [Clang 10.0.0 (clang-1000.11.45.5)]
                     platform: macOS-10.14.6-x86_64-i386-64bit
     optional deps. installed: ['typing-extensions', 'email-validator']

hi friends!, is there any way i can optionally use odmantic with tinydb too?

Bug

When i try to reference other model in EmbeddedModel, i get full document inserted instead of reference to it

Current Behavior

I have these models:

class Need(Model):
    name: str


class PetNeed(EmbeddedModel):
    need: Need = Reference()
    value: int = Field(default=0)


class Pet(Model):
    name: str
    needs: List[PetNeed] = Field(default=[])

If i try to do this:

hunger = Need(name='Hunger')
dog = Pet(name='Dog')
dog.needs.append(PetNeed(need=hunger))
await engine.save_all([hunger, dog])

I get this in the database:

{
    "_id": {
        "$oid": "60062e817a596be396df8ab8"
    },
    "name": "Dog",
    "needs": [{
        "need": {
            "name": "Hunger",
            "id": {
                "$oid": "60062e817a596be396df8ab7"
            }
        },
        "value": 0
    }]
}

As you can see full model being inserted into the database with both name and id instead of just reference id to it

However everything is fine if i reference Need model directly:

rest = Need(name='Rest')
cat = Pet(name='Cat', need=rest)
await engine.save_all([rest, cat])

Database:

{
    "_id": {
        "$oid": "6006326f0092a86615eeae91"
    },
    "name": "Cat",
    "need": {
        "$oid": "6006326f0092a86615eeae90"
    }
}

Expected behavior

ODMantic should put reference id in EmbeddedModel instead of putting full Model in the field

Environment

• ODMantic version: 0.3.2
• MongoDB version: 4.2.11
• Pydantic infos (output of python -c "import pydantic.utils; print(pydantic.utils.version_info())):

pydantic version: 1.7.3
            pydantic compiled: True
                 install path: D:\_Projects\python\bot\py\Lib\site-packages\pydantic
               python version: 3.9.1 (tags/v3.9.1:1e5d33e, Dec  7 2020, 17:08:21) [MSC v.1927 64 bit (AMD64)]
                     platform: Windows-10-10.0.18362-SP0
     optional deps. installed: ['typing-extensions']

Additional context

It is also impossible to make reference fields Optional or put them in a List, but I read in the docs that it is planned to make a generic Reference[T] type so it's ok

cannot accesss to http://art049.github.io/odmantic

I think that would be good idea add support for a flat file like tinydb in odmantic for use it in little projects using nosql... somethig like tortoise-orm that provide support for sql databases including sqlite...

something like:
engine = AIOEngine(system="aiotinydb", database="Name")
or
engine = AIOEngine(system="Motor", database="Name")

Bug

ODMantic can't save document to database when primary field of a document is EmbeddedModel.

Current Behavior

When defining EmbeddedModel as a primary field of a Model:

class Id(EmbeddedModel):
    user: int
    chat: int


class User(Model):
    id: Id = Field(primary_field=True)

ODMantic refuses to save the document into database:

user = User(id=Id(user=1, chat=1001))
await engine.save(user)

It either does nothing as if operation was successful, or throws an error:

bson.errors.InvalidDocument: cannot encode object: Id(user=1, chat=1001), of type: <class '__main__.Id'>

However everything is ok if i do this:

await engine.get_collection(User).insert_one(user.doc())

Also everything is good with queries:

user = await engine.find_one(User)
print(user) 
print(user.id)
print(user.id.chat)

# id=Id(user=1, chat=1001)
# user=1 chat=1001
# 1001

Expected behavior

ODMantic should work fine with EmbeddedModel as primary field of a Model

Environment

• ODMantic version: 0.3.2
• MongoDB version: 4.2.11
• Pydantic infos (output of python -c "import pydantic.utils; print(pydantic.utils.version_info())):

pydantic version: 1.7.3
            pydantic compiled: True
                 install path: D:\_Projects\python\bot\py\Lib\site-packages\pydantic
               python version: 3.9.1 (tags/v3.9.1:1e5d33e, Dec  7 2020, 17:08:21) [MSC v.1927 64 bit (AMD64)]
                     platform: Windows-10-10.0.18362-SP0
     optional deps. installed: ['typing-extensions']

Python: 3.7
code

from fastapi import FastAPI
from motor.motor_asyncio import AsyncIOMotorClient
from odmantic import AIOEngine, Model

class User(Model):
    name: str


client = AsyncIOMotorClient("mongodb://haas-authservice-db:27017/")
engine = AIOEngine(motor_client=client, database="auth-service")

app = FastAPI()

motor_collection = engine.get_collection(User)
print(motor_collection)

Error

AsyncIOMotorCollection(Collection(Database(MongoClient(host=['haas-authservice-db:27017'], document_class=dict, tz_aware=False, connect=False, driver=DriverInfo(name='Motor', version='2.3.0', platform='asyncio')), 'auth-service'), 'user'))

Process SpawnProcess-37:

Traceback (most recent call last):

File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap

self.run()

File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run

self._target(*self._args, **self._kwargs)

File "/venv/lib/python3.7/site-packages/uvicorn/subprocess.py", line 61, in subprocess_started

target(sockets=sockets)

File "/venv/lib/python3.7/site-packages/uvicorn/main.py", line 419, in run

loop.run_until_complete(self.serve(sockets=sockets))

File "/usr/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete

return future.result()

File "/venv/lib/python3.7/site-packages/uvicorn/main.py", line 426, in serve

config.load()

File "/venv/lib/python3.7/site-packages/uvicorn/config.py", line 328, in load

self.loaded_app = import_from_string(self.app)

File "/venv/lib/python3.7/site-packages/uvicorn/importer.py", line 20, in import_from_string

module = importlib.import_module(module_str)

File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module

return _bootstrap._gcd_import(name[level:], package, level)

File "<frozen importlib._bootstrap>", line 1006, in _gcd_import

File "<frozen importlib._bootstrap>", line 983, in _find_and_load

File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked

File "<frozen importlib._bootstrap>", line 677, in _load_unlocked

File "<frozen importlib._bootstrap_external>", line 728, in exec_module

File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed

File "/app/app.py", line 7, in <module>

from endpoints import router, conf

File "/app/endpoints.py", line 8, in <module>

from models import *

File "/app/models.py", line 34, in <module>

class User(Model):

File "/venv/lib/python3.7/site-packages/odmantic/model.py", line 424, in __new__

return super().__new__(mcs, name, bases, namespace, **kwargs)

File "/venv/lib/python3.7/site-packages/odmantic/model.py", line 363, in __new__

field.bind_pydantic_field(cls.__fields__[name])

KeyError: '_id'

Hi cool library.
I have a question, perhaps a stupid one.

How to make your own string values id?
I want nice to get such a tree structure.

https://docs.mongodb.com/manual/tutorial/model-tree-structures-with-parent-references/

db.categories.insertMany( [
   { _id: "MongoDB", parent: "Databases" },
   { _id: "dbm", parent: "Databases" },
   { _id: "Databases", parent: "Programming" },
   { _id: "Languages", parent: "Programming" },
   { _id: "Programming", parent: "Books" },
   { _id: "Books", parent: null }
] )

class Catalog(Model):
    id: str
    parent: Optional[str]


TypeError: can't automatically generate a primary field since an 'id' field already exists

Hello. Thanks for this great piece of software. The docs seems to me quite complete. My congrats.

I considering to use odmantic and I would like to store files in MongoDB. Could you provide an example? What do I have to take into account?

Thanks in advance.

Is there a JsonField for any json like PostgresSQL jsonb?

no need schema, but for filter and update

Bug

A clear and concise description of what the bug is.

Assigning default timestamp via default_factory includes a Z at the end.
tree_api.zip

Current Behavior

I created the test program tree_api.py from https://art049.github.io/odmantic/usage_fastapi/ and enhanced it by a updated-field (see attachment).

Then I start uvicorn via: uvicorn tree_api:app

Then I open the URL: http://127.0.0.1:8000/docs#/default/create_tree_trees__put and test the creation of a new tree object via Try it out on PUT /trees/

the generated JSON string is:
{
"name": "string",
"average_size": 0,
"discovery_year": 0,
"updated": "2021-02-12T11:22:55.542Z",
"id": "ffffffffffffffffffffffff"
}

Note: the updated-attribute has a timestamp with a last character Z. If I change the name and id and execute it I got an error message "**datetime objects must be naive (no timeone info)
tree_api.zip

**"

In detail:
{
"detail": [
{
"loc": [
"body",
"updated"
],
"msg": "datetime objects must be naive (no timeone info)",
"type": "value_error"
}
]
}

If I remove the last character Z, it works fine!

Expected behavior

... _A clear and concise description of what you expected to happen.

I expected, that the timestamp did not have the Z at the end and that it works directly.

Environment

• ODMantic version: 0.3.2

• FastApi: 0.63.0

• MongoDB version: MongoDB shell version v4.2.5

• Pydantic infos (output of python -c "import pydantic.utils; print(pydantic.utils.version_info())): 1.7.3

• Python 3.9.1

• Version of additional modules (if relevant):

Additional context

Add any other context about the problem here.

Allow migration to a new schema for mongodb

Bug

In the _save() method transaction's session is not passed to update_one() request.

It means that transaction is NOT USED NOW!

How to fix?

It's not enough to just pass the session to request.
In MongoDB transactions are not available for Standalone.
This simple way Standalone users will get exceptions for every save() call 😢

So we need to define the users of Standalone version and not start a transaction for them.
Example:

if self.client._server_property('server_type') == SERVER_TYPE.Standalone:
    await self._save(instance, None, **kwargs)
else:
    async with await self.client.start_session() as s:
        async with s.start_transaction():
            await self._save(instance, s, **kwargs)

P.S.: I don't know how to properly check ServerType without protected _server_property method :(

Feature request

Context

can we define index information when define a model?

I added a new field on the existing model. When I try to query, I hit KeyError due to missing attribute in all existing records/documents. I set the new field as optional and it still failed.

backend_1       |   File "/usr/local/lib/python3.7/site-packages/odmantic/model.py", line 547, in parse_doc
backend_1       |     doc[field_name] = raw_doc[field.key_name]
backend_1       | KeyError: 'location'

I updated the documents to add the field manually using mongo shell. the GET command worked again so clearly, adding new fields is not fully supported now. There should be a migration which we try to avoid.

Support for tailable cursors in engine.find call:

https://motor.readthedocs.io/en/stable/examples/tailable-cursors.html

Suggested implementation of $elemMatch $all and $size

MongoDB Reference

Example model:

class Thing(Model):
    array: List[int]

$elemMatch

Array containing the number 42 at least once:

Thing.array.any().eq(42)
eq(Thing.array.any(), 42)
Thing.array.any() == 42
42 in Thing.array # if possible using __contains__

$elemMatch

Array containing only elements greater than 12:

Thing.array.all().gt(42)
gt(Thing.array.all(), 42)
Thing.array.all() > 42

$size

Array containing more than 3 elements:

Thing.array.size().gte(3)
gte(Thing.array.size(), 3)
Thing.array.size() >= 3
len(Thing.array) >= 3 # if possible

I am using FastAPI and odmantic to access an existing MongoDB data source. I have constructed an odmantic Model for an existing collection for users:

class User(Model):
    __collection__ = "users"
    name: str = Field(max_length=50)
    email: str = Field(max_length=150)  # unique=True
    password: bytes
    logged_in: Optional[datetime.datetime]

When a User record is created, the logged_in property will be null/None.

In a pure odmantic environment, the logged_in property would still be persisted to the MongoDB document with a null/None value.

However, mongoengine by default an optional field will not set at all, so for the model above the logged_in field will not appear in the document at all.

This means that when odmantic encounters users that have never logged in, an error is raised:

File "~/lib/python3.8/site-packages/odmantic/model.py", line 547, in parse_doc
  doc[field_name] = raw_doc[field.key_name]
KeyError: 'logged_in'

As accessing existing data sources is likely to be a common scenario with odmantic, it would be good to determine how best to handle this, both for reading and writing data.

Exception while saving an object containing either a frozenset or a set:

from odmantic import Model, AIOEngine
from odmantic.bson import ObjectId

class UserModel(Model):
    username: str
    favorites_article_ids: FrozenSet[ObjectId] = frozenset()

engine = AIOEngine()
await engine.save(UserModel(username="jean")) 

Here is simple example:

from abc import ABC
from typing import List

from odmantic import Model, EmbeddedModel, Field


class Address(EmbeddedModel, ABC):
    address: str


class TestUser(Model, ABC):
    addresses: List[Address]


user = TestUser(addresses=[Address(address='Israel')])
user.addresses[0].address = 'Russia'  # will raise AttributeError: __fields_modified__

----------------------------------

Traceback (most recent call last):
  File "C:/Users/Benyamin/etc/zmanim_bot/zmanim_bot/api/test.py", line 16, in <module>
    user.addresses[0].address = 'Russia'  # will raise AttributeError: __fields_modified__
  File "C:\Users\Benyamin\.virtualenvs\zmanim_bot-luq9P_0V\lib\site-packages\odmantic\model.py", line 507, in __setattr__
    self.__fields_modified__.add(name)
AttributeError: __fields_modified__

Dependabot can't evaluate your Python dependency files.

As a result, Dependabot couldn't check whether any of your dependencies are out-of-date.

The error Dependabot encountered was:

Illformed requirement [">=1.6.0 < 1.8.0"]

View the update logs.

I'm considering separating interface classes and persistence classes. Interface classes are the ones used in OpenAPI, i.e. in request / responses. Persistence classes are used when storing or retrieving objects from the store.

I would like to reuse pydantic models to create odmantic models, e.g. schemas.py (interface classes)

from pydantic import BaseModel, Field

class Publisher(BaseModel):
    name: str
    founded: int = Field(ge=1440)
    location: Optional[str] = None

And in models.py (persistence classes):

from odmantic import AIOEngine, Model
from schemas import Publisher

engine = AIOEngine()

class PublisherDB(Model, Publisher):
    pass

Although the metaclass does it's work, it seems that something does go wrong, because:

from models import PublisherDB, engine

engine.find(PublisherDB)
In [3]: res = await engine.find(PublisherDB, {})
---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
~/.local/share/virtualenvs/registry-KQbdKffM/lib/python3.8/site-packages/odmantic/model.py in parse_doc(cls, raw_doc)
    559         try:
--> 560             instance = cls.parse_obj(obj)
    561         except ValidationError as e:

~/.local/share/virtualenvs/registry-KQbdKffM/lib/python3.8/site-packages/pydantic/main.cpython-38-x86_64-linux-gnu.so in pydantic.main.BaseModel.parse_obj()

~/.local/share/virtualenvs/registry-KQbdKffM/lib/python3.8/site-packages/odmantic/model.py in __init__(self, **data)
    475     def __init__(self, **data: Any):
--> 476         super().__init__(**data)
    477         object.__setattr__(self, "__fields_modified__", set(self.__odm_fields__.keys()))

~/.local/share/virtualenvs/registry-KQbdKffM/lib/python3.8/site-packages/pydantic/main.cpython-38-x86_64-linux-gnu.so in pydantic.main.BaseModel.__init__()

ValidationError: 2 validation errors for PublisherDB
name
  field required (type=value_error.missing)
founded
  field required (type=value_error.missing)

During handling of the above exception, another exception occurred:

DocumentParsingError                      Traceback (most recent call last)
<ipython-input-3-b4c923edf0b7> in <module>
----> 1 res = await engine.find(PublisherDB, {})

~/.local/share/virtualenvs/registry-KQbdKffM/lib/python3.8/site-packages/odmantic/engine.py in __await__(self)
     64         instances = []
     65         for raw_doc in raw_docs:
---> 66             instances.append(self._parse_document(raw_doc))
     67             yield
     68         self._results = instances

~/.local/share/virtualenvs/registry-KQbdKffM/lib/python3.8/site-packages/odmantic/engine.py in _parse_document(self, raw_doc)
     54 
     55     def _parse_document(self, raw_doc: Dict) -> ModelType:
---> 56         instance = self._model.parse_doc(raw_doc)
     57         object.__setattr__(instance, "__fields_modified__", set())
     58         return instance

~/.local/share/virtualenvs/registry-KQbdKffM/lib/python3.8/site-packages/odmantic/model.py in parse_doc(cls, raw_doc)
    560             instance = cls.parse_obj(obj)
    561         except ValidationError as e:
--> 562             raise DocumentParsingError(
    563                 errors=e.raw_errors,  # type: ignore
    564                 model=cls,

DocumentParsingError: 2 validation errors for PublisherDB
name
  field required (type=value_error.missing)
founded
  field required (type=value_error.missing)
(PublisherDB instance details: id=ObjectId('5fc4e4c38c1d0709f79f1684'))

Support for the new version of the Motor library.

like in the documentation, everything below works out fine, the problem is if I want to use Publisher.name and Book.title as primary_field=True...

class Publisher(Model):
    name: str = Field(primary_field=True)
    ...
    ...

class Book(Model):
    title: str = Field(primary_field=True)
    ....
    ...

hachette = Publisher(name="Hachette Livre", founded=1826, location="FR")
harper = Publisher(name="HarperCollins", founded=1989, location="US")

books = [
    Book(title="They Didn't See Us Coming", pages=304, publisher=hachette),
    Book(title="This Isn't Happening", pages=256, publisher=hachette),
    Book(title="Prodigal Summer", pages=464, publisher=harper),
]

await engine.save_all(books)

the output

python test.py

2- Creating books and Publisher...

success:  False 
error: 
 'id'
'Publisher' object has no attribute 'id'

but it works fine if I change:

class Publisher(Model):
    id: str = Field(primary_field=True)
    ...
    ...

class Book(Model):
    id: str = Field(primary_field=True)

this is not a big problem but maybe this can be fixed

Bug

datetime field works fine with pydantic
But when I use datetime field in odmantic models - openapi.json creation crashes :(

Traceback

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 396, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/usr/local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/fastapi/applications.py", line 190, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/starlette/applications.py", line 111, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 181, in __call__
    raise exc from None
  File "/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.8/site-packages/starlette/exceptions.py", line 82, in __call__
    raise exc from None
  File "/usr/local/lib/python3.8/site-packages/starlette/exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 566, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 227, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 41, in app
    response = await func(request)
  File "/usr/local/lib/python3.8/site-packages/fastapi/applications.py", line 143, in openapi
    return JSONResponse(self.openapi())
  File "/usr/local/lib/python3.8/site-packages/fastapi/applications.py", line 121, in openapi
    self.openapi_schema = get_openapi(
  File "/usr/local/lib/python3.8/site-packages/fastapi/openapi/utils.py", line 348, in get_openapi
    definitions = get_model_definitions(
  File "/usr/local/lib/python3.8/site-packages/fastapi/utils.py", line 25, in get_model_definitions
    m_schema, m_definitions, m_nested_models = model_process_schema(
  File "pydantic/schema.py", line 525, in pydantic.schema.model_process_schema
  File "pydantic/schema.py", line 566, in pydantic.schema.model_type_schema
  File "pydantic/schema.py", line 222, in pydantic.schema.field_schema
  File "pydantic/schema.py", line 476, in pydantic.schema.field_type_schema
  File "pydantic/schema.py", line 821, in pydantic.schema.field_singleton_schema
ValueError: Value not declarable with JSON Schema, field: name='created' type=_datetime required=True

How to reproduce

Just add datetime field to your tree example from odmantioc docs.
E.g.: birth: datetime
Then launch fastapi and open /docs or /redoc

from typing import List
from datetime import datetime

from fastapi import FastAPI, HTTPException

from odmantic import AIOEngine, Model, ObjectId


class Tree(Model):
    name: str
    average_size: float
    discovery_year: int
    birth: datetime


app = FastAPI()

engine = AIOEngine()


@app.put("/trees/", response_model=Tree)
async def create_tree(tree: Tree):
    await engine.save(tree)
    return tree


@app.get("/trees/", response_model=List[Tree])
async def get_trees():
    trees = await engine.find(Tree)
    return trees


@app.get("/trees/count", response_model=int)
async def count_trees():
    count = await engine.count(Tree)
    return count


@app.get("/trees/{id}", response_model=Tree)
async def get_tree_by_id(id: ObjectId):
    tree = await engine.find_one(Tree, Tree.id == id)
    if tree is None:
        raise HTTPException(404)
    return tree

Hi!

Can I use pydantic config parameter allow_population_by_field_name with odmantic models?

Bug

modified mark is not cleared for embedded models on save

Current Behavior

On save() method __fields_modified__ set is cleared only for the top-level instance.

Expected behavior

On save() method __fields_modified__ set should be cleared for the entire tree of embedded instances.

How can i do this?

class Gateway(Model):
    serial_number: str
    name: str
    ip: IPv4Address
    devices: Optional[List[PeripheralDvice]] = Field(max_items=10)

    class Config:
        json_encoders = {IPv4Address: lambda v: str(v)}

from odmantic import Model

class User(Model):
    scopes: list[str]


print(User(scopes=[]))

Caused errors like this:

Traceback (most recent call last):
  File "C:\xxxx\main.py", line 3, in <module>
    class User(Model):
  File "C:\xxxx\lib\site-packages\odmantic\model.py", line 381, in __new__
    mcs.__validate_cls_namespace__(name, namespace)
  File "C:\xxxx\lib\site-packages\odmantic\model.py", line 219, in __validate_cls_namespace__
    substituted_type = validate_type(field_type)
  File "C:\xxxx\lib\site-packages\odmantic\model.py", line 176, in validate_type
    setattr(type_, "__args__", new_arg_types)
AttributeError: readonly attribute

Hi! First of all, thank you for sharing your amazing project. I've already started using it in my code.

I noticed two mistakes in the Raw Query Usage section of the documentation:

1. Creating instances from a raw documents states that the parse_doc method can be used to create new instance from a raw document. However, the example code shows the extraction of document from an existing instance.

2. Similarly, section Extract documents from existing instances states that the doc method can be used to extract document from existing instance. However, the example code shows the creation of a new instance from a raw document.

Upon investigation, I found that the code snippets had been referenced correctly in raw_query_usage.md but the code snippet files themselves had been incorrectly named.

To fix this:

• I renamed the create_from_raw.py to extract_from_existing.py (this will also remove any ambiguity from the name).
• Then, I renamed the extract_from_raw.py to create_from_raw.py.
• Lastly, I update the raw_query_usage.md to reference the newly renamed files.

You can view my commits here. If you think it's good enough, I can send in a pull request.

PS: I noticed at least one typo, but those are fine by me. ;) If you want, I can fix the typos before I send in a pull request.

Desctription

Seems that inserting via engine requires some additional permissions.
Please describe it in the docs or (better) decrease permissions level similar to the motor way.

My code

async def save_via_motor(tree: Model):
    client = AsyncIOMotorClient(MONGO_URI)
    db = client.get_database('forest')
    collection = db.get_collection('trees')
    await collection.insert_one(tree.dict())

async def save_via_odmantic(tree: Model):
    client = AsyncIOMotorClient(MONGO_URI)
    engine = AIOEngine(motor_client=client , database='forest')
    await engine.save(tree)

Expected behavior

Bot functions works the same way

In fact

Saving via engine occurs exception:

 pymongo.errors.OperationFailure: not authorized on forest to execute command { update: "trees", ordered: true, writeConcern: { w: "majority" }, bypassDocumentValidation: true, lsid: { id: UUID("ea4686c2-7ae0-47da-a484-09b9d9c51ae9") }, txnNumber: 1, $clusterTime: { clusterTime: Timestamp(1607689696, 16), signature: { hash: BinData(0, A637C05EC63A4FB3EF374716F3FB175FDA5CDA30), keyId: 6866034364808007110 } }, $db: "trees", $readPreference: { mode: "primary" } }, full error: {'operationTime': Timestamp(1607689695, 32), 'ok': 0.0, 'errmsg': 'not authorized on trees to execute command { update: "trees", ordered: true, writeConcern: { w: "majority" }, bypassDocumentValidation: true, lsid: { id: UUID("ea4686c2-7ae0-47da-a484-09b9d9c51ae9") }, txnNumber: 1, $clusterTime: { clusterTime: Timestamp(1607689695, 16), signature: { hash: BinData(0, A637C05EC63A4FB3EF374716F3FB175FDA5CDA40), keyId: 6866037364809007110 } }, $db: "trees", $readPreference: { mode: "primary" } }', 'code': 13, 'codeName': 'Unauthorized', '$clusterTime': {'clusterTime': Timestamp(1607689695, 32), 'signature': {'hash': b'\xa67\xc0^\xc6:O\xb3\xef7G\x16\xf3\xfb\x17_\xda\\\xda@', 'keyId': 6866034364808007110}}}

I already read the document, yet i cann't find a way to sort the results

PyCharm does not see the asc and desc methods in model fields.

MyModel.timestamp.asc() displayed as error.

query.asc(MyModel.timestamp) looks fine.

Of course both methods work fine.

Motor supports additional options for databases

def get_database(self, name=None, codec_options=None, read_preference=None,
                     write_concern=None, read_concern=None):
    ...

I always use feature with setting read_preference=ReadPreference.SECONDARY_PREFERRED.
It's really helps with highload apps.

Why don't you pass this options to AIOEngine init?