octabytes / fireo

Google Cloud Firestore modern and simplest convenient ORM package in Python. FireO is specifically designed for the Google's Firestore

A modern and simplest convenient ORM package in Python. FireO is specifically designed for the Google's Firestore, it's more than just ORM. It implements validation, type checking, relational model logic and much more facilities.

Get Started!

Available in other language

1. FireO is available also in nodeJS FireO nodeJS

Installation

pip install fireo

Example Usage

from fireo.models import Model
from fireo.fields import TextField

class User(Model):
    name = TextField()


u = User()
u.name = "Azeem Haider"
u.save()

# Get user
user = User.collection.get(u.key)
print(user.name)

With Typed Model

from fireo.typedmodels import TypedModel

class User(TypedModel):
    name: str
    age: int

# Use the model as usual:
user = User(name='John', age="30")
user.save()

print(user.to_dict())

Documentation

Full documentation is available in the FireO Doc.

Contributing

Bug reports and pull requests are welcome. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.

1. Fix bug or add new features
2. Write tests for your functionality
3. Mention in Documentation, what you have done and how others can use it

To run the tests while developing on this package, you'll have to setup a Google service account and setup credentials with the following command:

export GOOGLE_APPLICATION_CREDENTIALS="KEY_PATH"

See the Google Cloud documentation for more details.

Code Contributors

This project exists thanks to all the people who contribute. [Contribute].

License

This is official FireO Repository. Powered by OctaByte Licensed under Apache License 2.0

fireo's Issues

Updating boolean value to False does not work

class User(Model):
online = BooleanField()

u = User()
u.save()
user_key = u.key

u = User()
u.online = False
u.update(user_key)

Gives the following error:
Traceback (most recent call last):
File "/Users/dafzal/code/insurita/backend_prototype/firebase_test.py", line 35, in
u.update(user_key)
File "/Users/dafzal/code/insurita/venv/lib/python3.7/site-packages/fireo/models/model.py", line 419, in update
return self.class.collection._update(self, transaction=transaction, batch=batch, **updated_fields)
File "/Users/dafzal/code/insurita/venv/lib/python3.7/site-packages/fireo/managers/managers.py", line 206, in _update
return self.queryset.update(mutable_instance, transaction, batch, **kwargs)
File "/Users/dafzal/code/insurita/venv/lib/python3.7/site-packages/fireo/queries/query_set.py", line 80, in update
return UpdateQuery(self.model_cls, mutable_instance, **kwargs).exec(transaction_or_batch)
File "/Users/dafzal/code/insurita/venv/lib/python3.7/site-packages/fireo/queries/update_query.py", line 90, in exec
return query_wrapper.ModelWrapper.from_query_result(self.model, self._raw_exec())
File "/Users/dafzal/code/insurita/venv/lib/python3.7/site-packages/fireo/queries/update_query.py", line 83, in _raw_exec
ref.update(self._parse_field())
File "/Users/dafzal/code/insurita/venv/lib/python3.7/site-packages/google/cloud/firestore_v1/document.py", line 381, in update
batch.update(self, field_updates, option=option)
File "/Users/dafzal/code/insurita/venv/lib/python3.7/site-packages/google/cloud/firestore_v1/batch.py", line 112, in update
reference._document_path, field_updates, option
File "/Users/dafzal/code/insurita/venv/lib/python3.7/site-packages/google/cloud/firestore_v1/_helpers.py", line 862, in pbs_for_update
raise ValueError("Cannot update with an empty document.")
ValueError: Cannot update with an empty document.

I think the problem is in _parse_field
if v:
field_dict[f.db_column_name] = v

^ means you cant update a model for any v where bool(v) = False

Optimize code for FireO

Review the code if any method you can optimize. Write better code for this

Adding default arg to Field overwrites value that already exists.

It appears that when you add a default to a field, even if that field exists in the DB, it will be overwritten by the default value.

For example:

class BillingHistory(Model):
id = IDField()
amount = NumberField(required=True, default=15)

b = BillingHistory(id = '123', amount = 11)
b.save() # amount == 11 in Firestore
b.update() # amount == 15 in Firestore (but should have stayed to 11 since it existed)

Meta.missing_field = merge...

When using the Meta missing_field = merge. I noticed that the merge setting does not update the cls._meta.field_list dict. This causes two issues:

• cls._get_fields() will not include the merged fields.
• cls.save()/cls.update() will not save any changes on the merged fields.

Also, I see no logic to determine what the Field type should be for the merged field--is this an oversight? Any clarification will be greatly appreciated.

Field validation prevents model updating

Model can't be updated if its fields contains validators (e.g. int_only, max_length, etc...).

Example:

class Person(Model):
    name = fields.TextField(required=True)
    type = fields.NumberField(required=True, int_only=True)

person = Person.collection.create(name="test", type=1)
person.name = "test2"
person.update()  # fireo.fields.errors.InvalidFieldType: Invalid field type. Field "type" expected <class 'int'> type, got <class 'NoneType'>

Collection group queries

Allow FireO to support collection group queries.

Simultaneous connections to multiple databases

How can one have simultaneous connections with multiple credentials to multiple databases? The documentation says this is possible: https://octabyte.io/FireO/authentication/#explicit-credentials

However, the db member is a global variable for the whole the module. Presumably, I would need multiple instances of the Database class, each with different credentials.

Connect From Google Colab Notebook

Hi,

I'm testing out here a Google Colab (https://colab.research.google.com/) and wanted to connect to Firebase using your ORM library.
Can you please correct my code and see what I'm doing wrong, I'm just trying to connect using a service account.

!pip install fireo
from fireo.models import Model
from fireo.fields import TextField
from fireo.database import connection
connection(from_file="crendentials.json")

This is the error I get on the connection line:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/fireo/database/database.py in connect(self, credentials, from_file)
     41             elif from_file:
---> 42                 self._conn = firestore.Client.from_service_account_json(from_file)
     43             else:

/usr/local/lib/python3.6/dist-packages/google/cloud/client.py in from_service_account_json(cls, json_credentials_path, *args, **kwargs)
     85         kwargs["credentials"] = credentials
---> 86         return cls(*args, **kwargs)
     87 

/usr/local/lib/python3.6/dist-packages/google/cloud/firestore_v1/client.py in __init__(self, project, credentials, database, client_info, client_options)
    109         super(Client, self).__init__(
--> 110             project=project, credentials=credentials, _http=None
    111         )

/usr/local/lib/python3.6/dist-packages/google/cloud/client.py in __init__(self, project, credentials, client_options, _http)
    249         _ClientProjectMixin.__init__(self, project=project)
--> 250         Client.__init__(self, credentials=credentials, client_options=client_options, _http=_http)

/usr/local/lib/python3.6/dist-packages/google/cloud/client.py in __init__(self, credentials, _http, client_options)
    135 
--> 136         if credentials and client_options.credentials_file:
    137             raise google.api_core.exceptions.DuplicateCredentialArgs(

AttributeError: 'ClientOptions' object has no attribute 'credentials_file'

The above exception was the direct cause of the following exception:

DBConnectionError                         Traceback (most recent call last)
<ipython-input-64-6a1afad9d26a> in <module>()
----> 1 connection(from_file="cred/infnet-f3892-firebase-adminsdk-ku8cj-d30235d835.json")

/usr/local/lib/python3.6/dist-packages/fireo/database/database.py in connect(self, credentials, from_file)
     44                 raise DBConnectionError("Credentials or service account json file required to connect with firestore")
     45         except Exception as e:
---> 46             raise DBConnectionError(e) from e
     47 
     48     @property

DBConnectionError: 'ClientOptions' object has no attribute 'credentials_file'

Provide a better documentation

Hi ,

First of all, thank you creating a ORM library for Firebase!

Appreciate if you can provide better documentation? Like how to inherit the Model class?

Like how do you import the TextField and etc. I mainly have to figure it out by diving into your tests and have a look how its implemented. It will be better if you can have a clearer documentation for at least the basic steps of creating an ORM.

Cheers

Generate pydoc for FireO

Every class and method are highly documented.
Use Sphinx to generate doc.

Add link to FireO Documentation

Auto Local time in DateTime Filed

DateTime field allow auto attribute to set the date automatically but this is server side date time.
Allow user to set auto date time locally.

Attribute name can be local_time possible values True or False

Quick start docs (https://octabyte.io/quick-start) does not exist

I stumbled into this project because of spam some mail, and this project seems to be nice. However, when I looked at the docs, https://octabyte.io/quick-start gives 404.

Maybe you didn't wrote the quick start guide?

Transaction isn't working with OOP style

Error: "TypeError: insert_in_transaction() missing 1 required positional argument: 'user'."

The code snippet is given below:

'''
from fireo.models import Model
from fireo.fields import IDField, TextField, NumberField
import fireo

  transaction = fireo.transaction()
  
  
  class UserTransac(Model):
      id = IDField()
      name = TextField()
      age = NumberField()
  
  
  class TransecTest:
      def __init__(self):
          self.model = UserTransac()
  
      @fireo.transactional
      def insert_in_transaction(self, transaction, user):
          self.model.from_dict(user).save(transaction=transaction)
  
  
  def main():
      user = {'name': "user.name", 'age': 88}
      TransecTest().insert_in_transaction(transaction, user)
  
  
  main()

"'

ListField of Models?

How can I save a ListField of previously defined models if I don't want to create a collection or a reference id for them?

Consider changing license from GPL to LGPL

Many Python projects do this to ensure they can be used within non-Open Source projects while still having protections from being forked and redistributed by others for money. (See Paramiko, for example).

That said, the project looks great and I'd almost certainly end up using it given this change. It's something that's desperately needed by the Firebase team.

Write Good Test for FireO

Test are already written for FireO these are not very good But they test all the require functionality.
Write more test with best practices

Default field not working for .update()

Now .update() does not overwrite value if already in Firestore, but it also does not write the default value if it is not in Firestore.

Example:

class Student(Model):
    name = TextField(required=True, default="Joe")

student = Student()
student.save() # name == Joe in DB
print(student.key) # students/KkdODGBGd3HrJN7boYoK

Now let's say you add a required field to the Student class:

class Student(Model):
    name = TextField(required=True, default="Joe")
    favoriteColor = TextField(required=True)
    age = NumberField(required=True, default=10)

new_student = Student.collection.get(student.key)
new_student.favoriteColor = 'red'
new_student.update() # age is not saved to Firestore even though it should save as 10

NestedModel fields throw exception when value is None

Model:

class Car(Model):
  make = TextField()

class Person(Model):
  favorite_car = NestedModel(Car, required=False)  

If you try to create a Person object and set favorite_car to None:

person = Person(favorite_car=None)

The following exception is thrown:

File "/usr/local/lib/python3.8/site-packages/fireo/models/model.py", line 142, in init
setattr(self, f.name, f.nested_model.from_dict(kwargs[f.name]))

File "/usr/local/lib/python3.8/site-packages/fireo/models/model.py", line 149, in from_dict
return cls(**model_dict)

TypeError: ModelMeta object argument after ** must be a mapping, not NoneType

The required attribute should check the field for None as well as the fields of type the NestedModel is based off of (which it already does). If required is True, and the NestedModel field is None, should through a more informative exception

Is it hard to implement set with merge=True option when saving a document?

Sorry if I put this as a question,
not a real issue.
I saw a precedent closed issue for upsert.
I know FireO save() is for creating, and update() is for updating, and currently there is no upsert available.
But with official firestore libraries is possible, by simply putting merge=True when saving.
Would it be too hard to simply add merge option to save() method, and on set() call inside save()?

I noticed update make a query first, so it's not atomic and not efficient, 2 operations instead of one.

(For me update() could even become useless, does not matter for me having an error if document is not present).

Again: thanks for the great job on this!

Can I perform an upsert?

Is it possible to perform an upsert with fireo, and if so, could you provide a quick example of how to do so?

How to set TextField as unique?

Pardon my ignorance, I tried finding it in the document. Is it possible to do that?

Internal field names being added to Model.field_list

On query the Model.field_list is being populated with the following internal fields:

'parent', '_id', 'id', '_key', 'update_doc'

My understanding is the Model.field_list should only include the fields in the document.

Reserved field names...

In the Model implementation you list a number of names as being reserved (see https://octabyte.io/FireO/reserved-words/).

This is problematic for a number of reasons, such as the reserved names are not being checked when missing_field = merge setting causes errors on collisions. I would like to propose an alternative naming/solution:

If you want to keep the property access for the Model then change your reserved keywords to be of the form: __.*__ as Firestore does not allow fields with those names. This will prevent collisions. For example, _meta would be __.meta__.

Cursor pagination bug for datetime and other non JSON serializable types

I recently came across this problem when implementing pagination and filtering by a fireo.fields.DateTime attribute.

For example:

# def Task(Model):
#       created_at: datetime = DateTime(auto=True)

tasks = Task.collection.filter("created_at", "<=", datetime.now()).order("created_at").fetch(10)
tasks.cursor # throws an error because datetime is not JSON serializable

The error is happening at fireo/queries/query_iterator.py", line 78, in cursor

encodedCursor = base64.b64encode(json.dumps(self.query.cursor_dict).encode('utf-8'))

question about firebase collection map mapping

I was trying to insert data into a collection called UserData however when I set the collection name called UserData it keeps creating collection called user_data. Is there anyway to set the name correctly? Thanks.

Paginating data with query cursors

With query cursors in Cloud Firestore, you can split data returned by a query into batches according to the parameters you define in your query.

Read More

Dynamic collection name

Please I would like to know if is possible to create a collection name dynamically based on external parameter (IDs, etc.. )
For example a model "Event" that allow a parameter to make a collection_name = 'event_'+str(id)

Thanx

Cursor implementation is using eval() method

Query cursor is a really powerful tool to implement pagination of a large dataset. A possible use case of this feature in API is to return the next page cursor value with response data.

But with the current implementation, it is not safe to do this, because the cursor string is passed to eval() method which is a security flaw.

Consider using json or pickle module to encode/decode filter options for queryset, that are stored in the cursor.

Another issue may be with the hacky user, that will change filter/limit/order values inside cursor, that will expose the whole collection data

Getting related document

************* NOT AN ISSUE JUST CLERIFICATION NEEDED ****************
This is my first Python ORM, so forgive my naivety.

I have 2 collections in my firestore.
collection 1 houses post
collection 2 houses authors

In each document in collection 1(post) there is an author_id to match to author in collection 2.

Now I am doing a query to fetch all documents in post, I want a relation to get the username of the author from the collection 2(authors) using the matching uid in collection post and uid in collection authors.

something of select_related() and prefetch_related() as in django
How can this be achieved?
Thank you

Quering and filtering slowing down view render

A query like the one below, drastically slows down my view render.

What would be the way to do such a query?

 @classmethod
    def get_updates(cls):
        questions = cls.collection.filter('status', '==', 'online').filter('type', '==', 'question').order('-updatedAt').fetch(4)
        products = cls.collection.filter('status', '==', 'online').filter('type', '==', 'market').order('-updatedAt').fetch(4)
        jobs = cls.collection.filter('status', '==', 'online').filter('type', '==', 'job').order('-updatedAt').fetch(4)
        blogs = cls.collection.filter('status', '==', 'online').filter('type', '==', 'blog').order('-updatedAt').fetch(4)
        pposts = cls.collection.filter('status', '==', 'online').order('-views').fetch(4)
        news = cls.collection.filter('status', '==', 'online').filter('type', '==', 'news').order('-updatedAt').fetch(4)
        return  questions, products, jobs, blogs, pposts, news

What would be the better way for such a view where it has to render content from different filters?

Transactions and batched writes

Currently FireO not support Transactions and batched writes. Add this feature in FireO

QueryIterator throws exception when result includes empty document

When query result includes empty document, query iteration fails.

To reproduce:

from fireo.fields import TextField
from fireo.models import Model

class Company(Model):
    name = TextField()

Company.collection.create(name="Abc_company")
Company.collection.create()
Company.collection.create()

assert len(list(Company.collection.fetch())) == 3

The following exception is thrown:

Traceback (most recent call last):
  File "app.py", line 12, in <module>
    assert len(list(Company.collection.fetch())) == 3
  File "/usr/local/lib/python3.8/site-packages/fireo//queries/query_iterator.py", line 41, in __next__
    m._update_doc = self.query._update_doc_key(m)
  File "/usr/local/lib/python3.8/site-packages/fireo//queries/filter_query.py", line 437, in _update_doc_key
    update_doc_key = model.key
AttributeError: 'NoneType' object has no attribute 'key'

Error when TextField is not required but has a max_length set

Hi, thanks for creating this great library, it's just what I was looking for, simple but powerful.

I think I found a bug though. On a TextField, setting max_length when required is False results in the error "TypeError: 'NoneType' object is not subscriptable". Here's an example, notice that the locality is not specified in the address dictionary.

class Address(Model):
    premises = TextField(required=True, max_length=256)
    thoroughfare = TextField(required=True, max_length=256)
    locality = TextField(max_length=256)
    city = TextField(max_length=256, required=True)
    postcode = TextField(required=True, max_length=8, format='upper',
                         to_lowercase=True, validator=validate_postcode)

address = {
    "premises": "Flat 8, 189",
    "thoroughfare": "Furlington Road",
    "city": "Leeds",
    "postcode": "LS23 3HJ"
}

a = Address.from_dict(address)
a.save()
print(a.to_dict())

And here is the error message I get:

  File "/app/app.py", line 59, in <module>
    c.save()
  File "/usr/local/lib/python3.8/site-packages/fireo/models/model.py", line 342, in save
    return self.__class__.collection.create(self, transaction, batch, **self._get_fields())
  File "/usr/local/lib/python3.8/site-packages/fireo/managers/managers.py", line 189, in create
    return self.queryset.create(mutable_instance, transaction, batch, **field_list)
  File "/usr/local/lib/python3.8/site-packages/fireo/queries/query_set.py", line 52, in create
    return CreateQuery(self.model_cls, mutable_instance, **kwargs).exec(transaction_or_batch)
  File "/usr/local/lib/python3.8/site-packages/fireo/queries/create_query.py", line 130, in exec
    return query_wrapper.ModelWrapper.from_query_result(self.model, self._raw_exec())
  File "/usr/local/lib/python3.8/site-packages/fireo/queries/create_query.py", line 123, in _raw_exec
    ref.set(self._parse_field())
  File "/usr/local/lib/python3.8/site-packages/fireo/queries/create_query.py", line 93, in _parse_field
    self._nested_field_list(f, field_list, f.name)
  File "/usr/local/lib/python3.8/site-packages/fireo/queries/create_query.py", line 111, in _nested_field_list
    nested_field_list[n_f.db_column_name] = n_f.get_value(
  File "/usr/local/lib/python3.8/site-packages/fireo/fields/base_field.py", line 121, in get_value
    val = self.field_attribute.parse(val, ignore_required, ignore_default)
  File "/usr/local/lib/python3.8/site-packages/fireo/fields/field_attribute.py", line 111, in parse
    value = self.call_attr_method(attr, value)
  File "/usr/local/lib/python3.8/site-packages/fireo/fields/field_attribute.py", line 176, in call_attr_method
    return getattr(self.field, "attr_"+attr)(self.field_attr(attr), value)
  File "/usr/local/lib/python3.8/site-packages/fireo/fields/text_field.py", line 34, in attr_max_length
    return field_val[:attr_val]
TypeError: 'NoneType' object is not subscriptable```

Batch with manager create returns None.

Hi there!
I tried to batch as document below, but current manager's create with batch returned None in my case.
Is there way to work manager with batch for change value of object after create? If the behavior is right, remove update part.

https://github.com/octabytes/FireO/blob/gh-pages/managing-data/transaction-and-batches.md#batched-writes
1a6e0bc

city = City.collection.create(batch=batch, state='NYC', population=500000)
# Update the population for SF
city.population = 1000000
city.update(batch=batch)

tried with ver 1.4.1
https://github.com/octabytes/FireO/blob/v1.4.1/src/fireo/managers/managers.py#L153

batch = fireo.batch()
user_batch = User.collection.create(name="Name1", batch=batch)
print(user_batch)

> None

Collection.get fetch by id without collection name as prefix

When we are making a query -
User.collection.get(user_key)

user_key has to be the form of 'collection_name/{id}'

Can we support the form of just querying by ID and appending the collection name?

if '/' not in self.collection_path:
            # check collection path is the same as collection name
            if self.collection_path != self.model_cls.collection_name:
                raise errors.InvalidKey(f'Invalid key is given, expected "{model_name}" type key, '
                                        f'got "{self.collection_path}" type key')

In the above code, without a prefix, it returns invalid_key

Version:1.3.3

Inherited model "loose" IDField

When using abstract classes, I noticed IDField() is not working as expected.
when trying to set id, I got a random one.

To reproduce it:

import fireo
from fireo.models import Model
from fireo.fields import IDField, TextField

fireo.connection(from_file='firebase-service-account.json')

TEST1_COLLECTION = 'ttt1'
TEST2_COLLECTION = 'ttt2'


class Test1(Model):
    id = IDField()
    name = TextField()

    class Meta:
        collection_name = TEST1_COLLECTION


class BaseTest(Model):
    id = IDField()
    name = TextField()

    class Meta:
        abstract = True


class Test2(BaseTest):

    class Meta:
        collection_name = TEST2_COLLECTION


if __name__ == '__main__':

    t = Test1()
    t.id = '100'
    t.name = 'test1 whatever'
    t.save()

    t = Test2()
    t.id = '200'
    t.name = 'test2 yep'
    t.save()

    # Now read them
    t = Test1.collection.get(f'{TEST1_COLLECTION}/100')
    print(t.name)

    # This will fail
    t = Test2.collection.get(f'{TEST2_COLLECTION}/200')
    print(t.name)

DateTime() field gets updated to Null if not set in an update query

Given the following model:

class User(Model):
    user_id = IDField()
    name = TextField()
    date_added = DateTime()

A user is created with :

User.collection.create(user_id='1',name='Person Name', date_added=datetime.now())

His name is then changed, but the date_added is not touched:

user = User(name='New name')
user.update(key='1')

After this update, the date_added value is changed to Null in Firestore.

Looking at the (update_query.py)[https://github.com/octabytes/FireO/blob/master/src/fireo/queries/update_query.py], we can see why it's happening. If the value of the DateTime field is not set, it's value is returned.
It also seems like bool is also always returned even if it's value is not changed.

    for f in self.model._meta.field_list.values():
            # Check if it is nested model
            if isinstance(f, NestedModel):
                # Get nested model field
                self._nested_field_list(f, field_dict, f.name)
            else:
                v = f.get_value(self.query.get(f.name), ignore_required=True, ignore_default=True)
                if v is not None or type(v) is bool:
                    field_dict[f.db_column_name] = v
                if v is None and isinstance(f, DateTime):
                    field_dict[f.db_column_name] = v
        return field_dict

Is there a reason for this ?

Thank you,
Rémy

(BTW, really like FireO)

Ability to "attach" to an existing collection specifying its name

As in title, I would like to create a model that reflects an already created collection.
I think it would be quite common, but I could be wrong.
Generally, it would be nice creating a model specifying a collection name.

I got spam email

I just received an unsolicited email from you. I never signed up for your project or gave you my email address. You want my feedback? Ok. Don't fucking spam people.

Grrrrr....

Is it possible to reference a field from another field?

Just want to say great work on this project!

As for my question:
In other ORMs like NDB and Anom for datastore, you can create calculated fields. I'm migrating a sizable project from Datastore to Firestore, and I'm trying to implement the same functionality with FireO but can't figure it out.

I have something like this for the custom field:

class CalculatedField(Field):
  allowed_attributes = ['calc']
  
  def attr_calc(self, attr_val, field_val):
    if attr_val is not None:
      return attr_val()
    return None

Now in my model lets say I have something like this:

class Person(Model):
   name = TextField(required=True)
   name_lower = CalculatedField(calc=lamba self: self.name.lower())

And let's say I have a use case where I need to save the original formatted name, however the user entered it, but I also want to save the name in lower_case to make querying easier. I don't want to have to remember to always save the name twice, or if a new dev jumps on the project, he shouldn't have to just know that he or she needs to do that. But it doesn't seem like I can reference the model from the custom field in anyway or am I missing something?

I could do something like this:

class CalculatedField(Field):
  allowed_attributes = ['calc']
  
  def attr_calc(self, attr_val, field_val):
    if attr_val is not None:
      return attr_val(field_val)
    return None

class Person(Model):
  def __init__(self, *args, **kwargs):
    super().__init__(*args, **kwargs)
    self.name_lower = self.name

   name = TextField(required=True)
   name_lower = CalculatedField(calc=lamba x: x.name.lower())

And this works when creating new objects of Person, but this doesn't work if the name changes.

Thanks again!

Query by reference key not working

User(Models):
    fields

Address(Models):
    user = ReferenceField(User, auto_load=False)

I checked on firestore the reference field value is key or user.key
tried to query like below but not working.

Address.collection.filter(user=user.key).get()

No exception raised when indexes do not exist

Hello!

I've encountered a problem with FireO that I've struggled with for some days. I finally wrapped my head around the issue. The issue was that I had forgotten to add exemptions in Firestore. This resulted in me getting an empty array when using .group_fetch() instead of an exception.

This piqued my interest and I took a closer look at the source code. In the QueryIterator class, whenever an exception is caught it just stops the iteration without raising any exceptions.

I believe the intention is to raise an exception, or am I wrong? Maybe it's intended? Please do let me know.

---

Edit: Some more information regarding this.

It seems that this isn't limited to only group_fetch, but to queries in general.

Using the python-firestore library

The following code does raise an exception:

collection_group = db.collection_group(u'group_collection').where(u'example_id', u'==', '123')
results = collection_group.stream()

for result in results:
    print(result.id)

This is the exception that is raised:

google.api_core.exceptions.FailedPrecondition: 400 The query requires a COLLECTION_GROUP_ASC index for collection group_collection and field example_id. You can create it here: https://console.firebase.google.com/v1/r/project/project-here/firestore/indexes?create_exemption=exemption to create

Using FireO

The following code does not raise an exception:

from fireo import models, fields


class GroupCollection(models.Model):
  id = fields.IDField(required=True)
  example_id = fields.TextField(required=True, column_name='example_id')

  class Meta:
    collection_name = 'group_collection'

GroupCollection.collection.filter('example_id', '==', '123').group_fetch()

Both of these snippets are ran side-by-side in the same environment. While one succeeds at raising the exception (python-firestore), the other one does not (FireO).

I believe that FireO does not raise the exception because of this:

# fireo/queries/query_iterator.py, line 33-49

def __next__(self):
    try:
        doc = next(self.docs, None)
        if doc:
            # Suppose this is the last doc
            self.last_doc = doc
            m = query_wrapper.ModelWrapper.from_query_result(self.model_cls(), doc)
            m._update_doc = self.query._update_doc_key(m)
            # Suppose this is last doc
            self.last_doc_key = m.key
            return m
        self.fetch_end = True
        # Save last doc key in cursor
        self.query.cursor_dict['last_doc_key'] = self.last_doc_key
        raise StopIteration
    except Exception:
        raise StopIteration

The except Exception just stops the iteration. Is this intended?

Reference Field - Failed to save after instance created

class User(Model):
    name = TextField(required=True)


class UserActivity(Model):
    user = ReferenceField(User, auto_load=False)


# this is working
activity = UserActivity.collection.create(user=user)

# it will trigger below error on save
activity.save()

ReferenceTypeError: Invalid reference type. Field "user" required value type "User", but got "ReferenceDocLoader

the same error also happened on get then save UserActivity

Upgrade of google-cloud-firestore==2.0.2

Any plans to upgrade the firestore version?

The user requested google-cloud-firestore==2.0.2
Step #0 - "build image":     firebase-admin 4.5.1 depends on google-cloud-firestore>=1.4.0; platform_python_implementation != "PyPy"
Step #0 - "build image":     fireo 1.4.1 depends on google-cloud-firestore==1.9.0

Is it possible to save id as a field in a doc?

I'm used to add a field id in docs so I can do whereIn query in collections. Is it possible to do that?

Direct assignment for Nested Models

Currently nested model does not support to assign values directly in model.

For example

s = Student(address="Student_address")
s.user.name = 'Nested_Model'
s.save()

Here user is nested model. Currently this type of assigning is not allowed

Range attribute in NumberField

There should be a range attribute for NumberField So field not support values between these range.

Is there any chance empty documents will be supported?

Hi,
I know about issue #87.
But I need to read from an existing firestore database,
and I cannot avoid empty documents.
Will it ever be possible in the future?
Otherwise I am sadly forced not to use this fanstastic library.

Asyncio support?

Hi there!
I was wondering if you plan to support asyncio. I would feel this library would be a great complement to FastAPI and newer async frameworks out there.
Thanks!

filter fail , result is None

data:

code:

class Post(Model):
    id = IDField()
    category = ReferenceField(Category)
    lang = TextField(required=True)
    title = TextField(required=True)
    transtitle = MapField()
    content = TextField(required=True)
    tags = ListField()
    md5 = TextField()
    recommend = NumberField(default=0)
    active = BooleanField(default=False)
    created = DateTime(auto=True)

 p1 = Post.collection.filter(title='My First Post').get()
    
 print(p1)

result is None