objectbox / objectbox-dart

Flutter database for super-fast Dart object persistence

Getting Started • Documentation • Example Apps • Issues

Flutter database - easy to use & fast Dart object persistence 💙

---

💙 Jobs: We're looking for a Flutter Developer Relations Manager! 💙

---

Database for Flutter / Dart - intuitive APIs, simply fast. Persist local Dart objects with ease & speed, focus on the value your application provides.

ObjectBox provides a store with boxes to put objects into:

// Annotate a Dart class to create a box
@Entity()
class Person {
  @Id() 
  int id;
  String name;

  Person({this.id = 0, required this.name});
}

final box = store.box<Person>();

// Put a new object into the box
var person = Person(name: "Joe Green");
final id = box.put(person);

// Get the object back from the box
person = box.get(id)!;

// Update the object
person.name = "Joe Black";
box.put(person);

// Query for objects
final query = box.query(Person_.name.equals("Joe Black"))
        .order(Person_.name).build();
final people = query.find();
query.close();

// Remove the object from the box
box.remove(person.id);

Ready? Continue with the ➡️ Getting Started guide.

Why use ObjectBox

ObjectBox Flutter database is an excellent choice for storing Dart objects in cross-platform applications. Designed for high performance, the ObjectBox Flutter database is excellent for mobile and IoT devices. ObjectBox consumes minimal CPU, memory, and battery, ensuring that your software is not only efficient but also sustainable. By storing data locally on the device, ObjectBox allows you to cut cloud expenditures and create an app that does not require a connection. Get started with our intuitive native Dart API in minutes, without the hassle of SQL. Plus: We built a data synchronization solution that allows you to keep data in sync across devices and servers, both online and offline.

Features

🏁 High performance - superfast response rates enabling real-time applications.
🪂 ACID compliant - Atomic, Consistent, Isolated, Durable.
💻 Multiplatform - Android, iOS, macOS, Linux, Windows, any POSIX-system.
🌱 Scalable - grows with your app, handling millions of objects with ease.
💚 Sustainable - frugal on CPU, Memory and battery / power use, reducing CO2 emmissions.

🔗 Relations - object links / relationships are built-in.
💐 Queries - filter data as needed, even across relations.
🦮 Statically typed - compile time checks & optimizations.
📃 Schema migration - change your model with confidence.

Oh, and there is one more thing...

😮 Data Sync - keeps data in sync offline or online, between devices and servers.

Getting Started

Continue with our ➡️ Getting Started guide. It has resources and video tutorials on how to use ObjectBox in your Flutter or Dart app.

How does ObjectBox compare to other solutions?

• ObjectBox is fast. Have a look at our benchmarks below, or test it for yourself
• It's a full NoSQL SQLite alternative with intuitive Dart APIs you'll love 💙
• It comes with an out-of-the-box Data Sync, making it an effective self-hosted Firebase alternative

Flutter Database Performance Benchmarks

We tested across the four main database operations, CRUD (create, read, update, delete). Each test was run multiple times and executed manually outside of the measured time. Data preparation and evaluation were done outside of the measured time.

Here are the benchmarks for ObjectBox vs sqflite vs Hive 👇

You can run these yourself using our objectbox-dart-performance Flutter benchmark app.

How do you 💙 ObjectBox?

We're looking forward to receiving your comments and requests:

• Add GitHub issues
• Upvote issues you find important by hitting the 👍/+1 reaction button
• Fill in the anonymous feedback form to help us improve our products
• Drop us a line on Twitter via @ObjectBox_io
• ⭐ us on GitHub, if you like what you see or give us a 👍 on pub.dev

Thank you! 🙏

For general news on ObjectBox, check our blog!

Contributing

Do you want to check out the ObjectBox code itself? E.g. see in action, run tests, or even contribute code? Great! Clone/check out this repository and run this to generate code and get you started quickly:

./tool/init.sh

Also, make sure to have a look at the contribution guidelines - we are looking forward to your contribution.

Packages

This repository holds all ObjectBox Dart/Flutter packages as separate directories:

• objectbox - main library code
• objectbox_generator - code generator
• objectbox_flutter_libs - core binary library dependency for Flutter (Android/iOS) - no dart/flutter code
• objectbox_sync_flutter_libs - core binary library dependency with ObjectBox Sync enabled
• benchmark - used internally to microbenchmark and compare various implementations during development of objectbox-dart

There's also a separate repository benchmarking objectbox (and other databases) in Flutter: objectbox-dart-performance. And another one testing and comparing the performance of our FlatBuffers fork vs the upstream version: flatbuffers-benchmark.

Other languages/bindings

ObjectBox supports multiple platforms and languages:

• Java Database (+ Kotlin): runs on Android, desktop, and servers.
• Swift Database: build fast mobile apps for iOS and macOS.
• Go Database: great for data-driven tools and embedded server applications.
• C / C++ Database: native speed with zero copy access to FlatBuffer objects.

License

Copyright 2019-2024 ObjectBox Ltd. All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Note that this license applies to the code in this repository only. See our website on details about all licenses for ObjectBox components.

objectbox-dart's Issues

Generator - fully-contained model

Instead of loading model from JSON, include all information in the generated code. Some of the information is already there, in the generated Entity_ class's static members so it may be useful to reuse that but it's not mandatory.

Test null values for entity properties

Because every value can be null in Dart, correct serialization of it must be tested.

Improve developer experience - generated code

Currently, binding code must be generated manually using pub run build_runner build before running/building an application

Is there a way this could be streamlined?

Generator - entities in multiple files

Verify (= have a test case) the generator supports generating a single database model for entities spread around multiple files.

Index and Unique annotations

Create equivalents of the @Unique and @Index Java annotations for Dart.

Bulk get and put

Add to Box:

• put with a list of objects
• getAll to return all available objects

Clarify build time code introspection and code generation options

ObjectBox usually depends on build time tooling, e.g. parse source code with annotation in an AST so that we can generate some code that is compiled along with the user's code (entity (type) binding). Also, ObjectBox tracks the data model and IDs via a JSON file. The entity binding and the data model JSON itself should go in new tickets once we know better what we have.

What build time tooling is available in Dart? What can it do?

Links:

• https://dart.dev/tools/build_runner
• https://medium.com/flutter-community/part-2-code-generation-in-dart-annotations-source-gen-and-build-runner-bbceee28697b

Provide examples of Isolates i.e. don't do everything on the main thread

TODO

• free datastructures when exiting
• message passing, e.g. ReceivePort, SendPort
• transactions, atomicity guarantee etc.

Generator - support 3rd party annotations

Currently, the generator omits (doesn't store) an entity field with an unknown annotation.
From generator.dart:

            // skip unknown annotations
            print(
                "warning: skipping field '${f.name}' in entity '${element.name}', as it has the unknown annotation type '$annotType'");
            continue;
          }

Instead, we should process those fields normally and ignore unknown annotations.

Publish to pub.dev

Prepare docs, resolve code quality issues (#31 #28) and publish, ideally using CI, from the master branch?

See https://dart.dev/guides/libraries/create-library-packages#distributing-a-library

Package for Android

Goal: make ObjectBox simple to use when deployed to Android

First, we need to have a binary distribution of the dynamic lib for Android (ObjectBox Java for Android comes with JNI API; not the C API...).

Resources:

• https://github.com/flutter/flutter/wiki/Binding-to-native-code-via-FFI
• https://github.com/dart-lang/sdk/blob/master/samples/ffi/sqlite/docs/android.md

Open questions:

• How to package our bundle (dart + native) exactly? AAR? Flutter dependency? Are there even guidelines yet?

Once done: create a ticket to do the same for iOS

Test integer limits for entity properties

It's necessary to test whether minimum and maximum integer values for all integer types currently supported by the generator are stored and re-fetched correctly.

Automatically generate IDs and UIDs for entities and properties

• when writing an entity class, it should be enough to specify the respective annotation without any parameters, i.e. @Entity() class MyEntity { ... } should suffice
• all instance variables of the entity class shall be incorporated into the final model and only those with special properties (e.g. @Id()) need to be annotated
• all IDs and UIDs need to be generated by objectbox_model_generator, together with these rules on how to resolve changes and conflicts
• in contrast to the current situation, this means that the existing generated file (specifically the _MyEntity_OBXModel object) needs to be parsed beforehand in order to detect changes

Switch to Dart v2.6

Dart v2.6 is going to break compatibility in FFI (it was in "preview" after all, so no surprise there). From what I've seen with the current dev version (.8), it was rather significant so I have a feeling we'll have to drop Dart v2.5 compatibility to be able to switch to v2.6

See the changes that have already landed in the dev release and the FFI project board for what's coming.

Generator test-case expected-update

The code generator is tested using "golden/expected" files - verifying the generated code matches the expected code exactly. While this is a great way to make sure the code doesn't change unexpectedly and there are no regressions, it poses a challenge when actually changing the generator code (as a developer).

To achieve a good developer experience, the test-cases expected code should be updatable by calling a single command. For example, in Go, tests are normally called as go test tests/generator while there's an option to update the .expected files calling go test tests/generator -update.

After the files are updated, it's up to the developer to verify the changes are correct (by manually reviewing them), before committing the code to git.

Basic Query (Builder) support

Goal: minimal solution including a query builder and a reusable query object. For reference, please check the Java query API.

• QueryBuilder to allow "equal" criteria for string and int properties (pass property IDs to identify properties if we don't generate a meta model yet)
• QueryBuilder allows building a Query object
• Query object methods: find() and findFirst(). find() returns all matching objects, findFirst() just the first one.

Ensure Flutter compatibility

It was brought up on reddit that there might at least one compatibility issue with Flutter:

I noticed in the box class you're using dart:mirrors. As far as I know that doesn't work in flutter, did that recently get changed? Here is some discussion about it from a while ago.

So, ensure everything run on Flutter.

Bonus task: can we unit test using Flutter?

Provide helpful error messages

If I'm not wrong, the check(cond) does not result in helpful messages.

To be more friendly to users, we should tell a) what's wrong and b) how to fix it.

Use valgrind to simplify no-leak regression testing

Seems like we can detect leaks using valgrind:

dart2native ./test/box_test.dart -o box-test
valgrind --show-mismatched-frees=no ./box-test

==13549== LEAK SUMMARY:
==13549==    definitely lost: 0 bytes in 0 blocks
==13549==    indirectly lost: 0 bytes in 0 blocks
==13549==      possibly lost: 0 bytes in 0 blocks
==13549==    still reachable: 91,012 bytes in 188 blocks
==13549==         suppressed: 0 bytes in 0 blocks

And If I manually comment out a free() call on an ffi.Pointer (e.g. in box.count()), it shows a leak:

==13580== LEAK SUMMARY:
==13580==    definitely lost: 128 bytes in 16 blocks
==13580==    indirectly lost: 0 bytes in 0 blocks
==13580==      possibly lost: 0 bytes in 0 blocks
==13580==    still reachable: 91,012 bytes in 188 blocks
==13580==         suppressed: 0 bytes in 0 blocks

Similarly, when I comment out a call to obx_query_close() and run valgrind on the query-test, it catches a leak of the core resources as well:

==15074== LEAK SUMMARY:
==15074==    definitely lost: 6,200 bytes in 181 blocks
==15074==    indirectly lost: 218,547 bytes in 3,199 blocks
==15074==      possibly lost: 0 bytes in 0 blocks
==15074==    still reachable: 91,012 bytes in 188 blocks
==15074==         suppressed: 0 bytes in 0 blocks

---

Therefore, it is clear it can help detect leaks that could be potentially introduced in the future code changes. There is also a load of Warning: set address range perms: large range and mismatched free() / delete / delete [] reports but I don't think those are related to our code, at least for the latter. That would require further investigation though.

Goal

Create a script for project developers and CI. The script:

• compiles all /test/*_test.dart files using dart2native
• executes each compiled test using valgrind
• capture the leak summary and show an error to the user if there is a leak (any non-zero "lost" report)

The script should ideally return a non-zero code in case of an error, so it can easily be used in the CI.

Box provide missing methods

Box already has get/put as well as their bulk variants. To be more up to par with the rest of the bindings, the following methods should be implemented:

• count(optional limit)
• contains(id)
• isEmpty()
• remove(id)
• removeAll()

See box.go and go docs for reference

Use proper unit tests

package:test seems to be the standard for Dart unit testing?

• Our current main() based "test" could become the start of an example. Move to examples/dart/notes.dart (?)
• Define a test entity in a separate file
• Start with a unit test file to test Box

Internal ObjectBox exceptions are not reported correctly

Operating system: Ubuntu 16.04 [to be confirmed for other OSs]
Dart version: 2.4.1

When trying to call ObjectBox functions with e.g. invalid parameters, normally the error should be reported on the command line as well as being reflected as an error code retrieved from obx_last_error_code. As an example, try creating an entity with ID -1 (@Entity(id: -1, uid: 1)), which is illegal in ObjectBox. Instead of seeing a meaningful message, the following is output, in other cases with the OBX_ERROR_NO_ERROR_INFO error code:

001-10:36:16.1248 [ERROR] Cannot handle thrown exception, no ptr

After some investigation, it became obvious that this issue is due to this problem in the Dart FFI, i.e. something rather fundamental in C++.

Use UTF-8 strings

Native ObjectBox uses UTF-8, while Dart's String uses UTF-16.

Check our CString wrapper Dart class, which should do proper conversion.

PS.: Is there a better way to process chars than iterating over each character? E.g. is there a byte buffer, we could interpret as UTF-8? If FFI does not offer that yet, please check if they already have an issue for that (or create one).

CI tests for the flutter demo

We're using GitHub Actions to pub run test - execute tests in the test folder - see .github/workflows/`

Another workflow should be added to build the flutter demo/example.

Refactoring

• move common.lastErrorString to binding/helpers.dart - it's the only place where it's called from
• since generated "flatbuffers_id": ${(prop['id'] as int) - 1}, is always property.Id - 1 it can be removed completely (as in other language bindings)
• remove box.close() - that's not how a store should be closed
• split flatbuffers marshalling out of box.dart
• use dartfmt -l 120 (and add it to contribution guide?)

Support all types

Double-check that we at least support the basic Dart types for entity properties and as query conditions: int, double, String, bool.

If we don't support byte arrays yet, create a separate ticket; as far as I can tell there is Uint8List and potentially ByteData to be supported.

Flutter example app

The typical ObjectBox example is a simple note taking app. So let's do that using Flutter...

Destination: this repo inside a new examples/flutter/ dir.

For reference:
https://github.com/objectbox/objectbox-examples

Allow setting store options

Using obx_opt_* functions, properties like the store path, database filename etc. can be set. See here for the list of possible options.

Use a Store constructor with named arguments to easily specify any non-default options.

Write transactions

Support write transactions by Store::runInTransaction
See Go for reference: https://github.com/objectbox/objectbox-go/blob/master/objectbox/objectbox.go#L107

Property query

Another chunk of code coming this way, stay tuned.

Box insert - assign new IDs on passed objects

The passed object should be updated to also contain ID after a successful insert. This applies to box.put() and box.putMany()

Example for put():

final object = TestEntity.initText("Hello");
int putId = box.put(object);
expect(putId, 1);
expect(object.id, 1);

Consider @Id properties as long

We shouldn't require to specify Type.Long for ID properties. There's no alternative to it, so we should verify the Dart type as int and set the ObjectBox type to Long.

Update the readme.

Utf8 without casts

Depends on dart-archive/ffi#6

API docs

Public APIs should have Doc comments so they would show up on pub.dev docs page

Avoid copying >1

    // TODO (perf) find a way to get access to the underlying memory of Uint8List to avoid a copy
    //  In that case, don't forget to change the caller (FlatbuffersManager) which expect to get a copy
    // if (data.length == bytes.length) {
    //   bytes._dataPtr = data.some-way-to-get-the-underlying-memory-pointer
    //   return ptr;
    // }

I saw this bit of code and did some research, then came to the conclusion, that dart doesn't have a List to UnsafePointer type (yet).

To avoid copying twice (i.e. once for collecting bytes in the buffer, to flatten the entity, twice for copying the buffer to the Pointer alloc), you might as well not use the (flat buffer) builder, and directly allocate a Pointer<Uint8>(?) with the the proper payload. The new builder should allocate, align and copy at the last moment, when finish is called.

I'm not completely sure if the changes should be made on google's builder, or this project's.

Read and write chunks of memory for CString

Currently, writing strings to and reading strings from memory is done character by character. Try to find a more efficient method to do that.

Is mostly related to CString (#7), but also memory chunks in general.

Unit test for ModelInfo::generateUid()

The code seems a little fragile, with the bitshifts and so on so it should be covered by a unit test.

Async property query returns result too early

If you run the branch #51 and #58, and compare the count and find result on the PropertyQuery, you'll find that the find function sometimes returns a partially correct result set, with certain random bytes (every time you run it), that are the same as the random bytes as count consistently returns.

This leads me to think that these obx functions return via ffi before the actual correct values are populated on the array alloc. This reminds me of the async await the Future discussion #49 .

Internal packages visibility

Currently, user code sees all symbols defined in lib/src/binding & lib/src/modelinfo.

I'm rather concerned about binding as it fairly pollutes the symbol-space (all the C typedefs, enums, helper functions, etc..)

We should limit the visibility so that those functions are not propagated to user code.

Generator tests

Currently, generator has a single "integration" test - verifying the code is generated properly.
It could use more tests - especially all supported types and annotations should be covered by tests.

Effective Dart

Why don't use an Effective Dart for this project?
Like here for constant names definitions you break it.

Thanks!

Prepare CONTRIBUTING.md

• See https://help.github.com/en/articles/setting-guidelines-for-repository-contributors
• Move the relevant section from README.
• Include information for quality contributions: tests for new/changed code, formatting/code-style compliance, docs

iOS support

After the flutter mobile example is prepared, it can be used to try if we can get the library to work on iOS. Seems like a .framework would be necessary. Hopefully, we can extract it from an objectbox-swift release

Avoid android users specifying `objectbox-android` dependency

Currently, to use objectbox-dart with flutter on Android, a user must manually specify a dependency on objectbox-android, which is a gradle artifact published by the objectbox-java project. Ideally, we should specify this dependency here in the dart/flutter lib, so the consumers wouldn't have to do it.

Follow up from #2
See my question here: flutter/flutter#33227 (comment)
A branch where I tried something (it didn't work though): https://github.com/objectbox/objectbox-dart/tree/flutter-plugin

Async/Await/Futures

Looking at some other dart code I've got an impression the API should be asynchronous (returning Futures). Thoughts?

Flatbuffers marshalling performance

The way we currently marshal and unmarshal flatbuffers into objects isn't optimal. We should get rid of the intermediary propVals map<string, dynamic>. Ideally creating objects (or flatbuffers in the other direction) directly in the generated code.

ffi:Pointer finalizer

We should track the development of this issue.
It might be available on the bleeding edge dart.

Differently signed dart from brew prevents loading C library

I got this, by moving the lib one level lower (project root):

...
 Invalid argument(s): Failed to load dynamic library (dlopen(libobjectbox.dylib, 1): no suitable image found.  Did find:
        libobjectbox.dylib: code signature in (libobjectbox.dylib) not valid for use in process using Library Validation: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed.)
  dart:ffi                                             new DynamicLibrary.open
...

It might be a new security thing. I'm not the only one. According to this thread, we either need objectbox's or dart's signing key, or remove the signing hash from the compiling dart binary. I fixed it by doing the last bit.

From bash or zsh wherever:

codesign --remove-signature `which dart`

Originally posted by @Buggaboo in https://github.com/objectbox/objectbox-dart/timeline

Query equality operator overload

Queries currently use some handy operator overloads, e.g. box.query(Note_.text == "text") in addition to the standard box.query(Note_.text.equals("text")).
Equality seems to be a special case, causing a linter issue. Make sure it's really OK to use it the way we do, otherwise, we'll have to drop it.

Linter reports:

error: 'QueryStringProperty.==' ('Condition Function(String)') isn't a valid override of 'Object.==' ('=='). (invalid_override at [objectbox] lib/src/query/query.dart:86)

info: Equality operator `==` invocation with references of unrelated types. (unrelated_type_equality_checks at [objectbox] test/query_test.dart:52)

Also, we clearly break the requirements given here: https://api.dartlang.org/stable/2.5.2/dart-core/Object/operator_equals.html

`build_runner build` should not touch examples

Currently, when one runs pub run build_runner build in the library root, it also generates the code for examples, considering all files the same project, effectively combining their databases. How can we prevent that?