vvvvalvalval / datomock

Mocking and forking Datomic Peer connections in-memory.

datomock

Mocking and forking Datomic connections in-memory.

Notes:

• This library is not an in-memory re-implementation of Datomic - just a thin wrapper on top of the Datomic Peer Library. All the heavy lifting is done by Datomic's 'speculative writes' (a.k.a db.with(tx)) and Clojure's managed references (atoms)
• Only for Peers, not Clients.

Project maturity: beta quality. Note that you will probably not need to use this library in production.

Usage

(require '[datomic.api :as d])
(require '[datomock.core :as dm])

(def my-conn (d/connect "datomic:mem://hello-world"))

;; ... create a mock connection from a Database value:
(def starting-point-db (d/db my-conn))
(def mocked-conn (dm/mock-conn starting-point-db))

;; which is essentially the same as: 
(def mocked-conn (dm/fork-conn my-conn))

;; dm/fork-conn is likely what you'll use most.

Rationale and semantics

Mocked connections use Datomic's speculative writes (db.with()) and Clojure's managed references to emulate a Datomic connection locally.

The main benefit is the ability to 'fork' Datomic connections. More precisely, if conn1 is forked from conn2:

• at the time of forking, conn1 and conn2 hold the same database value;
• subsequent writes to conn1 will leave conn2 unaffected
• subsequent writes to conn2 will leave conn1 unaffected

Because Datomic database values are persistent data structures, forking is extremely cheap in both space and time.

Applications

• Write expressive tests: write tests as a tree of scenarios exploring various alternatives. In particular, this makes it very easy to write system-level tests that run fast. Forget about setup and teardown phases: they are respectively replaced by forking and garbage-collection.
• Cheap, safe debugging: instantly reproduce your production environment on your local machine. Save and re-use as many checkpoints of your state as you need as you debug. Dry-run data patches and migrations safely before committing them to production.
• Explore new database schemas: in particular, you can experiment with changes to your database schema without committing to them.
• Staging environments / QA / CI: want one staging environment (Peer) for each pull-request on your app? Just have each of them use an in-memory fork of a shared database (or even your production database).
• Ephemeral demos: want to let people experiment with your app without accumulating their manual changes? Just have them work on a fork, and discard it afterwards.
• Ephemeral dev environments: similarly, it's usually better to always work on the same data when developing, and have the manual changes you've made while experimented be discarded at the end of the session.

Useful links:

• Application architecture with Datomic: branching reality: a blog post providing a more in-depth analysis of forkability and its applications.
• Full Stack Teleport Testing with Om Next & Datomic: a Clojure/West 2017 talk about how Ladder implement system-level testing using Om and Datomic.

How it works

Essentially, by putting a Datomic Database value in a Clojure Managed Reference (currently an Atom, may evolve to use an Agent instead) and using db.with() to update the state of that reference.

That's it, you now know how to re-implement Datomock yourself!

Actually, there are a few additional complications to make this work smoothly:

• Log: the reference needs to hold not only a Database Value, but also a Log Value (for some strange reason, in Datomic, Log Values are not part of Database values).
• Futures-based API: to match the interface of Datomic Connections, the library needs to provide a Futures-based API, which requires some additional work on top of Clojure references.
• txReportQueue: the library needs to provide an implementation of that as well.

Mocked connections vs datomic:mem

How is this different than using Datomic memory databases, as in (d/connect "datomic:mem://my-db") ?

Mocked connections differ from Datomic's memory connections in several ways:

• you create a memory connection from scratch, whereas you create a mocked connection from a starting-point database value
• a mocked connection is not accessible via a global URI

Compatibility notes

This library requires Datomic 0.9.4470 or higher, in order to provide an implementation of the most recent methods of datomic.Connection.

However, if you need to work with a lower version, forking this library and removing the implementation of the syncSchema(), syncExcise() and syncIndex() should work just fine.

This library works with Datomic 1.0.6527, but mock connections do not support transaction io-stats. A call like (d/transact mock-conn :io-context true) will not include an :io-stats key in the result. Query io-stats are supported insofar as return value shapes will be correct, but the underlying Datomic mem database doesn't provide any useful information in :io-stats :reads.

License

Copyright © 2016 Valentin Waeselynck and contributors.

Distributed under the MIT License.

datomock's Issues

Java Wrapper

Would be nice to have a Java wrapper, maybe something like a datomock.java.Datomock class with mock and fork static methods

support the Log API

Allow to disable print-method (Docs?)

Allow to disable the awesome print-method, that is awesome to debug in small scenarios, but on big ones, freezes the repl/make logs way longer.

Client API support?

Quickly skimmed through the client API and it has with-db. Now that cloud is a thing it would be amazing to quickly fork prod DB's. Anything holding this back?

https://docs.datomic.com/client-api/datomic.client.api.html#var-with-db

Fork at a given T point (yes, it's possible) ; no it's not

First of all, thanks for Datomock, this is super useful for testing! The one thing that's been bugging me was the inability to fork at arbitrary T point.

I'm aware that d/as-of does not allow subsequent speculative transactions. However, I've been able to develop a variant of as-of that does https://www.reddit.com/r/Clojure/comments/yemxzi/datomic_with_asof_or_the_elusive_branching_off_a/. I've been using it for local dev for a couple of weeks now and it seems to work well enough.

Do you think you'd be interested in incorporating it into your library? I'm thinking fork-conn taking T as an optional arg. This would be very valuable for debugging production issues, or writing reproducible tests. I considered publishing my as-of'' in a library of my own, but as you can see, the solution is too simple to warrant a separate dependency. If you think you'd want this kind of feature here though, I'd be happy to produce a PR.

Some considerations:

On the reddit thread someone brought up performance as a potential issue. It's true that d/filter has a significantly higher cost than d/as-of, and that cost stacks while d/as-of does not. Having said that, for the purposes of datomock, you'd only really want to branch off once, which has negligible cost. I've been experimenting with using primitive arithmetic but that didn't help much. The only thing that seems to make a difference is using (.tx datom) instead of destructuring (compared to the code in reddit post).

It's important to note that the resulting DB d/is-filtered will be true. The only way to work around it I can think of is to (reify Database ..) which is extra overhead. I don't mind much either way, let me know what you think.

The other difference from regular connections is that the T values are not continuous. This means, the next speculative transaction won't be T+1. It will be (d/next-t (d/db original-connection)). I don't recon this would ever be an issue for anyone, but it's only fair to point out.

`transact` on mocked connection throws, instead of on the deref

I've noticed a subtle difference with datomock connections over the Datomic ones. When you transact on Datomic, any problem isn't thrown until you deref the result. When mocking the connection the exception happens on transact instead of the deref.

Agent-based implementation

It may be interesting to offer the user the choice 2 implementations: atom-based and agent-based (currently there's only atom-based).

Potential benefits:

1. In the atom-based implementation, when the .transact() or .transactAsync() method returns, the transaction has systematically been committed, meaning that any subsequent call to .db() will reflect the transaction. We don't want users to erroneously rely on this in dev and have bad surprises in production. We could even go further expose an option to emulate latency.
2. more generally an Agent seems closer to the execution model of a real transactor (transactions are queued and executed asynchronously)
3. under some circumstances, this may increase the speed of tests by relieving the main thread of transaction work.

Not compatible with datomic 1.0.6527

Datomic 1.0.6527 added additional methods to the Connection interface related to io-stats to support the new arities of transact and transact async. The datomic.api/transact and transact-async wrappers seem to always use this new method, even if called with only one argument.

This means datomock fails with 1.0.6527. Exceptions look like this:

Uncaught exception, not in assertion.
Exception: java.lang.AbstractMethodError: Method datomock/impl/MockConnection.transact(Ljava/util/List;Ljava/lang/Object;)Ldatomic/ListenableFuture; is abstract
 at datomock.impl.MockConnection.transact (impl.clj:-1)
    datomic.api$transact.invokeStatic (api.clj:107)
    datomic.api$transact.doInvoke (api.clj:105)
    ...

A quick fix is to add the additional method signatures but ignore the io-stat argument:

(defrecord MockConnection [...]
  Connection
  ...
  (transact [this tx-data _] (.transact this tx-data))
  (transactAsync [this tx-data _] (.transact this tx-data))
  ...
)