Multiple Database error about tinydb

[msiemens]

Commit 58eb3dd adds a small note that using TinyDB from multiple processes is not supported.

[msiemens]

Am I correct in assuming that you're running the Python scripts in parallel from multiple Python instances?

[dirkk0]

Yes?

On Mon, Oct 26, 2015 at 9:21 PM Markus Siemens [email protected]
wrote:

Am I correct in assuming that you're running the Python scripts in
parallel from multiple Python instances?

—
Reply to this email directly or view it on GitHub
#72 (comment).

[msiemens]

Okay, that's propably the problem. TinyDB isn't designed to be used from multiple Python instances in parallel. We don't even support usage from multiple threads in one Python instance. I don't think we can solve this issue without adding a lot of complexity by adding some sort of read/write locking.

[eugene-eeo]

@dirkk0 I would recommend writing a storage that uses file locking. If you can't access fcntl-like syscalls then maybe you can try a more naive approach: E.g. creating a lockfile, then deleting it when a modification is done and having your other processes waiting until your lockfile is deleted then you can achieve parallelism.

Although I hope you'd opt for a more complex route and perhaps direct all read/write requests to a single DB process using some form of IPC (e.g. XMLRPC).

[dirkk0]

Thanks for the reply! Since re-opening the database works for me, and efficiency is not an issue in this particular case, everything is fine for me.

I still consider this a bug, though. So feel free to close this issue, if you think different.

Thanks again,
Dirk

[msiemens]

I think this is a wont-fix kind of bug as it is due to TinyDB's design. Nevertheless this should be documented better.