Comments (10)
When I created this I assumed the file system did atomic writes, but clearly not. Seems weird that fs can't handle this for you.
from configstore.
I spent a little time looking at this today. Unfortunately, it looks like the only way to incorporate a file lock would be to make the entire configstore API asynchronous.
Given that this is already a fairly small library, a change like that would essentially amount to a complete rewrite. There's also something to be said for the simplicity of the synchronous interface for use cases that aren't concerned with concurrent access.
Do you have any thoughts on what the best path forward on this might be? I'm considering the possibility of just forking/building an asynchronous concurrent-access-safe version of configstore, but wanted to get your insights first.
from configstore.
I don't see why it would have to be async. Can you elaborate?
from configstore.
If the lock is already held by another process when we go to read/write the config file, then the two options are either to give up entirely, or to wait some amount of time and check the lock again.
In the "just give up" option, clients of the library have to be prepared to handle a "lock already taken" exception potentially being raised by any configstore operation. They then have to implement their own logic to either manually retry after some period of time or otherwise handle the failure. This avoids data corruption, but pushes the problem of actually dealing with concurrent use on to the users of the library.
For the "automatically retry" option, we don't want to just hammer the filesystem in a loop checking the lock, so we need to wait some period of time between checks (proper-lockfile does this with nice exponential backoff). The only way I can think of to do this synchronously would be to roll our own file lock implementation that uses fibers or some other native extension to do a synchronous wait. Both the complexity of building a home-grown file lock and introducing the need for native bindings feel like points against that idea.
There could definitely be another alternative I'm not seeing, but the only other one I've thought of is an asynchronous interface.
from configstore.
Another alternative could be to use atomic file writing and just let the last concurrent write win. One concurrent write will overwrite another, but that might not be such a big problem?
from configstore.
I think having the last concurrent write win is good enough.
About the point about the automatically retry, the dummy implementation is to lock the process with a loop: while (now < t300msLater) {}
. This is blocking the process - but that's kind of the point of a synchronous API. That would be way easier than starting to play with fibers.
from configstore.
I have no idea how to implement "last concurrent write win" behavior :) With node-proper-lockfile it's probably feasible to implement "first concurrent write win", because subsequent calls to lockSync would throw an error..
from configstore.
@sheerun We can use atomic file writing: https://github.com/iarna/write-file-atomic It works by writing to a temporary file first, then renaming and overwriting the actual file when done. We would then get "last concurrent write win" for free.
from configstore.
That's certainly better than corrupted configuration files :)
from configstore.
@SBoudrias Blocking the process is the point of a synchronous API, but using a busy loop that pegs one CPU core seems less than ideal.
@sindresorhus Atomic writes definitely solve the corruption problem, and like you mentioned, it's easy to drop in a package to handle that. If you decide just avoiding total corruption is good enough for this project, then 👍.
In casual testing, though, the race condition I hit more often was caused by the read-mutate-write sequences in set
and del
– competing processes end up clobbering one another's changes when those steps interleave, so you get data loss. Not as dire as straight-up corrupting the file, but has a way bigger window it can happen in, so ¯_(ツ)_/¯
from configstore.
Related Issues (20)
- v5 is not working with SNAP HOT 6
- release for CVE-2020-8116 HOT 1
- Versioning the config store HOT 3
- How to handle arrays? HOT 1
- Whether to use `configstore` or `<package name>` should be a *user* option HOT 1
- thank you HOT 1
- v3.1.3 is now breaking node 6 builds HOT 3
- Allow user to override location of `configstore` directory using an environment variable HOT 1
- Requiring Node.js 10+? HOT 2
- You want to provide the same path of electronic store HOT 2
- What parameters are needed to set the path to electron-sotre HOT 2
- Current version not working with node js application HOT 1
- After Requiring Error HOT 4
- Secure temp dir in NodeJS without dependencies
- in v6.0.0 with typescript, giving error regarding dynamic import HOT 2
- is gracefull fs still needed? HOT 3
- Does the persistent file change consider supporting monitoring configuration file changes? HOT 1
- Relax 5.x version requirement on make-dir to allow 4.x (as well as 3.x)? HOT 1
- Make rewriting of JSONs with syntax errors optional
- Addition of dependabot to project HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from configstore.