race condition for… file metadata? about portalocker

[wolph]

With NFS... doesn't sound too weird, NFS and locking are rarely friends in my experience ;)

The thing with locks on unix systems is that they're (by default at least) advisory locks. Both clients need to choose to lock and the underlying filesystem needs to support it and the filesystem needs to be mounted with locking enabled. And with NFS, that last one is often the culprit. I should note that with NFS the locking can be enforced instead of being advisory.

So... common pitfalls with NFS locking:

• The mount flags, most people disable NFS locking due to performance reasons (it can be really slow). You can check if the (mount) command includes the nolock parameter
• The NFS version of all clients needs to be the same. The locking system between NFS 3 and NFS 4 is different and won't react to each other
• It could simply be a bug because of old kernels or other things... Look at D10 in the NFS FAQ: http://nfs.sourceforge.net/
• Perhaps this library is not using the correct locking flags for NFS. I honestly never tried because NFS locks are generally too slow to be usable in my experience.

In any case, I would start by looking at the server version, client versions and mount flags. See if they all match up and if nolock is not set as a mount flag.

[niacdoial]

yep, everything matches up, and the closest thing to a nolock option is local_lock=none, which… actually has the opposite effects for my purposes?

also the locking itself seems to work, because the various script2s always hang before I press enter on script1?

also also, I tried adding fh.flush()\n os.fsync(fh.fileno()) to script1, without any effect. I have no idea how I didn't see this part of the docs before, kinda disappointed by myself right now.

[wolph]

Yes, local_lock=none should be the right flag to have.

I'm not really sure what else to try honestly... it sounds like you're doing everything right.

[wolph]

I'm not sure if it's entirely relevant for you, but for cross-system locking I've also implemented a redis based locking system: https://github.com/wolph/portalocker#redis-locks

It works across multiple threads, systems, etc... in a completely safe manner