gofrs / flock Goto Github PK
View Code? Open in Web Editor NEWThread-safe file locking library in Go (originally github.com/theckman/go-flock)
License: BSD 3-Clause "New" or "Revised" License
Thread-safe file locking library in Go (originally github.com/theckman/go-flock)
License: BSD 3-Clause "New" or "Revised" License
Can this library be used to prevent an external program from deleting or modifying an external program while I try and run the external program using os.Exec()
?
Let's track the unit test coverage of flock
using codecov.
Thanks for the great package. It has solved quite a few challenges for me. That said, I recently ran into something and I'm in a bit of a bind without having access to the file handle held by the Flock object.
I understand from other conversations in the issues that the main purpose of this library is to support lock-file based locking, rather than locking the file you intend on modifying. That is a great solution when all parties involved in modifying the file agree on the strategy. However, I need to integrate with some software on Windows which does not use that approach: it simply locks the file it intends on modifying. As mentioned elsewhere, windows locking strategy limits what you can do: you can really only operate on the file handle that was used to create the lock.
I get why it's good to keep the interface pure so that users are lead to the lock-file approach. However, this package is the best os-independent golang file locking library I've found; none of the others really measure up to it. It'd be a shame to have to reimplement so much of it just to access its internal file handle.
For now, I've forked the repo until I can come up with a better strategy (I'm under somewhat of a time crunch). That said, if there is interest in actually doing this work, we could consider starting a PR with my commit. Alternatively, I'm willing to work to add other solutions as long as they satisfy my key requirement of being able to perform operations on the os.File underlying the lock.
Flock does not exactly follow the sync.Locker interface, as it's Lock() method returns an error. sync.Mutex panics in certain occasions. I think we could have a version of flock which also panics instead of returning errors, to ensure we have the correct interface.
func WriteByString(path string, data string) bool {
fileFlock := flock.New(path)
if err := fileFlock.RLock(); err != nil {
return false
}
file, err := os.Open(path)
if err != nil {
return false
}
if _, err := io.WriteString(file, data); err != nil {
fmt.Println(err.Error())
return false
}
return true
}
error:
write ./set.txt: Access is denied.
package main
import (
"io/ioutil"
"os"
"github.com/gofrs/flock"
)
func main() {
file := "test.txt"
lock := flock.New(file)
if err := lock.Lock(); err != nil {
panic(err)
}
defer lock.Unlock()
f, err := os.Open(file)
if err != nil {
panic(err)
}
_, err = ioutil.ReadAll(f)
if err != nil {
panic(err)
}
}
panic: read test.txt: The process cannot access the file because another process has locked a portion of the file.
goroutine 1 [running]:
main.main()
C:/github/fslock/main.go:24 +0x16b
exit status 2
Went and looked back at this, now I remember why I wanted an extra Close() function.
In try, anywhere after
Line 147 in 886344b
Perhaps the best solution would be to just close the file if the lock/try failed, then a user doesn't need to worry so much about a close function.
The following code works without errors in Unix and ends with error "Remove of lock file failed with: remove C:\Users\i019379\go\src\testLocking\lock.lock: The process cannot access the file because it is being used by another process." in Windows.
Thus in Windows it's impossible to remove lock file if more then one process tried to make locking.
func main(){
wd, _ := os.Getwd()
lockfile := filepath.Join(wd, "lock.lock")
lock1 := flock.New(lockfile)
locked1, _ := lock1.TryLock()
if !locked1{
fmt.Println("Error - First tryLock failed")
return
}
lock2 := flock.New(lockfile)
locked2, _ := lock2.TryLock()
if locked2{
fmt.Println("Second tryLock succeeded")
return
}
err := lock2.Unlock()
if err!=nil {
fmt.Println("Unlock of second lock failed")
return
}
err = lock1.Unlock()
if err!=nil {
fmt.Println("Unlock of first lock failed")
return
}
err = os.Remove(lockfile)
if err!=nil {
fmt.Println("Remove of lock file failed with: "+err.Error())
return
}
}
๐๐ป
I'm not sure if this is expected behaviour for this package or not, it was unexpected to me at least, but if I pass a file path that doesn't exist, I will successfully be able to acquire a lock and then when reading the contents of the file it returns empty content.
The reason this was unexpected was that in my project I have a test that validates an error is returned if a path fails to be read, but it was failing because it expected an error and now (since implementing this package) it was no longer getting an error because the file did exist (and for my use case that was unexpected).
An example of what I'm describing is:
package main
import (
"context"
"fmt"
"os"
"time"
"github.com/gofrs/flock"
)
const (
// FileLockTimeout is the amount of time to wait trying to acquire a lock.
FileLockTimeout = 10 * time.Second
// FileLockRetryDelay is the mount of time to wait before attempting a retry.
FileLockRetryDelay = 500 * time.Millisecond
)
func main() {
filename := "does_not_exist"
fileLock := flock.New(filename)
lockCtx, cancel := context.WithTimeout(context.Background(), FileLockTimeout)
defer cancel()
locked, err := fileLock.TryLockContext(lockCtx, FileLockRetryDelay)
if err != nil {
fmt.Printf("error acquiring file lock for '%s': %v", fileLock.Path(), err)
return
}
if locked {
fmt.Println("got a lock", fileLock.Path())
fi, err := os.Stat(fileLock.Path())
if err != nil {
fmt.Printf("error stating file '%s': %v", fileLock.Path(), err)
return
}
fmt.Printf("%+v\n", fi) // seems the file now exists
data, err := os.ReadFile(fileLock.Path())
if err != nil {
fmt.Printf("error reading file '%s': %v", fileLock.Path(), err)
return
}
fmt.Printf("file content: %+v\n", string(data)) // empty file content
if err := fileLock.Unlock(); err != nil {
fmt.Printf("error releasing file lock for '%s': %v", fileLock.Path(), err)
}
}
}
The solution in my case was to add a .Stat()
call up front before calling flock.New()
but I wanted to be sure this wasn't a bug. Maybe it's I just don't understand how file locks are supposed to work (very likely).
Any feedback/guidance appreciated.
Thanks!
2022/09/06 23:09:27 Command output was:
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:28:30: undefined: syscall.LOCK_EX
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:39:30: undefined: syscall.LOCK_SH
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:57:20: undefined: syscall.Flock
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:67:20: undefined: syscall.Flock
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:97:20: undefined: syscall.Flock
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:97:50: undefined: syscall.LOCK_UN
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:119:29: undefined: syscall.LOCK_EX
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:131:29: undefined: syscall.LOCK_SH
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:151:17: undefined: syscall.Flock
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:151:52: undefined: syscall.LOCK_NB
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:151:52: too many errors
2022/09/06 23:09:27 ----------------------------
2022/09/06 23:09:27 Error compiling plan9/386: exit status 2
2022/09/06 23:09:57 Command output was:
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:28:30: undefined: syscall.LOCK_EX
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:39:30: undefined: syscall.LOCK_SH
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:57:20: undefined: syscall.Flock
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:67:20: undefined: syscall.Flock
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:97:20: undefined: syscall.Flock
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:97:50: undefined: syscall.LOCK_UN
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:119:29: undefined: syscall.LOCK_EX
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:131:29: undefined: syscall.LOCK_SH
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:151:17: undefined: syscall.Flock
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:151:52: undefined: syscall.LOCK_NB
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:151:52: too many errors
2022/09/06 23:09:57 ----------------------------
2022/09/06 23:09:57 Error compiling plan9/amd64: exit status 2
2022/09/06 23:13:03 ----------------------------
2022/09/06 23:13:03 Command output was:
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:28:30: undefined: syscall.LOCK_EX
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:39:30: undefined: syscall.LOCK_SH
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:57:20: undefined: syscall.Flock
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:67:20: undefined: syscall.Flock
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:97:20: undefined: syscall.Flock
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:97:50: undefined: syscall.LOCK_UN
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:119:29: undefined: syscall.LOCK_EX
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:131:29: undefined: syscall.LOCK_SH
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:151:17: undefined: syscall.Flock
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:151:52: undefined: syscall.LOCK_NB
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:151:52: too many errors
2022/09/06 23:13:03 ----------------------------
2022/09/06 23:13:03 Error compiling solaris/amd64: exit status 2
2022/09/06 23:13:03 ----------------------------
2022/09/06 23:13:03 Command output was:
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:28:30: undefined: syscall.LOCK_EX
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:39:30: undefined: syscall.LOCK_SH
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:57:20: undefined: syscall.Flock
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:67:20: undefined: syscall.Flock
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:97:20: undefined: syscall.Flock
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:97:50: undefined: syscall.LOCK_UN
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:119:29: undefined: syscall.LOCK_EX
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:131:29: undefined: syscall.LOCK_SH
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:151:17: undefined: syscall.Flock
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:151:52: undefined: syscall.LOCK_NB
../../../../go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:151:52: too many errors
2022/09/06 23:13:03 ----------------------------
2022/09/06 23:13:03 Error compiling js/wasm: exit status 2
2022/09/06 23:13:03 4 compile failures:
js/wasm
plan9/386
plan9/amd64
solaris/amd64
package main
import (
"github.com/gofrs/flock"
"log"
"sync"
"sync/atomic"
)
func main() {
fileLock := flock.New("go-lock.lock")
defer fileLock.Close()
ok := uint64(1)
count := int64(0)
doFunc := func() {
_, err := fileLock.TryLock()
if err != nil {
// handle locking error
return
}
defer fileLock.Unlock()
if v := atomic.AddInt64(&count, 1); v != 1 {
atomic.StoreUint64(&ok, 0)
}
defer atomic.AddInt64(&count, -1)
}
concurrency := 2000
var wg sync.WaitGroup
wg.Add(concurrency)
for i := 0; i < concurrency; i++ {
go func() {
defer wg.Done()
doFunc()
}()
}
wg.Wait()
log.Println("ok", ok)//2022/06/10 14:09:45 ok 0
}
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/go"
GOPRIVATE=""
GOPROXY="https://goproxy.cn,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16.9"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/projects/xxx.xxx.cn/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1083649893=/tmp/go-build -gno-record-gcc-switches"
The third clause of the attached BSD license reads:
- Neither the name of linode-netint nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
Presumably this is a copy/paste error, rather than this software actually deriving from linode-netint (if it is, then the appropriate copyright should also be added).
Given this has already been licensed for ~4 years using a 3rd clause that is not terribly effective, I'd recommend to just relicense to a 2-clause BSD rather than changing the reference to "flock".
Would it be possible to add support for shared locks (LOCK_SH
) in addition to exclusive locks (LOCK_EX
)? Seems like a natural extension, though I'm not sure what the proper API for it would be.
The WinAPI LockFileEx function documentation states:
If the locking process opens the file a second time, it cannot access the specified region through this second handle until it unlocks the region.
However, flock.Flock
does not expose its fh
field. Furthermore, setFh()
is hardcoded to create an os.O_RDONLY
-flagged file handle, excluding write operations even if a user of flock used reflection to obtain the file handle.
FYSA, contrary to the root README, this package does not implement the sync.Locker
interface, primarily because Lock()
has the wrong signature (returns an error rather than having no return). Feel free to close this as WONTFIX, but it'd be nice to update the doc if it's not something worth fixing in code.
I found an issue with TryLock()
on AIX in the scenario where two separate programs are trying to lock the same file.
Scenario: Program A locks file /tmp/lock.file
with TryLock()
. While Program A is holding this lock, program B calls TryLock()
on the same file.
Expect: Program B should return from TryLock()
immediately, failing to obtain lock and nil error
Actual: Program B blocks indefinitely until program A releases the lock
I tested this on both Linux and Windows and they worked as I expected.
I ran the Unit Tests on AIX and they pass without issue. It appears to me that this scenario is tested for at flock_test.go:103
, but I am wondering if this is not the same scenario given the sys calls in the unit tests are made from the same process.
I dug in a little deeper and figured out the issue. I am proposing a fix in #52
Issue for deciding whether changing flock.NewLock
-> flock.New
, is worth the compatibility change.
I'm not sure if this is possible, but I'd be happy to try and help implement it if it's feasible.
My use case is in dokku, where we use this to ensure there aren't concurrent mutations of the docker state by ps:retire
(since the command happens in the background). I'd like to be able to somehow detect that the lock is ancient and then delete the lock in that case (or at least exit with a different exit code). Right now, we don't have a file handle exposed via api that I could use to detect this and I'm looking for the "correct" way to do it if it is at all possible.
Trying to build on Illumos results in the following error:
root@build_env:/tmp# go get -u github.com/gofrs/flock
go: downloading golang.org/x/sys v0.0.0-20210403161142-5e06dd20ab57
# github.com/gofrs/flock
/root/go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:28:22: undefined: syscall.LOCK_EX
/root/go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:39:22: undefined: syscall.LOCK_SH
/root/go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:57:12: undefined: syscall.Flock
/root/go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:67:12: undefined: syscall.Flock
/root/go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:97:12: undefined: syscall.Flock
/root/go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:97:42: undefined: syscall.LOCK_UN
/root/go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:119:21: undefined: syscall.LOCK_EX
/root/go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:131:21: undefined: syscall.LOCK_SH
/root/go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:151:9: undefined: syscall.Flock
/root/go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:151:44: undefined: syscall.LOCK_NB
/root/go/pkg/mod/github.com/gofrs/[email protected]/flock_unix.go:151:44: too many errors
This seems to present a solution.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.