Go package for reading from continously updated files (tail -f)
License: MIT License
A Go package striving to emulate the features of the BSD tail program.
t, err := tail.TailFile("/var/log/nginx.log", tail.Config{Follow: true})
for line := range t.Lines {
fmt.Println(line.Text)
}See API documentation.
Tail comes with full support for truncation/move detection as it is designed to work with log rotation tools.
go get github.com/hpcloud/tail/...
This package needs assistance for full Windows support.
../../hpcloud/tail/watch/inotify.go:13:2: no buildable Go source files in /opt/local/go16/manual/src/github.com/hpcloud/tail/vendor/gopkg.in/fsnotify.v1
Thanks for the excellent library :),
I want to request support for seeking by line number, as in the linux tail command:
-n, --lines=[+]NUM
output the last NUM lines, instead of the last 10; or use -n
+NUM to output starting with line NUM
Both options would be useful, I know that Go doesn't natively support seeking files by line number, but could probably come up with something fairly efficient, like:
If I find some time I'll try to work on this, and will update the issue if I do.
Is it possible that the raw result(ex./var/log/audit/audit.log) can be decoded?
the function comment states blocks until the missing file comes into existence. If the file already exists, block until it is recreated - yet the polling watcher returns immediately if the file exists.
i think the inotify watcher should do the same, else there is a potential race condition in tail.go:reopen.
Hello,
I'm using this func
err := t.Stop()
to stop tailing activity on a file, however this not stops the tailing. Is this the correct aproach to stop tailing ? If not, how can I achieve my goal ?
Thanks in advance
go version go1.3 windows/amd64
Windows 7 64 bits
Le processus ne peut pas accéder au fichier car ce fichier est utilisé par un autre processus. mean The process can't access the file because it's used by another process.
go test
2014/06/26 09:39:29 Waiting for /no/such/file to appear...
2014/06/26 09:39:29 Waiting for _no_such_file to appear...
--- FAIL: TestMaxLineSize (0.10 seconds)
tail_test.go:356: remove .test/maxlinesize/test.txt: Le processus ne peut pas accéder au fichier car ce fichier est utilisé par un autre processus.
--- FAIL: TestOver4096ByteLine (0.10 seconds)
tail_test.go:356: remove .test/Over4096ByteLine/test.txt: Le processus ne peut pas accéder au fichier car ce fichier est utilisé par un autre processus.
--- FAIL: TestOver4096ByteLineWithSetMaxLineSize (0.10 seconds)
tail_test.go:356: remove .test/Over4096ByteLineMaxLineSize/test.txt: Le processus ne peut pas accéder au fichier car ce fichier est utilisé par un autre
processus.
--- FAIL: TestLocationFull (0.10 seconds)
tail_test.go:356: remove .test/location-full/test.txt: Le processus ne peut pas accéder au fichier car ce fichier est utilisé par un autre processus.
2014/06/26 09:39:29 Stopping tail as file no longer exists: .test/location-full/test.txt
2014/06/26 09:39:29 Re-opening truncated file .test/maxlinesize/test.txt ...
2014/06/26 09:39:29 Successfully reopened truncated .test/maxlinesize/test.txt
2014/06/26 09:39:29 Stopping tail as file no longer exists: .test/Over4096ByteLine/test.txt
2014/06/26 09:39:29 Re-opening truncated file .test/Over4096ByteLineMaxLineSize/test.txt ...
2014/06/26 09:39:29 Successfully reopened truncated .test/Over4096ByteLineMaxLineSize/test.txt
2014/06/26 09:39:29 Seeked .test/location-end/test.txt - &{Offset:0 Whence:2}
--- FAIL: TestLocationEnd (0.20 seconds)
tail_test.go:356: remove .test/location-end/test.txt: Le processus ne peut pas accéder au fichier car ce fichier est utilisé par un autre processus.
2014/06/26 09:39:30 Re-opening truncated file .test/location-end/test.txt ...
2014/06/26 09:39:30 Successfully reopened truncated .test/location-end/test.txt
2014/06/26 09:39:30 Seeked .test/location-end/test.txt - &{Offset:-6 Whence:2}
--- FAIL: TestLocationMiddle (0.21 seconds)
tail_test.go:356: remove .test/location-end/test.txt: Le processus ne peut pas accéder au fichier car ce fichier est utilisé par un autre processus.
--- FAIL: TestReOpenInotify (0.10 seconds)
tail_test.go:356: remove .test/reopen-inotify/test.txt: Le processus ne peut pas accéder au fichier car ce fichier est utilisé par un autre processus.
--- FAIL: TestReOpenPolling (0.30 seconds)
tail_test.go:356: remove .test/reopen-polling/test.txt: Le processus ne peut pas accéder au fichier car ce fichier est utilisé par un autre processus.
2014/06/26 09:39:30 Re-opening truncated file .test/reseek-inotify/test.txt ...
2014/06/26 09:39:30 Successfully reopened truncated .test/reseek-inotify/test.txt
--- FAIL: TestReSeekInotify (0.20 seconds)
tail_test.go:356: remove .test/reseek-inotify/test.txt: Le processus ne peut pas accéder au fichier car ce fichier est utilisé par un autre processus.
--- FAIL: TestReSeekPolling (0.20 seconds)
tail_test.go:356: remove .test/reseek-polling/test.txt: Le processus ne peut pas accéder au fichier car ce fichier est utilisé par un autre processus.
2014/06/26 09:39:31 Leaky bucket full (.test/rate-limiting/test.txt); entering 1s cooloff period.
2014/06/26 09:39:31 Re-opening truncated file .test/reseek-polling/test.txt ...
2014/06/26 09:39:31 Successfully reopened truncated .test/reseek-polling/test.txt
2014/06/26 09:39:32 Leaky bucket full (.test/rate-limiting/test.txt); entering 1s cooloff period.
--- FAIL: TestRateLimiting (1.30 seconds)
tail_test.go:356: remove .test/rate-limiting/test.txt: Le processus ne peut pas accéder au fichier car ce fichier est utilisé par un autre processus.
2014/06/26 09:39:32 Seeked .test/tell-position/test.txt - &{Offset:0 Whence:0}
2014/06/26 09:39:32 Seeked .test/tell-position/test.txt - &{Offset:12 Whence:0}
--- FAIL: TestTell (0.00 seconds)
tail_test.go:356: remove .test/tell-position/test.txt: Le processus ne peut pas accéder au fichier car ce fichier est utilisé par un autre processus.
FAIL
exit status 1
FAIL github.com/ActiveState/tail 3.369s
Following files on Windows is buggy, using both file events and polling. Sometimes you never get events for new lines.
Current code misses the NotifyDeleted()/NotifyTruncated()/NotifyModified() signals if the other end of the channel is busy processing somethings else. Changing the channels [https://github.com/hpcloud/tail/blob/v1.0.0/watch/filechanges.go#L4] into buffered channel of size 1 ensures that one consolidated instance for each of these signals are retained till the other end is frees up for processing.
I'm trying this out on Windows, and everything seems to be working out of the box except I'm getting a NUL character between every character that's printed out.
Here's the code:
t, err := tail.TailFile("foo", tail.Config{Follow: true, Logger: tail.DiscardingLogger})
if err != nil {
log.Fatal(err)
return
}
for line := range t.Lines {
fmt.Println(line.Text)
}
Any idea why this is happening?
Deferred RemoveWatch() [https://github.com/hpcloud/tail/blob/v1.0.0/watch/inotify.go#L78] may at times cause the WatchCreate (called indirectly by NotifyDeleted() ) to run before RemoveWatch(). This prevents the WatchCreate from adding a watcher on the file’s parent directory for watching file creates, since the data structures still reflects as if the file is being being watched.
Possibly Fixes
Issue #90
running Go 1.8 under macOS Sierra 10.12.3 (16D32), HEAD at faf842b
~/.../hpcloud/tail $ go test
2017/02/23 23:27:19 Waiting for /no/such/file to appear...
2017/02/23 23:27:19 Waiting for .test/waits-for-file-to-exist/test.txt to appear...
2017/02/23 23:27:19 Waiting for test.txt to appear...
2017/02/23 23:27:20 Waiting for _no_such_file to appear...
2017/02/23 23:27:20 Seeked .test/location-end/test.txt - &{Offset:0 Whence:2}
2017/02/23 23:27:20 Seeked .test/location-middle/test.txt - &{Offset:-6 Whence:2}
2017/02/23 23:27:21 Re-opening moved/deleted file .test/reopen-inotify/test.txt ...
2017/02/23 23:27:21 Waiting for .test/reopen-inotify/test.txt to appear...
2017/02/23 23:27:22 Re-opening moved/deleted file .test/reopen-polling/test.txt ...
2017/02/23 23:27:22 Waiting for .test/reopen-polling/test.txt to appear...
2017/02/23 23:27:22 Successfully reopened .test/reopen-polling/test.txt
2017/02/23 23:27:22 Re-opening moved/deleted file .test/reopen-polling/test.txt ...
2017/02/23 23:27:22 Waiting for .test/reopen-polling/test.txt to appear...
2017/02/23 23:27:23 Successfully reopened .test/reopen-polling/test.txt
2017/02/23 23:27:23 Re-opening moved/deleted file .test/reopen-polling/test.txt ...
2017/02/23 23:27:23 Waiting for .test/reopen-polling/test.txt to appear...
2017/02/23 23:27:23 Stopping tail as file no longer exists: .test/reseek-inotify/test.txt
--- FAIL: TestReSeekInotify (0.20s)
tail_test.go:499: tail ended early; expecting more: [h311o w0r1d endofworld]
2017/02/23 23:27:23 Re-opening truncated file .test/reseek-polling/test.txt ...
2017/02/23 23:27:23 Successfully reopened truncated .test/reseek-polling/test.txt
2017/02/23 23:27:24 Leaky bucket full (.test/rate-limiting/test.txt); entering 1s cooloff period.
2017/02/23 23:27:24 Stopping tail as file no longer exists: .test/reseek-polling/test.txt
2017/02/23 23:27:25 Leaky bucket full (.test/rate-limiting/test.txt); entering 1s cooloff period.
2017/02/23 23:27:25 Seeked .test/tell-position/test.txt - &{Offset:0 Whence:0}
2017/02/23 23:27:25 Seeked .test/tell-position/test.txt - &{Offset:12 Whence:0}
2017/02/23 23:27:25 Waiting for .test/block-until-file-exists/test.txt to appear...
FAIL
exit status 1
FAIL github.com/hpcloud/tail 5.565s
Using OS X El Capitan. Tail.Lines is open, but nothing ever comes out. I've poked around with various options to see if I can get it working, but nothing ever works unless I use Poll: true in the Config passed to TailFile. I'll try and open a PR with a failing test sometime in the next week or two.
when waiting on BlockUntilExists due to file recreations (tail.go:reopen), if the user calls tail.Stop - it would block forever if the file is never going to be recreated.
this sometime causes the test to hang:
2013/05/28 12:38:07 Re-opening moved/deleted/truncated file .test/reseek-inotify/test.txt ...
2013/05/28 12:38:07 Waiting for .test/reseek-inotify/test.txt to appear...
Stopping...
os: centos 7
Repeat steps：
touch /tmp/test.log
cat tail.go
package main
import (
"github.com/ActiveState/tail"
"log"
)
func main() {
t, err := tail.TailFile("/tmp/test.log", tail.Config{Follow:true, ReOpen:true})
if err != nil {
log.Panic(err)
}
for line := range t.Lines {
log.Printf(line.Text)
}
}
go run tail.go &
rm /tmp/test.log
touch /tmp/test.log
echo "111" >> /tmp/test.log
you can see tail.go failed to reopen /tmp/test.log, because /tmp/test.log is already opened by tail.go,rm didn't really delete this file.
If the tailed file has dos line endings, they are not trimmed properly.
Given the following input file:
line number 1
line number 2
If the file has dos line endings the "%q\n" formatted lines are:
"line number 1\r"
"line number 2\r"
I am expecting the same lines as if it had unix line endings:
"line number 1"
"line number 2"
Steps to repeat:
Write a simple Tail client in the fashion of:
package main
import (
"github.com/ActiveState/tail"
"log"
)
func main() {
t, err := tail.TailFile("/tmp/log", tail.Config{Follow:true, ReOpen:true})
if err != nil {
log.Panic(err)
}
for line := range t.Lines {
log.Printf(line.Text)
}
}
Then, run the program (go run .../test.go). In another window, run echo line1 > /tmp/log and then echo line2 >> /tmp/log.
Expected result:
The Go program should print out two lines, line1 and line2.
Actual result:
The Go program only prints out one line.
Platform:
Linux maw 3.8.0-26-generic #38-Ubuntu SMP Mon Jun 17 21:43:33 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
Hi,
Nice library!
I'd like to be able to tell when the file is reopened. However having to deal with an event (channel, error code, ...) in case of reopen feels like a kludge. What about simply adding a byte position of the start of each line in the Line structure. There is already the timestamp. This way users can detect when the file was reopened because the position is no longer monotonically increasing.
What do you think?
I have been trying to tail multiple files, and not getting responses on 2nd (...n) files. Unit test here, possibly im doing something else wrong, and poked around trying to fix underlying issue but didn't immediately come to me so thought I would drop this test in case anyone else sees it.
tail__.go and watch/polling__.go only builds on Windows/Linux. Build fails on OS X(darwin) with
go test -v ./...
# github.com/ActiveState/tail/watch
watch/polling.go:73: undefined: permissionErrorRetry
# _/Users/rickard/src/go-code/src/github.com/ActiveState/tail/watch
watch/polling.go:73: undefined: permissionErrorRetry
FAIL github.com/ActiveState/tail [build failed]
make: *** [test] Error 2
Error was introduced in: c2a48be
Can you please add some usage examples. For example, tailing a log and sending json metrics to a NoSQL database.
Currently the Config.Logger is of type *log.Logger, which is a Logger struct:
http://golang.org/pkg/log/#Logger
This makes using alternative logging implementations, eg https://github.com/Sirupsen/logrus fairly difficult, because it is a bare structure reference, and not an Interface -- so the only real option is to pass an io.Writer into log.New, and then proxy that data to Logrus.
Ideally tail could make an interface of the functions it actually wants for logging, and making a proxy type to logrus would be easy.
Hi, I'm new to Go and have written a very simple program to watch some logs (which rotate). However although the program runs fine for some time, eventually it deadlocks with the following message:
fatal error: all goroutines are asleep - deadlock!
The code I'm using looks like this:
t, err := tail.TailFile("path/to/some/log", tail.Config{
Follow: true,
ReOpen: true})
if err != nil {
log.Fatal(err)
}
for line := range t.Lines {
// Some action based on a log line
}
I can provide a stacktrace if necessary
BTW: I'm on Windows.
Exits the Program without throwing any error on Ubuntu Linux x86_64, though it works perfectly on Macintosh.
*Already Added these flags: *
Follow: true,
ReOpen: true
Last line to be printed by Lib is as below:
2016/05/05 11:41:38 Re-opening moved/deleted file /home/ubuntu/server/tomcat/apache-tomcat-7.0.69/logs/catalina.out ...
2016/05/05 11:41:38 Waiting for /home/ubuntu/server/tomcat/apache-tomcat-7.0.69/logs/catalina.out to appear...
After this it just exists without any error.
would be nice :-)
$ go get -v github.com/ActiveState/tail
github.com/ActiveState/tail (download)
Fetching https://gopkg.in/tomb.v1?go-get=1
Parsing meta tags from https://gopkg.in/tomb.v1?go-get=1 (status code 200)
...
How do you get github to redirect to gopkg.in?
When go get it go get github.com/hpcloud/tail/...,the following error happend:
# github.com/hpcloud/tail/watch
hpcloud/tail/watch/inotify.go:31: fw.w.WatchFlags undefined (type *fsnotify.Watcher has no field or method WatchFlags)
hpcloud/tail/watch/inotify.go:31: undefined: fsnotify.FSN_CREATE
hpcloud/tail/watch/inotify.go:35: fw.w.RemoveWatch undefined (type *fsnotify.Watcher has no field or method RemoveWatch)
hpcloud/tail/watch/inotify.go:46: fw.w.Event undefined (type *fsnotify.Watcher has no field or method Event)
hpcloud/tail/watch/inotify.go:46: select case must be receive, send or assign recv
hpcloud/tail/watch/inotify.go:62: fw.w.Watch undefined (type *fsnotify.Watcher has no field or method Watch)
hpcloud/tail/watch/inotify.go:70: fw.w.RemoveWatch undefined (type *fsnotify.Watcher has no field or method RemoveWatch)
hpcloud/tail/watch/inotify.go:76: undefined: fsnotify.FileEvent
hpcloud/tail/watch/inotify.go:80: fw.w.Event undefined (type *fsnotify.Watcher has no field or method Event)
hpcloud/tail/watch/inotify.go:80: select case must be receive, send or assign recv
hpcloud/tail/watch/inotify.go:80: too many errorsBuilds using the newest version of howeyc/fsnotify, (those post this commit) don't pass tests/work : howeyc/fsnotify@3bfa915
I am in an environment with quite rapid/intense logging, and I am using tail to watch these log files. I have been hitting a repeatable issue where after log rotation, tail simply failed to pick up the new log files, and I believe I've tracked down why.
In ChangeEvents() in watch/polling.go, changes.NotifyDeleted() is used if the file either goes away or is a different file from the open file. In both cases, changes.NotifyDeleted() is called and then the goroutine exits---so only one deletion event will ever be sent from any given polling goroutine for any given deletion/rotation.
Unfortunately, these deletion events can get lost: looking in NotifyDeleted() in watch/filechanges.go, it only sends the deletion event down the channel if the channel is empty at the time. This means if there's still a pending modification event, then the deletion event will be dropped on the floor, and because the goroutine that sent the deletion event has returned, there will not be another deletion event sent, and the file will not be reopened.
I instrumented sendOnlyIfEmpty to log whether delete requests were in fact being dropped on the floor (by checking whether ch == fc.Deleted) and they were.
Locally, I've hacked around this by making NotifyDeleted() just send down the channel whether or not there's space and block if it has to, and this seems to have worked for my use case so far.
On OS X 10.10 (beta) with go 1.3
Running the tests approximately 50% of the times it fails with:
$ make test
go test -v ./...
panic: readdirent: invalid argument
goroutine 16 [running]:
runtime.panic(0x13d1c0, 0x2082a0200)
/opt/boxen/homebrew/Cellar/go/1.3/libexec/src/pkg/runtime/panic.c:279 +0xf5
github.com/ActiveState/tail.init·1()
/Users/rickard/src/go-code/src/github.com/ActiveState/tail/tail_test.go:23 +0x74
github.com/ActiveState/tail.init()
/Users/rickard/src/go-code/src/github.com/ActiveState/tail/tail_test.go:428 +0x17f
main.init()
github.com/ActiveState/tail/_test/_testmain.go:76 +0x46
goroutine 19 [finalizer wait]:
runtime.park(0x14590, 0x2495d8, 0x248989)
/opt/boxen/homebrew/Cellar/go/1.3/libexec/src/pkg/runtime/proc.c:1369 +0x89
runtime.parkunlock(0x2495d8, 0x248989)
/opt/boxen/homebrew/Cellar/go/1.3/libexec/src/pkg/runtime/proc.c:1385 +0x3b
runfinq()
/opt/boxen/homebrew/Cellar/go/1.3/libexec/src/pkg/runtime/mgc0.c:2644 +0xcf
runtime.goexit()
/opt/boxen/homebrew/Cellar/go/1.3/libexec/src/pkg/runtime/proc.c:1445
FAIL github.com/ActiveState/tail 0.006s
? github.com/ActiveState/tail/cmd/gotail [no test files]
=== RUN TestPour
--- PASS: TestPour (0.00 seconds)
=== RUN TestTimeSinceLastUpdate
--- PASS: TestTimeSinceLastUpdate (0.00 seconds)
=== RUN TestTimeToDrain
--- PASS: TestTimeToDrain (0.00 seconds)
PASS
ok github.com/ActiveState/tail/ratelimiter 0.005s
? github.com/ActiveState/tail/util [no test files]
? github.com/ActiveState/tail/watch [no test files]
make: *** [test] Error 1
I get the same error when running packer built from source.
Due to fsnotify/fsnotify#73 the logic in watch/inotify_tracker.go is now subject to frequent deadlocks: all you need is to try to Cleanup() a Tail object while fsnotify is trying to send a notification. With the recent fsnotify changes, Cleanup() ends up making InotifyTracker.run() call shared.removeWatch(winfo), which then calls shared.watcher.Remove(fname), which then waits until the main inotify loop is done handling the IN_IGNORED event indicating that the file is no longer being watched by the kernel. But for the inotify loop to handle this event, it needs to not be stuck on trying to send an event. Hence the deadlock if it's already in the process of trying to send an event.
https://github.com/hpcloud/tail/blob/master/ratelimiter/leakybucket.go
newfill float64 = b.Fill + float64(amount)
b.Fill -= float64(elapsed) / float64(b.LeakInterval)
It is strange: add amount, dec time ?!
should be:
--- a/src/github.com/hpcloud/tail/ratelimiter/leakybucket.go
+++ b/src/github.com/hpcloud/tail/ratelimiter/leakybucket.go
@@ -30,7 +30,8 @@ func (b *LeakyBucket) updateFill() {
if b.Fill > 0 {
elapsed := now.Sub(b.Lastupdate)
- b.Fill -= float64(elapsed) / float64(b.LeakInterval)
+ // fix bug: bucket algo err
+ b.Fill -= float64(b.Size) * float64(elapsed) / float64(b.LeakInterval)
if b.Fill < 0 {
b.Fill = 0
}
@@ -40,7 +41,6 @@ func (b *LeakyBucket) updateFill() {
func (b *LeakyBucket) Pour(amount uint16) bool {
b.updateFill()
-
var newfill float64 = b.Fill + float64(amount)
if newfill > float64(b.Size) {
@@ -54,7 +54,7 @@ func (b *LeakyBucket) Pour(amount uint16) bool {
// The time at which this bucket will be completely drained
func (b *LeakyBucket) DrainedAt() time.Time {
- return b.Lastupdate.Add(time.Duration(b.Fill * float64(b.LeakInterval)))
+ return b.Lastupdate.Add(time.Duration(b.Fill / float64(b.Size) * float64(b.LeakInterval)))
}
this leakybucket's rate: size/interval
burst: size
please see: https://en.wikipedia.org/wiki/Leaky_bucket#Parameters
Currently you import "github.com/howeyc/fsnotify". Could you please switch to import "gopkg.in/fsnotify.v0" instead? It should be a drop-in replacement. This helps when I have other code that needs to use fsnotify.
Tailing a file reads the whole file from start. We need a switch which will only read new lines after the start of tailing.
When I attempt to build, this error occurs:
# github.com/hpcloud/tail/
../github.com/hpcloud/tail/tail.go:18: non-canonical import path "github.com/hpcloud/tail//vendor/gopkg.in/tomb.v1" (should be "github.com/hpcloud/tail/vendor/gopkg.in/tomb.v1")
../github.com/hpcloud/tail/tail.go:18: can't find import: "gopkg.in/tomb.v1"
Any idea how to fix this?
I'm using this package like this:
tail.TailFile("/path/to/some.log", tail.Config{Follow: true, ReOpen: true, Location: &tail.SeekInfo{Offset: 0, Whence: 2}})So when the file is first opened for tailing the seek is waiting for the next line. When the file is truncated/moved/deleted by logrotate or similar, the reopen happens, but the initially applied tail.SeekInfo is not applied on the reopen, and I get the last N of lines in the log (whatever the default is) before the follow commences.
Not clear to me if this is intended behavior, or a bug.
go get launchpad.net/tomb
will give following result
Not checking SSL certificate for launchpad.net
Not checking SSL certificate for launchpad.net
bzr: ERROR: Invalid http response for https://launchpad.net/tomb/.bzr/branch-format: Unable to handle http code 503: Service Unavailable
bzr: warning: some compiled extensions could not be loaded; see <https://answers.launchpad.net/bzr/+faq/703>
package launchpad.net/tomb: exit status 3Windows OS, go 1.7.4
cfg := tail.Config{ReOpen: true, MustExist: false, Follow: true}
t, err := tail.TailFile("D:\log\hotelprice\server\hotelprice_log4.log", cfg)
if err != nil {
fmt.Println(err)
}
for line := range t.Lines {
fmt.Println(line.Text)
}
the hotelprice_log4.log file is not exist.
I'm using this to tail across the wire and need to cancel when the client disconnects. Tail is stuck in sendLine (tail.go line 418), could use a context.Context here as I'm now piling up stuck goroutines. Thoughts?
A go get gopkg.in/ActiveState/tail.v0 (to use the old stable API) will also fetch the master repo, which is due to the import of the fullly-qualified "github.com/ActiveState/tail/..." URL in several .go files.
Replacing them with relative URLs does not work at all:
> make fulltest
[...]
can't load package: /go/src/github.com/ActiveState/tail/tail.go:15:2: local import "./ratelimiter" in non-local package
Consequently it becomes impossible to go get on the old API unless the user manually clones the repo under $GOPATH/src/github.com/ActiveState/tail
(this shouldn't affect the Stackato folks who use git commit hash to clone the repo)
even for a simple example like:
t, err := tail.TailFile("/var/log/nginx.log", tail.Config{Follow: true})
for line := range t.Lines {
fmt.Println(line.Text)
}after reading the file to tail, changes after reaching EOF are not detected with fsnotify and polling.
example, i want to tail a bytes.Buffer object.
required for streaming logs from docker container.
Is it possible using the library to get notified, when the file gets truncated or deleted?
An fd-per-watch quickly consumes more file descriptors than are allowed. Would you accept a patch that uses a single inotify instance and multiple watchers? This appears to be the approach recommended by the Linux kernel docs.
There's a stumbling block that would require modifying the API. The new approach would eliminate the global inotify tracker variable and create new file watchers using a supplied instance of a tracker.
Please let me know what you think!
P.S. This is for Papertrail's remote_syslog2, to resolve this issue.
./gotail -F /root/messages
Jun 22 14:42:43 node-247 SWARM: time="2017-06-22T14:42:44.507886233+08:00" level=debug msg="3"
when delete /root/messages and recreate it, gotail not detect file, sos
I want to suggest another path, which is unix sockets.
For example, we are using many docker containers on one host, and I'd like to be able to get the logs on real time from the docker.sock.
Thanks
D.
The following code reproduces this.
// test.go
package main
import (
"github.com/hpcloud/tail"
"io/ioutil"
)
func main() {
ioutil.WriteFile("test.log", []byte("hoge\n"), 0644)
t, _ := tail.TailFile("test.log", tail.Config{})
t.Stop()
}> go run test.go
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan receive]:
github.com/hpcloud/tail/vendor/gopkg.in/tomb%2ev1.(*Tomb).Wait(0xc82007a078, 0x0, 0x0)
/Users/skaji/src/github.com/hpcloud/tail/vendor/gopkg.in/tomb.v1/tomb.go:113 +0x58
github.com/hpcloud/tail.(*Tail).Stop(0xc82007a000, 0x0, 0x0)
/Users/skaji/src/github.com/hpcloud/tail/tail.go:164 +0x5f
main.main()
/Users/skaji/test.go:12 +0xee
goroutine 5 [chan send]:
github.com/hpcloud/tail.(*Tail).sendLine(0xc82007a000, 0xc82000a420, 0x4, 0x0)
/Users/skaji/src/github.com/hpcloud/tail/tail.go:418 +0x23a
github.com/hpcloud/tail.(*Tail).tailFileSync(0xc82007a000)
/Users/skaji/src/github.com/hpcloud/tail/tail.go:272 +0x686
created by github.com/hpcloud/tail.TailFile
/Users/skaji/src/github.com/hpcloud/tail/tail.go:133 +0x421
exit status 2
Inotify tail doesn't seem to properly work for files that are created and deleted quickly.
Using:
36ba8bc github.com/ActiveState/tail (heads/master)
Using the tailing config:
o := tail.Config{
Location: s,
// Poll: true,
ReOpen: true,
Follow: true,
MustExist: false,
}
func waitForChanges need to call Cleanup()
I've got a file which is growing fast (a kernel packet log). This is line based, and lines are usually around 900 bytes long.
Sometimes range t.Lines gives a partial line (not ending in "\n"), even though I have not set MaxLineSize. This line is then 512 bytes long.
It seems this is caused by this if:
https://github.com/ActiveState/tail/blob/master/tail.go#L217
although I would expect the next 'if' to be the one we want in this case (wait for more data to complete the line).
Linux 3.13.7, go 1.3 and this is used to open the file:
t, err := tail.TailFile(file, tail.Config{
Follow: true,
ReOpen: true,
Location: &tail.SeekInfo{
Offset: 0,
Whence: os.SEEK_END,
},
})
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.