GithubHelp home page GithubHelp logo

lumberjack's Introduction

lumberjack GoDoc Build Status Build status Coverage Status

Lumberjack is a Go package for writing logs to rolling files.

Package lumberjack provides a rolling logger.

Note that this is v2.0 of lumberjack, and should be imported using gopkg.in thusly:

import "gopkg.in/natefinch/lumberjack.v2"

The package name remains simply lumberjack, and the code resides at https://github.com/natefinch/lumberjack under the v2.0 branch.

Lumberjack is intended to be one part of a logging infrastructure. It is not an all-in-one solution, but instead is a pluggable component at the bottom of the logging stack that simply controls the files to which logs are written.

Lumberjack plays well with any logging package that can write to an io.Writer, including the standard library's log package.

Lumberjack assumes that only one process is writing to the output files. Using the same lumberjack configuration from multiple processes on the same machine will result in improper behavior.

Example

To use lumberjack with the standard library's log package, just pass it into the SetOutput function when your application starts.

Code:

log.SetOutput(&lumberjack.Logger{
    Filename:   "/var/log/myapp/foo.log",
    MaxSize:    500, // megabytes
    MaxBackups: 3,
    MaxAge:     28, //days
    Compress:   true, // disabled by default
})

type Logger

type Logger struct {
    // Filename is the file to write logs to.  Backup log files will be retained
    // in the same directory.  It uses <processname>-lumberjack.log in
    // os.TempDir() if empty.
    Filename string `json:"filename" yaml:"filename"`

    // MaxSize is the maximum size in megabytes of the log file before it gets
    // rotated. It defaults to 100 megabytes.
    MaxSize int `json:"maxsize" yaml:"maxsize"`

    // MaxAge is the maximum number of days to retain old log files based on the
    // timestamp encoded in their filename.  Note that a day is defined as 24
    // hours and may not exactly correspond to calendar days due to daylight
    // savings, leap seconds, etc. The default is not to remove old log files
    // based on age.
    MaxAge int `json:"maxage" yaml:"maxage"`

    // MaxBackups is the maximum number of old log files to retain.  The default
    // is to retain all old log files (though MaxAge may still cause them to get
    // deleted.)
    MaxBackups int `json:"maxbackups" yaml:"maxbackups"`

    // LocalTime determines if the time used for formatting the timestamps in
    // backup files is the computer's local time.  The default is to use UTC
    // time.
    LocalTime bool `json:"localtime" yaml:"localtime"`

    // Compress determines if the rotated log files should be compressed
    // using gzip. The default is not to perform compression.
    Compress bool `json:"compress" yaml:"compress"`
    // contains filtered or unexported fields
}

Logger is an io.WriteCloser that writes to the specified filename.

Logger opens or creates the logfile on first Write. If the file exists and is less than MaxSize megabytes, lumberjack will open and append to that file. If the file exists and its size is >= MaxSize megabytes, the file is renamed by putting the current time in a timestamp in the name immediately before the file's extension (or the end of the filename if there's no extension). A new log file is then created using original filename.

Whenever a write would cause the current log file exceed MaxSize megabytes, the current file is closed, renamed, and a new log file created with the original name. Thus, the filename you give Logger is always the "current" log file.

Backups use the log file name given to Logger, in the form name-timestamp.ext where name is the filename without the extension, timestamp is the time at which the log was rotated formatted with the time.Time format of 2006-01-02T15-04-05.000 and the extension is the original extension. For example, if your Logger.Filename is /var/log/foo/server.log, a backup created at 6:30pm on Nov 11 2016 would use the filename /var/log/foo/server-2016-11-04T18-30-00.000.log

Cleaning Up Old Log Files

Whenever a new logfile gets created, old log files may be deleted. The most recent files according to the encoded timestamp will be retained, up to a number equal to MaxBackups (or all of them if MaxBackups is 0). Any files with an encoded timestamp older than MaxAge days are deleted, regardless of MaxBackups. Note that the time encoded in the timestamp is the rotation time, which may differ from the last time that file was written to.

If MaxBackups and MaxAge are both 0, no old log files will be deleted.

func (*Logger) Close

func (l *Logger) Close() error

Close implements io.Closer, and closes the current logfile.

func (*Logger) Rotate

func (l *Logger) Rotate() error

Rotate causes Logger to close the existing log file and immediately create a new one. This is a helper function for applications that want to initiate rotations outside of the normal rotation rules, such as in response to SIGHUP. After rotating, this initiates a cleanup of old log files according to the normal rules.

Example

Example of how to rotate in response to SIGHUP.

Code:

l := &lumberjack.Logger{}
log.SetOutput(l)
c := make(chan os.Signal, 1)
signal.Notify(c, syscall.SIGHUP)

go func() {
    for {
        <-c
        l.Rotate()
    }
}()

func (*Logger) Write

func (l *Logger) Write(p []byte) (n int, err error)

Write implements io.Writer. If a write would cause the log file to be larger than MaxSize, the file is closed, renamed to include a timestamp of the current time, and a new log file is created using the original log file name. If the length of the write is greater than MaxSize, an error is returned.


Generated by godoc2md

lumberjack's People

Contributors

4a6f656c avatar bz2 avatar caibirdme avatar elithrar avatar glaslos avatar jaormx avatar kangxiaoning avatar natefinch avatar nikhita avatar tbutts avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

lumberjack's Issues

Filename field

Hi

I have just updated packages on one of our projects and I see the Filename property has been removed and replaced by Dir and NameFormat. Question is how do I set a custom filename to a log that I want to log to.

Thanks in advance for your response.

Stewart

Please tag most recent release with number greater than all previous releases

Related to #65, making it a tiny bit difficult to continue using this excellent logging library in the age of Go modules.

*  (HEAD -> v2.0, origin/v2.0, origin/HEAD)94d9e49: Deen 2019-04-12 use 0755 to create new dir (#68)
* 4b74a4d: 康晓宁 2019-04-12 fix a typo (#62)
* 2e8fbee: Juan Osorio Robles 2019-04-11 Make default file permissions more restrictive (#83)
*  (tag: v2.0.0)7d6a187: Nate Finch 2018-08-17 Fix test timing (#64)
* aee4629: Tyler Butters 2017-09-11 Update docs, adding `Compress` setting details (#49)
* df99d62: Nate Finch 2017-05-31 switch to travis (#44)
*  (tag: v2.1, origin/master)a96e638: Joel Sing 2017-06-01 Add support for log file compression (#43)
*  (tag: v2.0)dd45e6a: Nate Finch 2016-11-04 update docs w/ backup format info

It doesn´t work in Windows

With a dockerization of the environment and a Windows Host, writing to the host doesn´t create new files.

Set perms on file

I need to set the perms on the log file to 666. How should I achieve this? I can write code to do it in my app, but at what point should I do it? Is the file created when I instantiate the logger object or at the first log entry write? and does lumberjack maintain the perms when the file rolls? Perhaps I should simply create a empty file with the correct perms before starting lumberjack?

Rotate on daily based?

Is that possible to rotate logger based on daily instead maxSize ? Since we want to rotate log file daily irrespectie of the log file size.

Thanks,
Sriram

Memory Leak?

I'm quite new to Go (decades of programming in C, Python and other languages) so please excuse if this is a dumb question.

I coded the following trivial test and let it run for several days. It writes a timestamp and message count to a log file once every 5 seconds. When first launched, top showed 672K memory usage. The memory report has slowly increased every day since. As of now after 5 days running it's at 4880K.

Otherwise the log rotation and compression are happening as expected. The code was built with go1.10.2 darwin/amd64 and is running on OS X El Capitan 10.11.6 Darwin Kernel Version 15.6.0.

// This is a test of go logging using the lumberjack module
// to handle file rotation.
package main

import (
    "log"
    "gopkg.in/natefinch/lumberjack.v2"
    "time"
)

func main() {
log.SetOutput(&lumberjack.Logger{
    Filename:   "/Users/mellis/gologging/logging.log",
    MaxSize:    1, // megabytes
    MaxBackups: 3,
    MaxAge:     7, //days
    Compress:   true, // disabled by default
})

    log.Println("Entering main")
    for i := 0; i < 300000; i++ {
        time.Sleep(5 * time.Second)
        log.Println("message ", i)
    }
    log.Println("Exiting main")
}

What am I doing wrong?
Thanks!

lumberjack_test.go:377: exp: 1 (int), got: 2 (int)

I'm not sure if this issue is related but my applications are not logging anymore since I updated to go 1.4rc1.

If it's a bug in Go 1.4, you may want to hurry since "[it's] scheduled to be released on 1 Dec 2014".

go version go1.4rc1 windows/amd64

go get github.com/natefinch/lumberjack

cd %GOPATH%\src\github.com\natefinch\lumberjack

d:\Dev\go\gopath\src\github.com\natefinch\lumberjack>go test
lumberjack_test.go:377: exp: 1 (int), got: 2 (int)
--- FAIL: TestMaxAge (0.06s)
FAIL
exit status 1
FAIL    github.com/natefinch/lumberjack 0.212s

First write rotation won't clean up backups

If you start your application with multiple backups in your log folder, and your first write to lumberjack would cause the existing file to go over the max size, we just rename the existing file and create a new one, without going through cleanup of old backups.

filename not match the localTime

-rw-r--r-- 1 root root 12M Jan 2 17:08 master-2019-01-02T09-08-11.581.log
-rw-r--r-- 1 root root 12M Jan 2 17:08 master-2019-01-02T09-08-55.835.log
-rw-r--r-- 1 root root 12M Jan 2 17:09 master-2019-01-02T09-09-36.542.log
-rw-r--r-- 1 root root 12M Jan 2 17:10 master-2019-01-02T09-10-17.229.log
-rw-r--r-- 1 root root 12M Jan 2 17:10 master-2019-01-02T09-10-58.189.log
-rw-r--r-- 1 root root 12M Jan 2 17:11 master-2019-01-02T09-11-38.575.log
-rw-r--r-- 1 root root 12M Jan 2 17:12 master-2019-01-02T09-12-22.933.log

the local time is 2019-01-02 17:12 ,but the file name is master-2019-01-02T09-12-22.933.log

go get fails because it can't find go-import tags

go get gopkg.in/natefinch/lumberjack.v2 fails with the following output:

package gopkg.in/natefinch/lumberjack.v2: unrecognized import path "gopkg.in/natefinch/lumberjack.v2" (parse https://gopkg.in/natefinch/lumberjack.v2?go-get=1: no go-import meta tags ())

Rotation loses data

testing this code with sleep works, but without sleep, it's losing the file that should contain "1"

dir, err := ioutil.TempDir("/tmp", "testrotation")

if err != nil {
	log.Fatal(err)
}


fmt.Println(dir)

tmpfn := filepath.Join(dir, "tmpfile.txt")
logger := &lumberjack.Logger{
	Filename:   tmpfn,
	MaxSize:    1, //rotate logs if it reaches 1 megabyte
}

log.SetOutput(logger)

log.Println("1")
logger.Rotate()
log.Println("2")
time.Sleep(2000 * time.Millisecond)
logger.Rotate()
log.Println("3")

UTC time

I have tried setting LocalTime false, true and not using the parameter. Neither of which is resulting in UTC time prepended to the log. Is there something obvious i am missing?

package main

import (
"log"
"gopkg.in/natefinch/lumberjack.v2"
"time"
"fmt"
)
func main() {

log.SetOutput(&lumberjack.Logger{
Filename: "sed.log",
MaxSize: 50, // megabytes
MaxAge: 28, //days
LocalTime: false,
})
localTime := time.Now()
utcTime := localTime.UTC()

// Display the results
fmt.Printf("Local Time: %v\n", localTime)
fmt.Printf("UTC Time: %v\n", utcTime)

log.Println("DEBUG %s", localTime)
log.Println("DEBUG %s", utcTime)

}

go run lumber.go
Local Time: 2017-05-09 12:13:26.205047356 -0700 PDT
UTC Time: 2017-05-09 19:13:26.205047356 +0000 UTC

cat sed.log

2017/05/09 12:13:26 DEBUG %s 2017-05-09 12:13:26.205047356 -0700 PDT
2017/05/09 12:13:26 DEBUG %s 2017-05-09 19:13:26.205047356 +0000 UTC

Log file rotation is failing with exception

We are using lumberjack Version 2
Log file rotation is failing with exception slice out of bound exception
panic: runtime error: slice bounds out of range

Stack trace of exception.

goroutine 1 [running]:
github.com/natefinch/lumberjack.(_Logger).cleanup(0xc0820122a0, 0x0, 0x0)
D:/ADMWorkspace/Cloud Workspace/SDL_operations/src/Godeps/_workspace/src/github.com/natefinch/lumberjack/lumberjack.go:269 +0x692
github.com/natefinch/lumberjack.(_Logger).rotate(0xc0820122a0, 0x0, 0x0)
D:/ADMWorkspace/Cloud Workspace/SDL_operations/src/Godeps/_workspace/src/github.com/natefinch/lumberjack/lumberjack.go:179 +0xbc
github.com/natefinch/lumberjack.(_Logger).Write(0xc0820122a0, 0xc082036d00, 0x7a, 0xca, 0x0, 0x0, 0x0)
D:/ADMWorkspace/Cloud Workspace/SDL_operations/src/Godeps/_workspace/src/github.com/natefinch/lumberjack/lumberjack.go:131 +0x405
bytes.(_Buffer).WriteTo(0xc082030460, 0xc94520, 0xc0820122a0, 0x0, 0x0, 0x0)
c:/go/src/bytes/buffer.go:206 +0xcf
io.copyBuffer(0xc94520, 0xc0820122a0, 0xc945a8, 0xc082030460, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)

Please help us to resolve this issue

Option to rotate based on time instead of size

Right now it appears that MaxSize is what determines when rotation should be done. However a common way to rotate log files (e.g. with https://github.com/logrotate/logrotate) is by time ("daily","weekly", etc.).

One way to do this that would be quite flexible would be to add a Cron string which is parsed using something like https://godoc.org/github.com/robfig/cron - this way daily, weekly, hourly, etc. could all easily be implemented without special code. However, that adds a dependency where there is none right now.

Another approach would be to add a Pattern string which could be set to let's say "hourly" or "daily" - setting a timer in Go code specifically for the next hour or daily boundary is somewhat trivial, no external dependency needed. Then, later, if so desired, Pattern could be expanded to support a cron expression or other more complex rotation scheme, without breaking backward compatibility. I haven't looked too deeply into the code but I think this could be easy to add and would not break stuff for existing users.

Thoughts?

goroutine leak

millrun goroutine leak.

I use zap to wrap lumberjack, my code is following

w := zapcore.AddSync(&lumberjack.Logger{
  Filename:   "/var/log/myapp/foo.log",
  MaxSize:    500, // megabytes
  MaxBackups: 3,
  MaxAge:     28, // days
})
core := zapcore.NewCore(
  zapcore.NewJSONEncoder(zap.NewProductionEncoderConfig()),
  w,
  zap.InfoLevel,
)
logger := zap.New(core)

when I do testing, I got the following error:

Too many goroutines running after all test(s).
1 instances of:

**/vendor/gopkg.in/natefinch/lumberjack%2ev2.(*Logger).millRun(...)
**/vendor/gopkg.in/natefinch/lumberjack.v2/lumberjack.go:379 +0x58
created by **/vendor/gopkg.in/natefinch/lumberjack%2ev2.(*Logger).mill.func1
***/vendor/gopkg.in/natefinch/lumberjack.v2/lumberjack.go:390 +0x7e
ERROR: *manager.**TestSuite is failed by leak goroutine check.

I tried to used Close() method, but it still failed.

I read source code and fount that nowhere to close the logger.millCh, so this goroutine will live forever.

Lumberjack does not test if provided Filename is writeable when initializing Logger

Using Lumberjack v2: gopkg.in/natefinch/lumberjack.v2 v2.0.0

		logFile := &lumberjack.Logger{
			Filename:   "/unwriteable/file.log",
			MaxSize:    10, // megabytes
			MaxBackups: 3,
			MaxAge:     28, //days
			Compress:   true,
		}

Given that "/unwriteable/file.log" has 444 permissions, I found that Logger didn't complain at all that it couldn't write to the provided file. Is this intended behavior?

Log Rotator not working when passing file as variable

Hi Team,

I am passing filename as variable in function but it seems log rotator is not working .Here is the snippet of my code:

logfile := logdir + "/monitoring.log"
log.SetOutput(&lumberjack.Logger{
Filename: logfile,
MaxSize: 5, // megabytes
MaxBackups: 3,
})

Simultaneous log access

Hi.
Can you clarify the statement "Lumberjack assumes that only one process is writing to the output files. Using the same lumberjack configuration from multiple processes on the same machine will result in improper behavior."

Does this mean that only one goroutine within an executable can use log.Write, or that only one executable can write to any particular file?

Thank you.

Why does openNew not use O_APPEND when open a new logfile?

When open existing log file, the openExistingOrNew function use OpenFile with O_APPEND flag, but when creating a new log file, openNew use OpenFile without O_APPEND flag. The two behavior are different. Could anybody tell me why ? In my opinion, openNew should use O_APPEND flag as same as openExistingOrNew does.

The file opened with O_APPEND flag will allow multi-thread/multi-process writing if every write is smaller than 4k. We do not need to use mutex to protect the write.

File doesn't appear to close on Windows

package main

import (
    "log"
    "time"

    "github.com/natefinch/lumberjack"
)

func main() {
    lj := &lumberjack.Logger{
        Dir:        `C:\Logs\`,
        NameFormat: time.RFC822 + ".log",
        MaxSize:    lumberjack.Gigabyte,
        MaxBackups: 3,
        MaxAge:     28,
    }
    log.SetOutput(lj)

    log.Println("I want something to happen!")

    if err := lj.Close(); err != nil {
        log.Println("error", err)
    }
}

This code snippet has two interesting problems that are occurring on Windows 8.1

When using log.Print() sample

The log file is simply not written to, all Write() calls succeed, but in the end the file is 0KB (though it is created).

In one application that's larger and not self-contained this if statement is evaluating true despite all logging calls succeeding without failure https://github.com/natefinch/lumberjack/blob/master/lumberjack.go#L151

In this small self-contained app that if statement is not evaluating true, but the behavior is the same.

After debugging statements were inserted and using log.Println() in the sample

This leads me to believe there may be a race condition somewhere as all I did was insert print statements. Windows fails and I can't run go -race right now, is there a chance there's this kind of problem in the library?

fatal error: all goroutines are asleep - deadlock!

goroutine 1 [semacquire]:
sync.runtime_Semacquire(0xc0840071e4)
        C:/Users/ADMINI~1/AppData/Local/Temp/2/makerelease155988687/go/src/pkg/runtime/sema.goc:199 +0x33
sync.(*Mutex).Lock(0xc0840071e0)
        C:/Users/ADMINI~1/AppData/Local/Temp/2/makerelease155988687/go/src/pkg/sync/mutex.go:66 +0xd9
log.(*Logger).Output(0xc0840071e0, 0x2, 0xc084004200, 0x30, 0x0, ...)
        C:/Users/ADMINI~1/AppData/Local/Temp/2/makerelease155988687/go/src/pkg/log/log.go:134 +0x98
log.Printf(0x4cf740, 0x2, 0x302d18, 0x1, 0x1)
        C:/Users/ADMINI~1/AppData/Local/Temp/2/makerelease155988687/go/src/pkg/log/log.go:276 +0x80
github.com/natefinch/lumberjack.(*Logger).Write(0xc08402f1e0, 0xc084004180, 0x30, 0x40, 0x0, ...)
        C:/Projects/Go/src/github.com/natefinch/lumberjack/lumberjack.go:118 +0x178
log.(*Logger).Output(0xc0840071e0, 0x2, 0xc0840055e0, 0x1c, 0x0, ...)
        C:/Users/ADMINI~1/AppData/Local/Temp/2/makerelease155988687/go/src/pkg/log/log.go:153 +0x408
log.Println(0x302ef8, 0x1, 0x1)
        C:/Users/ADMINI~1/AppData/Local/Temp/2/makerelease155988687/go/src/pkg/log/log.go:282 +0x6c
main.main()
exit status 2

lumberjack has increasing number of file descriptors open for the same logfile

I am using lumberjack in conjunction with zerolog like this:

logRotator := lumberjack.Logger{
		Filename:   "/tmp/logging.log",
		MaxBackups: 7,
		MaxSize:    500,
		MaxAge:     10,
	}

log := zerolog.New(&logRotator).Timestamp().Logger()

After checking the open files I see that the number of file descriptors is always increasing until it hit the max and you cannot do anything with the system.

This is what I get after (around) 4mins.

$ lsof | grep logging | wc -l
19374

version 2 on github does not match examples

It appears that when I try to pass a Filename as in the example using the v2 branch on github, it does not work. I checked the code and the Filename is not present in the v2 branch of github.

log.SetOutput(&lumberjack.Logger{
    Filename: "somefile",
    MaxSize:  10,
})

panic: runtime error: slice bounds out of range

I like the idea.
I tested it and met an issue.
Here is the code to reproduce.

package main

import (
    "log"
    "net/http"
    "time"

    "github.com/natefinch/lumberjack"
)

func main() {
    log.SetOutput(&lumberjack.Logger{
        Dir:        "log",
        NameFormat: "2006-01-02T01-01-01.000.log",
        MaxSize:    lumberjack.Megabyte,
        MaxBackups: 3,
        MaxAge:     28,
    })

    for {
        log.Println("----")
        time.Sleep(time.Microsecond)
    }
}

The output is:

panic: runtime error: slice bounds out of range

goroutine 16 [running]:
runtime.panic(0x6d14c0, 0x8ba0af)
    /usr/local/go/src/pkg/runtime/panic.c:279 +0xf5
github.com/natefinch/lumberjack.(*Logger).cleanup(0xc208004300, 0x0, 0x0)
    .../github.com/natefinch/lumberjack/lumberjack.go:269 +0x500
github.com/natefinch/lumberjack.(*Logger).rotate(0xc208004300, 0x0, 0x0)
    .../github.com/natefinch/lumberjack/lumberjack.go:179 +0xaf
github.com/natefinch/lumberjack.(*Logger).Write(0xc208004300, 0xc20803faa0, 0x19, 0x20, 0x0, 0x0, 0x0)

The reason is oldLogFiles() return empty slice.

default log permissions are too permissive

The default permissions for the logs are set to 644, which is very permissive, as anyone can read the log files; which can be a security issue depending on the context.

I propose having a default of 600, which allows the application to read and write its own logs; and leaving root/admin users still with the ability to read the logs (which would normally need to see the logs).

New Release

I suggest adding a new release, files have been updated recently that change the basic API (ex: Dir->Filename), but the current release is from ~4 years ago.

This complicates dependency management with glide/godep.

preamble lines that prefix every new log file

At program startup, I record the version info of the program to my log. It would be nice to declare these log lines as a part of a preamble that every new log file starts with. That way the critical version of code and the command-line and config arguments are preserved, even if it is a month later and the original log has long ago been rotated away.

Your rotate metod blocks logging at all while rotation, gzip files and other operations

func (l *Logger) Rotate() error {
	l.mu.Lock()
	defer l.mu.Unlock()
	return l.rotate()
}

So while you rename, gzip, reopen files logger Write blocks and wait while operation ends.
This is not right way.

Right way is do that in the following order:

  1. rename files + mowe current file to xxx.log.1
  2. open new log file.
  3. l.mu.Lock(), assign new opened file, l.mu.Unlock
  4. do all other operations all you want.(gzip some of files, etc.)

In this case you make Write to wait while switch file to new.
And you need add another mutex to logger to guarantee that Logger.Rotate() operations running only once a time.

panic: runtime error: slice bounds out of range in lumberjack.go:269

Hi,

After about 20-ish hours of working my application crashes with the following panic() in lumberjack.

Could there be some kind of out of order condition that could cause files[] to panic?

Using version:

                    "ImportPath": "github.com/natefinch/lumberjack",
                    "Comment": "v1.0-2-ga6f35ba",
                    "Rev": "a6f35bab25c9df007f78aa90c441922062451979"

panic: runtime error: slice bounds out of range

goroutine 61 [running]:
github.com/natefinch/lumberjack.(*Logger).cleanup(0xc208082d20, 0x0, 0x0)
    /obfuscated/Godeps/_workspace/src/github.com/natefinch/lumberjack/lumberjack.go:269 +0x513
github.com/natefinch/lumberjack.(*Logger).rotate(0xc208082d20, 0x0, 0x0)
    /obfuscated/Godeps/_workspace/src/github.com/natefinch/lumberjack/lumberjack.go:179 +0xb0
github.com/natefinch/lumberjack.(*Logger).Write(0xc208082d20, 0xc208e45900, 0x1cf, 0x247, 0x0, 0x0, 0x0)
    /obfuscated/Godeps/_workspace/src/github.com/natefinch/lumberjack/lumberjack.go:131 +0x316
bytes.(*Buffer).WriteTo(0xc20b172e00, 0x7f5f3f4a57e0, 0xc208082d20, 0x0, 0x0, 0x0)
    /usr/src/go/src/bytes/buffer.go:202 +0xda
io.Copy(0x7f5f3f4a57e0, 0xc208082d20, 0x7f5f3f4a5790, 0xc20b172e00, 0x0, 0x0, 0x0)
    /usr/src/go/src/io/io.go:354 +0xb2
github.com/Sirupsen/logrus.(*Entry).log(0xc2097d4140, 0x4, 0xc209018030, 0x2c)
    /obfuscated/Godeps/_workspace/src/github.com/Sirupsen/logrus/entry.go:94 +0x4d1
github.com/Sirupsen/logrus.(*Entry).Info(0xc2097d4140, 0xc217d53ba8, 0x1, 0x1)
    /obfuscated/Godeps/_workspace/src/github.com/Sirupsen/logrus/entry.go:119 +0x7f

This looks like this area of code for me:

    if l.MaxBackups > 0 {
        deletes = files[l.MaxBackups:]
        files = files[:l.MaxBackups]
    }

Rotation based on day

Hello,

Would you be interessed by adding the possibility to rotate log file based on day please ?
If I do not any mistake, rotation is only activated by the size.
The idea would be to also activate it by day : rotate everyday for example.

Thanks a lot.

Logging mutli-files

Hi,
I wanted to know if it was possible to write the logs to differents files.
I would like to write the log.Error in one file and the log.Info in another file.
Thanks in advance.

Cannot use lumberjack in Go WebServer App?

"Lumberjack assumes that only one process is writing to the output files. Using the same lumberjack configuration from multiple processes on the same machine will result in improper behavior."

Is it means that we shouldn't use Lumberjack in Golang WebServer or any Go application with Go routines concurrently write to the log file?

thank you

Callback on rotation

Would be nice if lumberjack on rotation of the log file performed a callback so that additional action could be performed on the logs. This could be shoehorned in with the compression rotation #13 issue as well.

Details

  1. The interface would be RotateLogs(os.File) so exact location of the new file could be determined.
  2. The callback would be performed in a new thread so the logger will not need to be held up by any tasks that are being performed by the callback (for example compression).
  3. A public variable should be available within lumberjack so that a "default" compression utility could be used.

Just some thoughts..

Add Flush() method to Logger

There's no way to flush the log files to disk ... I occasionally noticed that if the go process has been killed or the system rebooted that the log file is incomplete.

Having a Flush() method on Logger that just calls Sync() on the underlying os.File would allow me to run a goroutine that periodically calls Flush() to ensure that the logs are written to disk.

Lumberjack loggers are not garbage-collected, which can leak file handles

Currently, every instance of a lumberjack logger that has ever performed a write will have an associated running goroutine. Because this goroutine references the logger, the logger is never garbage-collected, even if it otherwise goes out of scope. Because the logger holds a handle to the file it is logging to, this can leak file descriptors.

If #56 is fixed using the currently proposed solution, then calling Close() on the logger will stop the goroutine, which will allow it to be GC'd.

However, given that not all callers will call Close(), it would be nice to have a design where, if a logger is GC'd, that would ensure that the associated file handles would be closed.

The branch at https://github.com/jdhenke/lumberjack/tree/avoid-goroutine-cleanup demonstrates an approach that implements this fix. In this approach, the "mill" goroutine ends once its work is complete. Locks are used to ensure that only 1 mill routine is running for a logger at any given time, and subsequent requests will queue mill work.

The following test demonstrates the file handle GC behavior. This test fails with the current code, but succeeds on the branch linked above:

func TestFileHandleReleasedOnGC(t *testing.T) {
	dir := makeTempDir("TestFileHandleReleasedOnGC", t)
	defer os.RemoveAll(dir)
	filename := logFile(dir)

	equals(0, len(linesWithString(lsofOutput(t), ".log")), t)

	logger1 := &Logger{
		Filename: filename,
	}
	_, _ = logger1.Write([]byte("foo!"))

	equals(1, len(linesWithString(lsofOutput(t), ".log")), t)

	_ = logger1
	runtime.GC()

	// there should be no more references to the logger, so GC should have closed file handle
	equals(0, len(linesWithString(lsofOutput(t), ".log")), t)
}

func linesWithString(content, find string) []string {
	var output []string
	for _, line := range strings.Split(content, "\n") {
		if strings.Contains(line, find) {
			output = append(output, line)
		}
	}
	return output
}

func lsofOutput(t *testing.T) string {
	lsofOutput, err := exec.Command("lsof", "-p", strconv.Itoa(os.Getpid())).Output()
	require.NoError(t, err)
	return string(lsofOutput)
}

Another advantage of this approach is that a logger will only have a mill goroutine running when it is actively doing mill work (whereas currently every logger that has ever performed a write will have an associated goroutine).

Is it possible to disable rotation?

In some environments, it might be advisable to disable log rotation and let another process handle that work.

I originally thought that setting MaxSize: 0 would disable log rotation, but the func (l *Logger) max() function returns the defaultMaxSize:

// max returns the maximum size in bytes of log files before rolling.
func (l *Logger) max() int64 {
	if l.MaxSize == 0 {
		return int64(defaultMaxSize * megabyte)
	}
	return int64(l.MaxSize) * int64(megabyte)
}

to preserve existing behavior, would you be open to adding a disableFileRotation field to the Logger struct and replacing the various:

if info.Size()+int64(writeLen) >= l.max() {
    return l.rotate()
}

with:

func(l *Logger) shouldRotate(currentSize, writeLen int64) bool {
   if l.disableFileRotation {
        return false
   }
   return currentSize + writeLen >= l.max()
}

This should be backward compatible behavior, since the default value for disableFileRotation would be false and only interested parties would set it to true.

Another alternative would be to set MaxSize to -1 and change behavior accordingly, but many consumer of lumberjack already error on negative values.

smaller sizes increments

it's too bad that the file size rotation limit is in megabytes - that's actually a huge number for embeded platforms. it would be better to rotate on kilobytes or, fundamentally, bytes: one can then use multiples to find the proper value using, e.g. humanize: fmt.Println("10 MB: %d", 10 * humanize.MiByte)...

i understand this may involve an API change, but maybe there could just be another MaxBytes settings?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.