GithubHelp home page GithubHelp logo

logrus's Introduction

Logrus :walrus: Build Status Build Status Go Reference

Logrus is a structured logger for Go (golang), completely API compatible with the standard library logger.

Logrus is in maintenance-mode. We will not be introducing new features. It's simply too hard to do in a way that won't break many people's projects, which is the last thing you want from your Logging library (again...).

This does not mean Logrus is dead. Logrus will continue to be maintained for security, (backwards compatible) bug fixes, and performance (where we are limited by the interface).

I believe Logrus' biggest contribution is to have played a part in today's widespread use of structured logging in Golang. There doesn't seem to be a reason to do a major, breaking iteration into Logrus V2, since the fantastic Go community has built those independently. Many fantastic alternatives have sprung up. Logrus would look like those, had it been re-designed with what we know about structured logging in Go today. Check out, for example, Zerolog, Zap, and Apex.

Seeing weird case-sensitive problems? It's in the past been possible to import Logrus as both upper- and lower-case. Due to the Go package environment, this caused issues in the community and we needed a standard. Some environments experienced problems with the upper-case variant, so the lower-case was decided. Everything using logrus will need to use the lower-case: github.com/sirupsen/logrus. Any package that isn't, should be changed.

To fix Glide, see these comments. For an in-depth explanation of the casing issue, see this comment.

Nicely color-coded in development (when a TTY is attached, otherwise just plain text):

Colored

With log.SetFormatter(&log.JSONFormatter{}), for easy parsing by logstash or Splunk:

{"animal":"walrus","level":"info","msg":"A group of walrus emerges from the
ocean","size":10,"time":"2014-03-10 19:57:38.562264131 -0400 EDT"}

{"level":"warning","msg":"The group's number increased tremendously!",
"number":122,"omg":true,"time":"2014-03-10 19:57:38.562471297 -0400 EDT"}

{"animal":"walrus","level":"info","msg":"A giant walrus appears!",
"size":10,"time":"2014-03-10 19:57:38.562500591 -0400 EDT"}

{"animal":"walrus","level":"info","msg":"Tremendously sized cow enters the ocean.",
"size":9,"time":"2014-03-10 19:57:38.562527896 -0400 EDT"}

{"level":"fatal","msg":"The ice breaks!","number":100,"omg":true,
"time":"2014-03-10 19:57:38.562543128 -0400 EDT"}

With the default log.SetFormatter(&log.TextFormatter{}) when a TTY is not attached, the output is compatible with the logfmt format:

time="2015-03-26T01:27:38-04:00" level=debug msg="Started observing beach" animal=walrus number=8
time="2015-03-26T01:27:38-04:00" level=info msg="A group of walrus emerges from the ocean" animal=walrus size=10
time="2015-03-26T01:27:38-04:00" level=warning msg="The group's number increased tremendously!" number=122 omg=true
time="2015-03-26T01:27:38-04:00" level=debug msg="Temperature changes" temperature=-4
time="2015-03-26T01:27:38-04:00" level=panic msg="It's over 9000!" animal=orca size=9009
time="2015-03-26T01:27:38-04:00" level=fatal msg="The ice breaks!" err=&{0x2082280c0 map[animal:orca size:9009] 2015-03-26 01:27:38.441574009 -0400 EDT panic It's over 9000!} number=100 omg=true

To ensure this behaviour even if a TTY is attached, set your formatter as follows:

	log.SetFormatter(&log.TextFormatter{
		DisableColors: true,
		FullTimestamp: true,
	})

Logging Method Name

If you wish to add the calling method as a field, instruct the logger via:

log.SetReportCaller(true)

This adds the caller as 'method' like so:

{"animal":"penguin","level":"fatal","method":"github.com/sirupsen/arcticcreatures.migrate","msg":"a penguin swims by",
"time":"2014-03-10 19:57:38.562543129 -0400 EDT"}
time="2015-03-26T01:27:38-04:00" level=fatal method=github.com/sirupsen/arcticcreatures.migrate msg="a penguin swims by" animal=penguin

Note that this does add measurable overhead - the cost will depend on the version of Go, but is between 20 and 40% in recent tests with 1.6 and 1.7. You can validate this in your environment via benchmarks:

go test -bench=.*CallerTracing

Case-sensitivity

The organization's name was changed to lower-case--and this will not be changed back. If you are getting import conflicts due to case sensitivity, please use the lower-case import: github.com/sirupsen/logrus.

Example

The simplest way to use Logrus is simply the package-level exported logger:

package main

import (
  log "github.com/sirupsen/logrus"
)

func main() {
  log.WithFields(log.Fields{
    "animal": "walrus",
  }).Info("A walrus appears")
}

Note that it's completely api-compatible with the stdlib logger, so you can replace your log imports everywhere with log "github.com/sirupsen/logrus" and you'll now have the flexibility of Logrus. You can customize it all you want:

package main

import (
  "os"
  log "github.com/sirupsen/logrus"
)

func init() {
  // Log as JSON instead of the default ASCII formatter.
  log.SetFormatter(&log.JSONFormatter{})

  // Output to stdout instead of the default stderr
  // Can be any io.Writer, see below for File example
  log.SetOutput(os.Stdout)

  // Only log the warning severity or above.
  log.SetLevel(log.WarnLevel)
}

func main() {
  log.WithFields(log.Fields{
    "animal": "walrus",
    "size":   10,
  }).Info("A group of walrus emerges from the ocean")

  log.WithFields(log.Fields{
    "omg":    true,
    "number": 122,
  }).Warn("The group's number increased tremendously!")

  log.WithFields(log.Fields{
    "omg":    true,
    "number": 100,
  }).Fatal("The ice breaks!")

  // A common pattern is to re-use fields between logging statements by re-using
  // the logrus.Entry returned from WithFields()
  contextLogger := log.WithFields(log.Fields{
    "common": "this is a common field",
    "other": "I also should be logged always",
  })

  contextLogger.Info("I'll be logged with common and other field")
  contextLogger.Info("Me too")
}

For more advanced usage such as logging to multiple locations from the same application, you can also create an instance of the logrus Logger:

package main

import (
  "os"
  "github.com/sirupsen/logrus"
)

// Create a new instance of the logger. You can have any number of instances.
var log = logrus.New()

func main() {
  // The API for setting attributes is a little different than the package level
  // exported logger. See Godoc.
  log.Out = os.Stdout

  // You could set this to any `io.Writer` such as a file
  // file, err := os.OpenFile("logrus.log", os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0666)
  // if err == nil {
  //  log.Out = file
  // } else {
  //  log.Info("Failed to log to file, using default stderr")
  // }

  log.WithFields(logrus.Fields{
    "animal": "walrus",
    "size":   10,
  }).Info("A group of walrus emerges from the ocean")
}

Fields

Logrus encourages careful, structured logging through logging fields instead of long, unparseable error messages. For example, instead of: log.Fatalf("Failed to send event %s to topic %s with key %d"), you should log the much more discoverable:

log.WithFields(log.Fields{
  "event": event,
  "topic": topic,
  "key": key,
}).Fatal("Failed to send event")

We've found this API forces you to think about logging in a way that produces much more useful logging messages. We've been in countless situations where just a single added field to a log statement that was already there would've saved us hours. The WithFields call is optional.

In general, with Logrus using any of the printf-family functions should be seen as a hint you should add a field, however, you can still use the printf-family functions with Logrus.

Default Fields

Often it's helpful to have fields always attached to log statements in an application or parts of one. For example, you may want to always log the request_id and user_ip in the context of a request. Instead of writing log.WithFields(log.Fields{"request_id": request_id, "user_ip": user_ip}) on every line, you can create a logrus.Entry to pass around instead:

requestLogger := log.WithFields(log.Fields{"request_id": request_id, "user_ip": user_ip})
requestLogger.Info("something happened on that request") # will log request_id and user_ip
requestLogger.Warn("something not great happened")

Hooks

You can add hooks for logging levels. For example to send errors to an exception tracking service on Error, Fatal and Panic, info to StatsD or log to multiple places simultaneously, e.g. syslog.

Logrus comes with built-in hooks. Add those, or your custom hook, in init:

import (
  log "github.com/sirupsen/logrus"
  "gopkg.in/gemnasium/logrus-airbrake-hook.v2" // the package is named "airbrake"
  logrus_syslog "github.com/sirupsen/logrus/hooks/syslog"
  "log/syslog"
)

func init() {

  // Use the Airbrake hook to report errors that have Error severity or above to
  // an exception tracker. You can create custom hooks, see the Hooks section.
  log.AddHook(airbrake.NewHook(123, "xyz", "production"))

  hook, err := logrus_syslog.NewSyslogHook("udp", "localhost:514", syslog.LOG_INFO, "")
  if err != nil {
    log.Error("Unable to connect to local syslog daemon")
  } else {
    log.AddHook(hook)
  }
}

Note: Syslog hook also support connecting to local syslog (Ex. "/dev/log" or "/var/run/syslog" or "/var/run/log"). For the detail, please check the syslog hook README.

A list of currently known service hooks can be found in this wiki page

Level logging

Logrus has seven logging levels: Trace, Debug, Info, Warning, Error, Fatal and Panic.

log.Trace("Something very low level.")
log.Debug("Useful debugging information.")
log.Info("Something noteworthy happened!")
log.Warn("You should probably take a look at this.")
log.Error("Something failed but I'm not quitting.")
// Calls os.Exit(1) after logging
log.Fatal("Bye.")
// Calls panic() after logging
log.Panic("I'm bailing.")

You can set the logging level on a Logger, then it will only log entries with that severity or anything above it:

// Will log anything that is info or above (warn, error, fatal, panic). Default.
log.SetLevel(log.InfoLevel)

It may be useful to set log.Level = logrus.DebugLevel in a debug or verbose environment if your application has that.

Note: If you want different log levels for global (log.SetLevel(...)) and syslog logging, please check the syslog hook README.

Entries

Besides the fields added with WithField or WithFields some fields are automatically added to all logging events:

  1. time. The timestamp when the entry was created.
  2. msg. The logging message passed to {Info,Warn,Error,Fatal,Panic} after the AddFields call. E.g. Failed to send event.
  3. level. The logging level. E.g. info.

Environments

Logrus has no notion of environment.

If you wish for hooks and formatters to only be used in specific environments, you should handle that yourself. For example, if your application has a global variable Environment, which is a string representation of the environment you could do:

import (
  log "github.com/sirupsen/logrus"
)

func init() {
  // do something here to set environment depending on an environment variable
  // or command-line flag
  if Environment == "production" {
    log.SetFormatter(&log.JSONFormatter{})
  } else {
    // The TextFormatter is default, you don't actually have to do this.
    log.SetFormatter(&log.TextFormatter{})
  }
}

This configuration is how logrus was intended to be used, but JSON in production is mostly only useful if you do log aggregation with tools like Splunk or Logstash.

Formatters

The built-in logging formatters are:

  • logrus.TextFormatter. Logs the event in colors if stdout is a tty, otherwise without colors.
    • Note: to force colored output when there is no TTY, set the ForceColors field to true. To force no colored output even if there is a TTY set the DisableColors field to true. For Windows, see github.com/mattn/go-colorable.
    • When colors are enabled, levels are truncated to 4 characters by default. To disable truncation set the DisableLevelTruncation field to true.
    • When outputting to a TTY, it's often helpful to visually scan down a column where all the levels are the same width. Setting the PadLevelText field to true enables this behavior, by adding padding to the level text.
    • All options are listed in the generated docs.
  • logrus.JSONFormatter. Logs fields as JSON.

Third party logging formatters:

You can define your formatter by implementing the Formatter interface, requiring a Format method. Format takes an *Entry. entry.Data is a Fields type (map[string]interface{}) with all your fields as well as the default ones (see Entries section above):

type MyJSONFormatter struct {
}

log.SetFormatter(new(MyJSONFormatter))

func (f *MyJSONFormatter) Format(entry *Entry) ([]byte, error) {
  // Note this doesn't include Time, Level and Message which are available on
  // the Entry. Consult `godoc` on information about those fields or read the
  // source of the official loggers.
  serialized, err := json.Marshal(entry.Data)
    if err != nil {
      return nil, fmt.Errorf("Failed to marshal fields to JSON, %w", err)
    }
  return append(serialized, '\n'), nil
}

Logger as an io.Writer

Logrus can be transformed into an io.Writer. That writer is the end of an io.Pipe and it is your responsibility to close it.

w := logger.Writer()
defer w.Close()

srv := http.Server{
    // create a stdlib log.Logger that writes to
    // logrus.Logger.
    ErrorLog: log.New(w, "", 0),
}

Each line written to that writer will be printed the usual way, using formatters and hooks. The level for those entries is info.

This means that we can override the standard library logger easily:

logger := logrus.New()
logger.Formatter = &logrus.JSONFormatter{}

// Use logrus for standard log output
// Note that `log` here references stdlib's log
// Not logrus imported under the name `log`.
log.SetOutput(logger.Writer())

Rotation

Log rotation is not provided with Logrus. Log rotation should be done by an external program (like logrotate(8)) that can compress and delete old log entries. It should not be a feature of the application-level logger.

Tools

Tool Description
Logrus Mate Logrus mate is a tool for Logrus to manage loggers, you can initial logger's level, hook and formatter by config file, the logger will be generated with different configs in different environments.
Logrus Viper Helper An Helper around Logrus to wrap with spf13/Viper to load configuration with fangs! And to simplify Logrus configuration use some behavior of Logrus Mate. sample

Testing

Logrus has a built in facility for asserting the presence of log messages. This is implemented through the test hook and provides:

  • decorators for existing logger (test.NewLocal and test.NewGlobal) which basically just adds the test hook
  • a test logger (test.NewNullLogger) that just records log messages (and does not output any):
import(
  "github.com/sirupsen/logrus"
  "github.com/sirupsen/logrus/hooks/test"
  "github.com/stretchr/testify/assert"
  "testing"
)

func TestSomething(t*testing.T){
  logger, hook := test.NewNullLogger()
  logger.Error("Helloerror")

  assert.Equal(t, 1, len(hook.Entries))
  assert.Equal(t, logrus.ErrorLevel, hook.LastEntry().Level)
  assert.Equal(t, "Helloerror", hook.LastEntry().Message)

  hook.Reset()
  assert.Nil(t, hook.LastEntry())
}

Fatal handlers

Logrus can register one or more functions that will be called when any fatal level message is logged. The registered handlers will be executed before logrus performs an os.Exit(1). This behavior may be helpful if callers need to gracefully shutdown. Unlike a panic("Something went wrong...") call which can be intercepted with a deferred recover a call to os.Exit(1) can not be intercepted.

...
handler := func() {
  // gracefully shutdown something...
}
logrus.RegisterExitHandler(handler)
...

Thread safety

By default, Logger is protected by a mutex for concurrent writes. The mutex is held when calling hooks and writing logs. If you are sure such locking is not needed, you can call logger.SetNoLock() to disable the locking.

Situation when locking is not needed includes:

  • You have no hooks registered, or hooks calling is already thread-safe.

  • Writing to logger.Out is already thread-safe, for example:

    1. logger.Out is protected by locks.

    2. logger.Out is an os.File handler opened with O_APPEND flag, and every write is smaller than 4k. (This allows multi-thread/multi-process writing)

      (Refer to http://www.notthewizard.com/2014/06/17/are-files-appends-really-atomic/)

logrus's People

Contributors

aarongreenlee avatar al3x avatar alxdm avatar aybabtme avatar bunyk avatar chimeracoder avatar coilysiren avatar dclendenan avatar dgsb avatar didip avatar dmathieu avatar fkollmann avatar flimzy avatar gigaroby avatar markphelps avatar mattbostock avatar meatballhat avatar neilisaac avatar noxiouz avatar onetwopunch avatar phemmer avatar rasky avatar roganartu avatar sirupsen avatar squirkle avatar stevvooe avatar tandr avatar tgwizard avatar thajeztah avatar yawn avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

logrus's Issues

Breaking out msg/time/level

The fact that msg, time, and level have special meaning, but are treated as standard fields leads to a lot of headaches IMO. I have more than once passed a field called "msg" that collided with the built-in msg (it quietly is thrown away). The fact that you have to type-assert these mandatory fields is also an opportunity for bugs.

I recommend that msg, time and level be promoted to actual struct fields in Entry. This will give them types and will allow these field names to exist in the Fields map.

Log rotation documentation

I'm not actually asking for log rotation as a logrus feature. I think it's best managed through a separate Writer. But is there a rotating Writer that's recommended for use with logrus?

formatted data being sent to syslog

When using the packaged syslog hook, the data being sent to syslog uses the format sent to the default destination. This means that syslog is receiving a payload with the severity level in the message content, when it already has severity as part of the syslog headers. It has a timestamp when syslog has a timestamp. It has ANSI color escape codes, and alignment formatting.

Basically each output destination needs its own formatter.

Cross compiling doesn't work

I'm using goxc to cross compile my project. Doing so now results in:

../../Sirupsen/logrus/text_formatter.go:28: undefined: IsTerminal
[goxc:xc] 2015/01/16 20:50:30 'go' returned error: exit status 2
[goxc:xc] 2015/01/16 20:50:30 Error: exit status 2
[goxc:xc] 2015/01/16 20:50:30 Have you run `goxc -t` for this platform (amd64p32,nacl)???

This error is repeated for every platform.

I'm using c6a969a and Go 1.4 on Ubuntu 14.04.
Running goxc in the examples/basic directory is enough to reproduce the problem. I have ran goxc -t of course and I know this worked previous to introducing logrus.

JSON prefix clash avoidance uses old messages

I'm not sure how many other people use logrus in this manner, but it's very common that I create a single log instance to use throughout a HTTP route or log-running method, so that I can set the fields once and continue to use this. I find this incredibly useful for tracking the file or route that I'm working on, etc.

Unfortunately, the prefix-clash copies the last-used message as fields.msg, causing each log line after the first to contain the message for the previous log. Here's a small program that demonstrates this:

package main

import (
    log "github.com/Sirupsen/logrus"
)


func init() {
  // Log as JSON instead of the default ASCII formatter.
  log.SetFormatter(&log.JSONFormatter{})
}

func main() {
    ll := log.WithFields(log.Fields{"test_field":"test_field_data"})
    ll.Info("Test Message 1")
    ll.Info("Test Message 2")
}

This will output the following (I've pretty printed it):

{
  "level": "info",
  "msg": "Test Message 1",
  "test_field": "test_field_data",
  "time": "2014-12-09T18:11:38-08:00"
}
{
  "fields.level": "info",
  "fields.msg": "Test Message 1",
  "fields.time": "2014-12-09T18:11:38-08:00",
  "level": "info",
  "msg": "Test Message 2",
  "test_field": "test_field_data",
  "time": "2014-12-09T18:11:38-08:00"
}

Notice how the second log has the correct message in the message field, but fields.msg contains the msg body from the previous. Furthermore, the fields data is not present in the first log at all.

Errors in fields are empty when using JSONFormatter

JSONFormatter seems to have trouble with errors passed to log.Fields

log.SetFormatter(&log.JSONFormatter{})
err := errors.New("this will come up as {}")
log.WithFields(log.Fields{
  "err": err,
}).Warn("the error field will be empty")

The code above will output out the following:

{"err":{},"level":"warning","msg":"the error field will be empty","time":"2015-01-16T20:51:40-08:00"}

The TextFormatter works as expected so this issue is specific to the JSONFormatter as far as I can tell.

Multiple output writers depending on log level

glog has this nice feature where it can log to stderr on top of logging to its normal output, if a certain threshold is met:

See flag -stderrthreshold=ERROR in glog.

Something like that would be very useful.

Logrus can overwrite user supplied fields

I'm opening this issue so we can discuss the drawbacks and advantages of the multiple options to solve the issue of logrus overwriting user supplied fields.

Num Options Nested Output fields as expected Potentially erases user fields
1 Extract msg/time/level to a struct Yes Yes No
2 Prefix user supplied keys. No No No
3 All in Data No Yes Will silently erase msg / time / level fields.
3 All in Data, prefix logrus fields No Yes Will silently erase prefix.msg / prefix.time / prefix.level fields.

option 1 #33

The argument against option 1 is that nested 'fields' might cause problems with log aggregators. I'm not sure about that.

option 2

The argument against 2 is that what the user logs will be different from what the logs contain. For instance:

log.WithFields(logrus.Fields{
    "msg": "derp message",
}).Info("received message")

The field msg will be prefixed with fields, avoiding collisions:

{
    "time": "00:00:00 ...", 
    "level": "info", 
    "msg": "received message", 
    "fields.msg": "derp message",
}

While it's true that the name of the user supplied field is not the same the one provided,
it's not very hard to figure out where it comes from. It also makes it clear that this field is a
user supplied field. With a clear documentation to that effect, I think that would be a viable
solution.

option 3 (current situation)

The argument against 3 is that it erases user supplied fields. The fact that it
does that is not documented, and the logger doesn't warn or panic or anything
when it erases the field.

All this together looks very bad to me, if the other
options aren't optimal, at least they are correct and don't lose data.

option 4 (mitigate current situation)

This is a sort of hybrid of option 2 and 3. The argument against it
is that it can still erase user supplied fields, although less likely; and
it's kind of a bandaid solution.

The concept is to use a prefix that is unlikely to be chosen by a user:

log.WithFields(logrus.Fields{
    "msg": "derp message",
}).Info("received message")

All logrus default fields are prefixed by the string logrus:

{
    "logrus.time": "00:00:00 ...", 
    "logrus.level": "info", 
    "logrus.msg": "received message", 
    "msg": "derp message",
}

You can then document that the keys of logrus are in a namespace logrus.something, and
tell users not to use that name space. Perhaps even check, when adding a field,
that users don't use logrus reserved names, and panic if they do.

Appengine Managed VMs

Logrus won't compile when using AppEngine Managed VM's because it's missing the IsTerminal() function, since there is a build flag on the terminal_notwindows.go file. Removing this flag allows it compile. The package won't run on regular app engine anyway since there are other syscalls.
Is there any reason to keep this build flag?

Default instance of logger

To have a similar behavior as the go standard log package, it may be interesting to have a default instance to do something like logrus.Debug("Something") directly without instanciating a new logger.

What do you think of that?

Papertrail logs entry.Data["level"] incorrectly

I've poked through the source to see if I could resolve this myself but couldn't figure it out.

Logs being sent to Papertrail are incorrectly logging entry.Data["level"] Papertrail sees all of the logs as %!s(<nil>)

Incorrect use of syslog hook plugin in README

In README:

import (
  log "github.com/Sirupsen/logrus"
  "github.com/Sirupsen/logrus/hooks/airbrake"
  "github.com/Sirupsen/logrus/hooks/syslog"
)

func init() {
  log.AddHook(new(logrus_airbrake.AirbrakeHook))
  log.AddHook(logrus_syslog.NewSyslogHook("udp", "localhost:514", syslog.LOG_INFO, ""))
}

But NewSyslogHook returns both a hook and an error, os it should be:

sh, err := log.AddHook(logrus_syslog.NewSyslogHook("udp", "localhost:514", syslog.LOG_INFO, ""))
if err == nil {
  log.AddHook(sh)
}

Also, log/syslog should be imported to get syslog.LOG_INFO.

Airbrake hook package name

It would be nice if the package name of the airbrake hook would match the import path:

There are two reasons for this.

  1. Mismatch between import strings and package name are always a bit confusing. Especially for newcomers to Go, but even for experienced users as it's usual that import string match package names.
  2. logrus_airbrake is an odd package name. Super scientific smoke test: http://www.pkgname.com/?pkgname=logrus_airbrake

Set level with string

It would be nice to be able to set the level with a string. I'm using something like this to set the level from CLI flags:

func SetLogLevel(level string) {
    switch level {
    case "panic":
        log.SetLevel(log.PanicLevel)
    case "fatal":
        log.SetLevel(log.FatalLevel)
    case "error":
        log.SetLevel(log.ErrorLevel)
    case "warn":
        log.SetLevel(log.WarnLevel)
    case "info":
        log.SetLevel(log.InfoLevel)
    case "debug":
        log.SetLevel(log.DebugLevel)
    }
}

Strange log entry fields behavior

Want to reuse several fields between log entries. Do the following:

l := log.WithFields(log.Fields{
    ".message-id": messageId,
    ".job-name": jobName,
})

then:

l.Info("started")
l.Info("finished")

got:

INFO[0000] started                                       .job-name=select-vk-groups .message-id=4204cb9a
INFO[0000] finished                                      .job-name=select-vk-groups .message-id=4204cb9a fields.level=info fields.msg=started fields.time=2014-08-23T02:49:37+03:00

I didn't expect these fields.* fields to be displayed. Also, looks like depend on the previous state: fields.msg=started

Add syslog hooks

It would be cool to have syslog hooks so logs can be routed to logstash/heka/papertrail/splunk.

Unexpected behavior when using hooks

I ran into an unexpected issue with the hooks functionality.

It was my impression that invoking log.Error("something") would set the entry.Data["error"] field to the message, in this case "something"

It seems the "error" field is only set if using log.WithFields().Error()

Is this by design or unintentional?

Package exported logger.

Like the standard log library, and the glog project; having a package global logger is very convenient to avoid having to pass a logger everywhere in your application.

I propose adding the following package exported funcs with a default logger to stdout:

  • WithField
  • WithFields
  • Debug
  • Info
  • Warn
  • Error
  • Panic
  • Fatal

Here I don't list the Print and Warning family of functions because I don't think they are useful given the other levels. Also, the f and ln families, because:

  • Printf like funcs are covered by WithField(s)
  • Println like funcs are redundant with the Print like funcs, since each log entry is a line anyways.

Allow configuring the "time" field format

Hello! Very happy with logrus so far, thank you for your work! :)

I'd like to be able to configure the appearance of the "time" field. Specifically, my intention is to use RFC3339Nano in UTC.

Any thoughts on how to approach this? I attempted to use a hook to replace the time field with my own string, but that doesn't seem to work; specifically, inside the hook the time field is changed, but after the hook it has the same original value (something not being passed as a pointer?).

Sample code attempted:

type LogrusTimeFixerHook struct {}
func (l *LogrusTimeFixerHook) Fire(entry *logrus.Entry) error {
  fmt.Println("Fire!")
  entry.Data["time"] = time.Now().UTC().Format(time.RFC3339Nano)
  fmt.Printf("%#v\n", entry.Data)
  return nil
}

func (l *LogrusTimeFixerHook) Levels() []logrus.Level {
  return []logrus.Level{ logrus.Panic, logrus.Fatal, logrus.Error, logrus.Warn, logrus.Info, logrus.Debug }
}

func init() {
  log.Formatter = new(logrus.JSONFormatter)
  log.Level = logrus.Warn
  log.Hooks.Add(new(LogrusTimeFixerHook))
  log.Warn("Logging is ready!")
}

output:

Fire!
datanow: logrus.Fields{"time":"2014-05-02T05:40:55.393196518Z", "level":"warning", "msg":"Logging is ready!"}
{"level":"warning","msg":"Logging is ready!","time":"2014-05-01 22:40:55.393034888 -0700 PDT"}

Improving logrus performance

I recently migrated some parts of our code from glog to logrus and noticed some serious decrease (close to 45%) in performance. We are not doing anything fancy, just level logging to a file. So I started to profile and noticed that a lot of time is spend on memmove and scanblock. When I accumulate all garbage collection related operations I see that roughly 26% of the time is spend on garbage collection.

One of the hotspots I discovered it the formatting and I spiked a few tweaks to introduce pooled buffers. This will change the API since buffers need to be released after they have been used.

My changes have a huge impact on the code and break almost all hooks. Not in a bad way, it's just because types are slightly different. So, before I spend a lot of time trying to complete this, I want to discuss if the logrus community would like to move into this direction?

The first benchmark's show an performance improvement 12% for small log entries and up to 35% for larger entries with an mean of 28%. My assumption is that is also significantly lowers memory usage and because it generates less garbage, we spend less time in Go's stop the world garbage collector.

My concrete question: is the logrus community willing to accept changes that effect the entire codebase and hook interface to gain performance and generate less garbage? If so, I want to discuss the next steps and willing to invest time in it to make it happen.

Logrus doesn't follow package naming convention

All of the sub-packages in logrus are prefixed with logrus_. The GO convention states that package names should be a single word, and the name of the directory they are in. Thus hooks/papertrail/papertrail.go would be package papertrail.

Yes, this is convention, and not hard requirement, but I just wanted to bring it up since changing things like this becomes more difficult once logrus goes 1.0.

Syslog hook doesn't log anything

Hi!

The syslog hook doesn't log anything unless a field "level" is set by the user.

It can be corrected by using entry.Level.String() in syslog.go. However, in this case, logging is done with ANSI codes (I get something like this: #033[34mINFO#033[0m[0000] proxy: start serving requests).

Many types don't need pointer receiver methods.

The default formatters and hooks all have pointer receivers on their method, but non of them actually need to.

Removing the pointer receivers would not break any clients and it would make the usage of them hooks easier/less awkward.

Failed hooks should use logrus formatter

When a hook fails to fire for some reason, an error message is printed. The errors aren't printed using the formatters, however:

Failed to fire hook: Post http://internal.errors/api/4/store/: dial tcp: lookup internal.errors: no such host
{"egid": 0, "eid": 0, "env": "staging", "host": "stage1nas1", "level": "error", "pid": 30926, "time": "2015-01-19T23:57:32Z", "version": "b02ca56faf126efd94db11646b0bf4e33ea79aba", "subsystem": "heartbeat"}
{"egid": 0, "eid": 0, "env": "staging", "host": "stage1nas1", "level": "info", "pid": 30926, "time": "2015-01-19T23:57:34Z", "version": "b02ca56faf126efd94db11646b0bf4e33ea79aba", "subsystem": "zookeeper"}
{"egid": 0, "eid": 0, "env": "staging", "host": "stage1nas1", "level": "info", "pid": 30926, "time": "2015-01-19T23:57:36Z", "version": "b02ca56faf126efd94db11646b0bf4e33ea79aba", "subsystem": "zookeeper"}
{"egid": 0, "eid": 0, "env": "staging", "host": "stage1nas1", "level": "info", "pid": 30926, "time": "2015-01-19T23:57:38Z", "version": "b02ca56faf126efd94db11646b0bf4e33ea79aba", "subsystem": "zookeeper"}
{"egid": 0, "eid": 0, "env": "staging", "host": "stage1nas1", "level": "info", "pid": 30926, "time": "2015-01-19T23:57:39Z", "version": "b02ca56faf126efd94db11646b0bf4e33ea79aba", "subsystem": "getter"}

The culprits are those lines:

    if err := entry.Logger.Hooks.Fire(level, entry); err != nil {
        entry.Logger.mu.Lock()
        fmt.Fprintf(os.Stderr, "Failed to fire hook: %v\n", err)
        entry.Logger.mu.Unlock()
    }

    reader, err := entry.Reader()
    if err != nil {
        entry.Logger.mu.Lock()
        fmt.Fprintf(os.Stderr, "Failed to obtain reader, %v\n", err)
        entry.Logger.mu.Unlock()
    }

    entry.Logger.mu.Lock()
    defer entry.Logger.mu.Unlock()

    _, err = io.Copy(entry.Logger.Out, reader)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Failed to write to log, %v\n", err)
    }

https://github.com/Sirupsen/logrus/blob/master/entry.go#L78-L97

They should be printing with the formatter, me thinks.

@sirupsen
cc: @macb

"time" should be a time.Time

Somewhat related to #24, it is problematic that the "time" field is a string. This means that formatters have to parse the Go date format back into a real time.Time before they can reformat it. The Go date format has no constant defining it (and the Go maintainers have rejected requests to add one), so the caller has to hard-code the Go date format just to parse it and reformat it to something else. This would work better if "time" were kept a time.Time. Then formatters could just format it. (This would also eliminate the possibility of parsing errors that need to be checked for.)

Add Notice level

It would be nice to have an additional level between Info and Warning Notice. I tend to use logging as follows:

  • Notice: System start/stop, significant-but-normal events
  • Info: Detail: User connected, transaction applied etc.
  • Debug: In depth debugging/tracing, e.g. packet dumps

I usually would have the default log level set to Notice and have Errors, Fatal and Panic logged to a remote server for analysis.

Modify entry in Hook

Good evening,

Here is a sample of what I'd like to do http://play.golang.org/p/Wwoo0eW0x5 (Currently, `e.Data["test"] = "changed" doesn't change anything to the output.)

Basically my question is, should a hook be able to modify the data of the entry? I think that yes, a hook should be like a middleware and should be able to filter things.

Rename hooks to middleware

Hooks imply order, specifically that they're run before the entry is actually logged—however, that's not the case, since for Logrus to be as customizable as possible the hooks should allow modifying the entry before it's logged. The proper name for such modules is a 'middleware', so I propose we rename hooks to middleware everywhere in Logrus.

@aybabtme

Split log file accroding to date ,level or other automatically!

I think this function is very usefull,Do we have a plan to support it.
Then the log files will look like "xxx.log.2015-02-25 xxx.log.wf.2015-02-25"
"xxx.log.2015-02-25" will include "INFO level logs"
"xxx.log.wf.2015-02-25" will include "WARN and FATAL level logs"

Package exported logger

Like the standard log library, and the glog project; having a package global logger is very convenient to avoid having to pass a logger everywhere in your application.

I propose adding the following package exported funcs with a default logger to stdout:

  • WithField
  • WithFields
  • Debug
  • Info
  • Warn
  • Error
  • Panic
  • Fatal

Here I don't list the Print and Warning family of functions because I don't think they are useful given the other levels. Also, the f and ln families, because:

  • Printf like funcs are covered by WithField(s)
  • Println like funcs are redundant with the Print like funcs, since each log entry is a line anyways.

systemd journal support

Systemd's journal support could be added by using the go-systemd lib:
https://github.com/coreos/go-systemd/blob/master/journal/send.go

It is kinda funny however to make it a "Hook" -- you kinda want it to be a log formatter, but the current interface for formatters assumes an io.Copy() interface, while for journal.Send you would want the 'formatter' to handle the 'write'.

A quick hack would be to allow Formatters to return nil, and in that case skip the io.Copy paths?

Println variants have inconsistent spacing

The Go fmt package has different spacing between arguments for Print and Println. This is described in the documentation.

Logrus does not use the same spacing as the go fmt/log packages inhibiting it's ability to be a drop in replacement. Can this be changed or is forking necessary here?

text log should remove trailing space

current code is generating every log line with a trailing space, which is not good and should be eliminated

tf := &TextFormatter{DisableColors: true}
byts, _ := tf.Format(&Entry{Message: "msg content"})
fmt.Printf("%q\n", string(byts))
"time=\"0001-01-01T00:00:00Z\" level=panic msg=\"msg content\" \n"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.