go-logfmt / logfmt Goto Github PK
View Code? Open in Web Editor NEWPackage logfmt marshals and unmarshals logfmt messages.
License: MIT License
Package logfmt marshals and unmarshals logfmt messages.
License: MIT License
I was wondering why the string "null"
gets quoted. I would have expected it to be rendered as k=null
.
Line 25 in 2fe45f2
Error: bufio.Scanner: token too long
occurs when scanning large data.
Fix is to set token size to maximum buffer size:
#10
I'm using go-kit/log
and need to redact log entries, i.e. remove usernames, passwords etc. For this purpose I'd like to pass the keyVals through a redact function. Since loggers are stacked, the removal must happen at the time log data is produced, not when it's written (i.e. can't do what in the io.Writer
). Since keyVals can be any interface, it would be great to reuse logfmts encoding to []byte/string
before passing the value to redaction.
Have the logger produce a record even if the key does not follow rules so at least the value is logged lost. Some possible solutions:
It handles a panic arising from attempting to print a null string, but if the stringer panics for some other reason it will re-throw the panic. For my purposes it would be better to note the panic in the log message and carry on.
I guess tastes vary, but it surprised me that there is a panic handler but only for one specific case.
At the moment this package does not support logging values of type array, chan, func, map, slice, or struct. Of these, I cannot think of a reason to support chan or func. But I think we need a good story for array, map, slice, and struct.
Brandur Leach's logfmt blog post does not explicitly explain how to represent compound types in logfmt, nor does the defacto logfmt definition at github.com/kr/logfmt. There are some ideas in an old Gist by Kieth Rarick. The Gist shows this JSON data:
{"dyno":"web", "metadata":{"stack":["line 1", "line 2"], "msg":"null"}}
converted to logfmt as follows.
dyno=web metadata={stack=["line 1" "line 2"] msg="null"}
But the parser at github.com/kr/logfmt does not handle this syntax.
Logrus relies on Go's %v
format in the fmt
package and writes:
dyno=web metadata=map[stack:[line 1 line 2] msg:null]
log15 uses Go's %+v
format and writes:
dyno=web metadata="map[msg:null stack:[line 1 line 2]]"
Neither of these logging packages produce output that can be easily parsed back into the original data structure.
I have not investigated what the non-Go logfmt libraries do.
Given that there is not a clear best practice for encoding compound data structures in logfmt, this package must either refuse to handle them (as it does now) or define a robust encoding for these data types.
@kr, @bmizerany, @inconshreveable, @sirupsen: I would appreciate it if any of you would like to comment on this issue.
either I'm misunderstanding the use of logfmt.MarshalKeyvals, or it doesn't work as expected. I'm doing the following:
keyvals := []string{"app", "myapp", "env", "prod", "error", "errormsg"}
fmt.Println("keyvals:", keyvals)
out, err := logfmt.MarshalKeyvals(keyvals)
fmt.Println(string(out), err)
but instead of getting formatted output, I get:
keyvals: [app myapp env prod error errormsg]
<nil>
so no output, and no error.
this is go version 1.14.5.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.