humanlogio / humanlog Goto Github PK

Hello!,

Where could I PR to either document that --skip-unchanged defaults to true and duplicate key/values are removed on a naked run, or change it so that it is not default functionality?

Thanks!

Hello 👋

if I create a config.json file like this

{
    "skip": ["foo", "bar"]
}

humanlog (v0.7.5) crashes with

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x8 pc=0x102bd0e54]

goroutine 1 [running]:
main.newApp.func6(0x140000db1e0)
	/home/runner/work/humanlog/humanlog/cmd/humanlog/main.go:301 +0x684
github.com/urfave/cli.HandleAction({0x102cd7e00?, 0x14000081680?}, 0x140000eb180?)
	/home/runner/work/humanlog/humanlog/vendor/github.com/urfave/cli/app.go:524 +0x58
github.com/urfave/cli.(*App).Run(0x140000eb180, {0x140000961f0, 0x1, 0x1})
	/home/runner/work/humanlog/humanlog/vendor/github.com/urfave/cli/app.go:286 +0x570
main.main()
	/home/runner/work/humanlog/humanlog/cmd/humanlog/main.go:65 +0xd8

I believe this happens because both fields are defined as string slice pointers *[]string, and when only one is specified the other is nil and thus causes a nil pointer dereference here:

if len(*cfg.Skip) > 0 && len(*cfg.Keep) > 0 {

(It does not happen when no configuration file is used because the fields get defaulted to non-nil empty string slice pointers in that case.)

A workaround is to define the other field as an empty JSON array in the configuration file. Ideally though, that shouldn't be required.

One fix could be to extend the condition by a nilness check before de-referencing. However, I do wonder if those fields could be regular string slices instead -- a quick scan of mine didn't yield that the code cares about distinguishing between absent and empty values, but I could have missed something. If not though, then my personal suggestion would be to de-pointerize the fields in order to simplify things.

Let me know what you think. Either way, I'd be happy to submit a quick PR.

Would be awesome to have this in the Arch User Repository as well :)

https://wiki.archlinux.org/title/Arch_User_Repository

Despite calling humanlog with --light-bg, additional values get colored with a very light color that is almost unreadable on a white background.

Apparently because of:

ValColor:          color.New(color.FgHiWhite),

Should different colors be used depending on --light-bg?

The default timestamp fields parsing seems to work well:

$ echo '{"ts":"2023-03-17T15:31:58.242777327Z","message":"msg"}' | humanlog -m 'message' --time-format '2006-01-02T15:04:05.000000000Z'
humanlog> reading stdin...
2023-03-17T15:31:58.242777327Z || msg

However, if I try and use the custom --time-fields option, that doesn't seem to be working?

$ echo '{"xts":"2023-03-17T15:31:58.242777327Z","message":"msg"}' | humanlog --time-fields 'xts' -m 'message' --time-format '2006-01-02T15:04:05.000000000Z'
humanlog> reading stdin...
0001-01-01T00:00:00.000000000Z || msg xts="2023-03-17T15:...

$ humanlog -v
humanlog version 0.7.5+9612436

As title says, I suggest you guys start using travis-ci with GoReleaser, it lets you cross build binaries to your releases.

Hi,

I have a long json payload in logfmt value, and I run into token too long.
Could you please increase the buffer, make it configurable, and perhaps at least catch the particular error and instead of crashing, mark the field as error and continue?

I found some background on the error: https://stackoverflow.com/a/37455465

Hi there!

Just wondering, is the github.com/kr/logfmt dependency still needed in the humanlog codebase?
A quick search shows it's only being used in handler.go.

I'm packaging humanlog for Debian (jesseduffield/lazygit-debian#31), and removing this package from go.mod would make it so one less dependency needs to be packaged.

Thanks!

I followed given instructions but not working on my mac.

Great library.
Minor edit to README:
go install github.com/aybabtme/humanlog/cmd/humanlog@latest

When I throw some log at humanlog, it fails with a runtime panic in the parser.

I can't share the log and before I try to narrow it down further let me first ask whether it would make more sense to replace the builtin parser with https://github.com/go-logfmt/logfmt?

Also don't log update check errors when there's no internet. Stuff like that.

From the main README.md I got the impression that humanlog can handle arbitrary logfmt data ("If you emit logs in JSON or in logfmt, you will enjoy pretty logs..."), and then I wondered why it left some simple logs completely unchanged, like:

msg=hello

After diving into the source I found that a log entry must have a level, time, and msg field (https://github.com/aybabtme/humanlog/blob/master/logrus_handler.go#L39). It would be useful to mention this.

About the implementation:

• "lvl" and "message" seem to be treated as aliases, but the sanity check doesn't include them, so I'm not sure whether those aliases really work?
• the check expects "msg" to be quoted. This is too strict, for example the go-logfmt/logfmt Encoder doesn't put quotation marks around values unless absolutely necessary.

Warning: Calling bottle :unneeded is deprecated! There is no replacement.
Please report this issue to the aybabtme/tap tap (not Homebrew/brew or Homebrew/core):
/usr/local/Homebrew/Library/Taps/aybabtme/homebrew-tap/humanlog.rb:9

I'm working to make better use of structured logging, and came across your humanlog utility.
Operating System: OSX
Terminal Client: Bash/Default

I have a small test log I created using gokit.
{"caller":"main.go:25","level":"debug","msg":"Testing Debug","service":"logtestapp","ts":"2016-07-21T19:50:18Z"} {"caller":"main.go:26","level":"info","msg":"Testing Info","service":"logtestapp","ts":"2016-07-21T19:50:18Z"} {"caller":"main.go:27","level":"warn","msg":"Testing Warn","service":"logtestapp","ts":"2016-07-21T19:50:18Z"} {"caller":"main.go:28","level":"error","msg":"Testing Error","service":"logtestapp","ts":"2016-07-21T19:50:18Z"} {"caller":"main.go:29","level":"crit","msg":"Testing Crit","service":"logtestapp","ts":"2016-07-21T19:50:18Z"}

I installed humanlog using the go get method from your README.
go get -u github.com/aybabtme/humanlog/...

Once installed, I run the following command, but am not seeing any change to display of the logs I'm providing, save for the humanlog> reading stdin... at the top of the ouput.
humanlog < logtestapp.log

Is there something I'm missing, or a potential misconfiguration I've encountered? Anything that can point me in the right direction would be greatly appreciated, thanks!

It took me some time to figure out how the boolean flags work. The flag --truncate is set to true by default, but the CLI help message only includes info about what this flag does and lacks information on how to set it to false.

I went into the humanlog source code and opened up the docs for the latest version of the CLI library it uses (urfave/cli). This led me to try the flag with the no- prefix. This didn't work because humanlog isn't using the latest version of the library.

I also tried --truncate false which didn't work.

I finally found an issue which describes flags being used as --flag=true which led me to try --truncated=false.

Ideally, I wouldn't need to search through the issues of a dependent library to figure out how to use humanlog.

Thank you very much for all the work you've put into this tool.

I would like to be able to add a custom columns from specific fields, one use case example:

The logs I am using (from python-json-logger" have a field called "name" that shows what module generated that log message
The way it looks now:

The default colors don't work for me:

Due to the way that the underlying rgbterm works, I can't just send a quick \033[40m to set the background color to dark; it only works for the first colored block printed, and then a SGR reset kills that.

It's possible to set the default background color, but it's very terminal dependent, and so my wrapper function for Scanner is getting really ugly:

func ScannerDarkly(src io.Reader, dst io.Writer, opts *humanlog.HandlerOptions) {
    var setBgColorCode string
    if strings.Contains(os.Getenv("TERM_PROGRAM"), "iTerm") {
        // iTerm...
        setBgColorCode = "\033]Ph%.2x%.2x%.2x\033\\"
    } else if strings.Contains(os.Getenv("TERM"), "256") || strings.Contains(os.Getenv("TERM"), "xterm") {
        setBgColorCode = "\033]6;rgb:%.2x/%.2x/%.2x\a"
    }
    if setBgColorCode != "" {
        dst.Write([]byte(fmt.Sprintf(setBgColorCode, bgColor[0], bgColor[1], bgColor[2])))
    }
    humanlog.Scanner(src, dst, opts)
}

I'm not cleaning up the color change, either, so when my program quits it leaves the background dark. Do you think it would be possible to make the code accept some kind of color palette configuration or something (perhaps with regular ANSI code fallbacks), instead of all of the hardcoded color codes?

Installing in a Linux VM on ARM MacBook. uname -m returns "aarch64" which then returns an error from the api call to https://api.humanlog.io/api/releases/humanlog in the installation script.

Example of manually reproducing:

$ curl -s --data "{\"os\":\"${os}\",\"arch\":\"${arch}\"}" https://api.humanlog.io/api/releases/humanlog
oups, something went wrong serving your request

Hi there,
Just a suggestion, but wouldn't it be better if the github.com/humanlogio/api/go dependency was internalised (by that, I mean put it in this repo instead of its own repo)
Thanks!

Hi,

This is eseentially the same issue as #53. My JSON payload is larger than 1MB thus crashing the program. I want to suggest using scanner.Err() == bufio.ErrTooLong to handle such cases.

tested; gitpod, codespaces, apparently WSL too.

gitpod /workspace/boilerplate (main) $ curl -L "https://humanlog.io/install.sh" | sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1296  100  1296    0     0   2661      0 --:--:-- --:--:-- --:--:--  2661
######################################################################## 100.0%
humanlog was installed successfully to /home/gitpod/.humanlog/bin/humanlog
sh: 36: SHELL: parameter not set

A workaround is:

gitpod /workspace/boilerplate (main) $ export SHELL=/bin/bash && curl -L "https://humanlog.io/install.sh" | sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1296  100  1296    0     0   2230      0 --:--:-- --:--:-- --:--:--  2230
######################################################################## 100.0%
humanlog was installed successfully to /home/gitpod/.humanlog/bin/humanlog
Manually add the directory to your $HOME/.bash_profile (or similar)
  export HUMANLOG_INSTALL="/home/gitpod/.humanlog"
  export PATH="$HUMANLOG_INSTALL/bin:$PATH"
Run '/home/gitpod/.humanlog/bin/humanlog --help' to get started

Interestingly enough, it looks like SHELL is set even on a clean shell where the humanlog install fails:

gitpod /workspace/boilerplate (main) $ echo $SHELL
/bin/bash

probably doesn't get inherited then, as this works:

export SHELL=$SHELL && curl -L "https://humanlog.io/install.sh" | sh

Whenever I run something like apt update or apt upgrade, there's a warning in the dpkg output:

dpkg-query: warning: parsing file '/var/lib/dpkg/status' near line 10716 package 'humanlog':
 missing 'Maintainer' field

And inspecting the generated .deb package:

$ curl -LkO https://github.com/aybabtme/humanlog/releases/download/0.5.0/humanlog_0.5.0_linux_amd64.deb
$ ar x humanlog_0.5.0_linux_amd64.deb
$ cat control
Package: humanlog
Version: 0.5.0
Section: 
Priority: 
Architecture: amd64
Installed-Size: 3076
Description: no description given

It seems that the Maintainer field is indeed not set.

It looks like goreleaser can set this when it generates the package: https://goreleaser.com/customization/nfpm/

Consider the following line:

time="2019-12-09T13:56:00Z" level=trace msg="crash" somekey=

This makes humanlog crash horribly:

[root@la1 ~]# ./humanlog < repro.log
humanlog> reading stdin...
panic: runtime error: index out of range

goroutine 1 [running]:
github.com/aybabtme/humanlog/parser/logfmt.scanAllKeyValue(0xc42009e000, 0x3c, 0x1000, 0x101, 0xc420045ab0)
        /Users/antoine/gocode/src/github.com/aybabtme/humanlog/parser/logfmt/logfmt_parser.go:38 +0x31e
github.com/aybabtme/humanlog/parser/logfmt.Parse(0xc42009e000, 0x3c, 0x1000, 0x101, 0xc420045ab0, 0x20)
        /Users/antoine/gocode/src/github.com/aybabtme/humanlog/parser/logfmt/logfmt_parser.go:18 +0x5b
github.com/aybabtme/humanlog.Scanner(0x5c64e0, 0xc42000c018, 0x5c6500, 0xc42000c020, 0x690ca0, 0x2, 0x2)
        /Users/antoine/gocode/src/github.com/aybabtme/humanlog/scanner.go:43 +0x405
main.newApp.func1(0xc420082160, 0xc420082160, 0xc420045ce7)
        /Users/antoine/gocode/src/github.com/aybabtme/humanlog/cmd/humanlog/main.go:124 +0x369
github.com/aybabtme/humanlog/vendor/github.com/urfave/cli.HandleAction(0x56f620, 0xc420098000, 0xc420082160, 0xc4200522a0, 0x0)
        /Users/antoine/gocode/src/github.com/aybabtme/humanlog/vendor/github.com/urfave/cli/app.go:501 +0xc8
github.com/aybabtme/humanlog/vendor/github.com/urfave/cli.(*App).Run(0xc420094000, 0xc42000e1b0, 0x1, 0x1, 0x0, 0x0)
        /Users/antoine/gocode/src/github.com/aybabtme/humanlog/vendor/github.com/urfave/cli/app.go:268 +0x60c
main.main()
        /Users/antoine/gocode/src/github.com/aybabtme/humanlog/cmd/humanlog/main.go:31 +0x11c

This is evidently caused by the somekey= having a zero length value

Hi @aybabtme , I was wondering if you could expose the Scan function and its dependencies(StdioSink and Config), which are currently internal. So that it can be used outside the module as a library. Lazygit currently uses the Scanner function from an older version of Humanlog. But as you know the in the latest version Scanner/Scan function is not usable outside Humanlog, since it has internal dependencies.

I was trying to package Humanlog for Debian as part of Lazygit packaging. Then noticed that the version differences. But in order to upgrade the Humanlog dependency in Lazygit we need to expose StdioSink and config. I'll be happy to send a PR, but would like to hear your suggestions and opinions first. Thanks! 😁

connect-go have changed their import path from github.com/bufbuild/connect-go to connectrpc.com/connect, and therefore needs to be updated in go.mod and the source files.

$ echo '{"level":"info", "time" : 1533094882, "msg":"logger construction succeeded"}' | humanlog

I get:

humanlog> reading stdin...
{"level":"info", "time" : 1533094882, "msg":"logger construction succeeded"}

And nothing else is happening. I'm hot sure i've built the project correctly, cd

$ cmd/humanlog && go build got me a seemingly working binary

Hi, running humanlog within a github codespace, results in the following error;

humanlog> reading default config file: replacing statefile at "/home/vscode/.state/humanlog/state.json" with "/tmp/humanlog_statefile3796472008": rename /tmp/humanlog_statefile3796472008 /home/vscode/.state/humanlog/state.json: invalid cross-device link

https://github.com/madzak/python-json-logger and other python json loggers tend to use asctime field.

This reflects the fieldname in python stdlib logging framework https://docs.python.org/3/library/logging.html

Would be great if asctime was automatically supported.

Calling c.IsSet(flag.Name) on a flag with both a long and short version doesn't work. I think I have a fix. PR incoming...