GithubHelp home page GithubHelp logo

knqyf263 / utern Goto Github PK

View Code? Open in Web Editor NEW
305.0 11.0 24.0 437 KB

Multi group and stream log tailing for AWS CloudWatch Logs.

License: MIT License

Go 95.89% Makefile 4.11%
golang go command-line standalone aws amazon cloudwatch-logs tail devops logging

utern's Introduction

utern

GitHub release Build Status Go Report Card MIT License

The “best of best” way to tail AWS CloudWatch Logs from your terminal.

Abstract

Utern allows you to tail log events from multiple log groups and log streams on AWS CloudWatch Logs. Each result is color coded for quicker debugging. Inspired by stern.

The query is a regular expression so the log group name and stream name can easily be filtered and you don't need to specify the exact name. If a stream is deleted it gets removed from tail and if a new stream is added it automatically gets tailed.

When a log group contains multiple log streams, Utern can tail all of them too without having to do this manually for each one. Simply specify the filter to limit what log events to show.

$ utern [options] log-group-query

So Simple!!

The log-group-query is a regular expression so you could provide "web-\w" to tail web-backend and web-frontend log groups but not web-123.

Features

  • Multi log groups tailing in parallel
    • Regular expression
  • Multi log streams tailing in parallel
    • Regular expression
  • Colorful
    • Quicker debugging
  • Flexible date and time parser
    • Human friendly formats, i.e. 1h20m to indicate 1 hour and 20 minutes ago
    • A full timestamp 2019-01-02T03:04:05Z (RFC3339)
  • Powerful built-in filter
  • Fast
    • Written in golang
  • Easy installation
    • Pre-built binaries

Installation

From source

$ git clone https://github.com/knqyf263/utern.git
$ cd utern
$ go install

Binary (Including Windows)

Go to the releases page, find the version you want, and download the zip file. Unpack the zip file, and put the binary to somewhere you want (on UNIX-y systems, /usr/local/bin or the like). Make sure it has execution bits turned on.

Mac OS X / Homebrew

You can use homebrew on OS X.

$ brew tap knqyf263/utern
$ brew install knqyf263/utern/utern

If you receive an error (Error: knqyf263/utern/utern 64 already installed) during brew upgrade, try the following command

$ brew unlink utern && brew uninstall utern
($ rm -rf /usr/local/Cellar/utern/64)
$ brew install knqyf263/utern/utern

RedHat, CentOS

Download rpm package from the releases page

$ sudo rpm -ivh https://github.com/knqyf263/utern/releases/download/v0.0.1/utern_0.0.1_Tux_64-bit.rpm

Debian, Ubuntu

Download deb package from the releases page

$ wget https://github.com/knqyf263/utern/releases/download/v0.0.1/utern_0.0.1_Tux_64-bit.deb
$ sudo dpkg -i utern_0.0.1_Tux_64-bit.deb

Examples

Some examples are shown below.

List all log groups

$ aws logs describe-log-groups --query "logGroups[].[logGroupName]" --output text

List all log streams

$ aws logs describe-log-streams --log-group-name log-group-name --query "logStreams[].[logStreamName]" --output text

All log streams

$ utern log-group-query

Filter log groups with regular expressions

$ utern "web-\w"

Filter log streams with regular expressions (--stream, -n)

$ utern --stream log-stream-query log-group-query

Filter log streams with a prefix of log stream name (--stream-prefix, -p)

If the log group has many log streams, --stream-prefix will be faster than --stream.

$ utern --stream-prefix log-stream-prefix log-group-query

Filter log streams with a prefix and regular expressions

$ utern -p log-stream-prefix -n log-stream-query log-group-query

Logs after 1 hour ago (--since, -s)

$ utern --since 1h log-group-query

Logs after 2019-01-02 03:04:05 UTC

RFC3339

$ utern --since 2019-01-02T03:04:05Z log-group-query

Logs from 10 minutes ago to 5 minutes ago

$ utern --since 10m --end 5m log-group-query

Usage

NAME:
   utern - Multi group and stream log tailing for AWS CloudWatch Logs

USAGE:
   utern [global options] command [command options] [arguments...]

VERSION:
   0.1.1

COMMANDS:
   help, h  Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --stream value, -n value         Log stream name (regular expression).
                                    Displays all if omitted. If the option
                                    "since" is set to recent time, this option
                                    usually makes it faster than the option
                                    "stream-prefix"
   --stream-prefix value, -p value  Log stream name prefix. If a log group
                                    contains many log streams, this option makes
                                    it faster.
   --since value, -s value          Return logs newer than a relative duration
                                    like 52, 2m, or 3h. (default: "5m")
   --end value, -e value            Return logs older than a relative duration
                                    like 0, 2m, or 3h.
   --profile value                  Specify an AWS profile.
   --code value                     Specify MFA token code directly
                                    (if applicable), instead of using stdin.
   --region value, -r value         Specify an AWS region.
   --filter value                   The filter pattern to use. For more
                                    information, see https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html.
   --timestamps                     Print timestamps
   --event-id                       Print event ID
   --no-log-group                   Suppress display of log group name
   --no-log-stream                  Suppress display of log stream name
   --max-length value               Maximum log message length (default: 0)
   --color                          Force color output even if not a tty
   --help, -h                       show help
   --version, -v                    print the version

Contribute

  1. fork a repository: github.com/knqyf263/utern to github.com/you/repo
  2. get original code: go get github.com/knqyf263/utern
  3. work on original code
  4. add remote to your repo: git remote add myfork https://github.com/you/repo.git
  5. push your changes: git push myfork
  6. create a new Pull Request

License

MIT

Author

Teppei Fukuda

utern's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

utern's Issues

Not printing log messages anymore

Hi, thanks for your app 👍
I found it about 6 months ago, and it worked like a charm.

I tried using it again but it doesn't work as expected. The version hasn't changed since then so something may have changed in Cloudwatch API.

  • executing without options
$ utern /log/group
+ /log/group
# nothing beyond that
  • getting logs from before 1 hour

It's printing the logs until the present moment but no real-time updates (watching)

$ utern --since 1h /log/group
+ /log/group
+ /log/group › log/stream/id (2023-01-14T16:46:07+00:00)
# some logs as expected
# ... producing new logs
# no more input

install failure

❯ go get -v -u github.com/knqyf263/utern
github.com/knqyf263/utern (download)
github.com/aws/aws-sdk-go (download)
github.com/jmespath/go-jmespath (download)
github.com/briandowns/spinner (download)
github.com/fatih/color (download)
github.com/pkg/errors (download)
github.com/urfave/cli (download)
github.com/cpuguy83/go-md2man (download)
github.com/cpuguy83/go-md2man/vendor/github.com/shurcooL/sanitized_anchor_name
github.com/knqyf263/utern/cache
github.com/pkg/errors
github.com/knqyf263/utern/color
github.com/briandowns/spinner/vendor/golang.org/x/sys/unix
github.com/cpuguy83/go-md2man/vendor/github.com/russross/blackfriday/v2
github.com/cpuguy83/go-md2man/md2man
github.com/urfave/cli
github.com/briandowns/spinner/vendor/github.com/mattn/go-isatty
github.com/briandowns/spinner/vendor/github.com/mattn/go-colorable
github.com/briandowns/spinner/vendor/github.com/fatih/color
github.com/briandowns/spinner
github.com/knqyf263/utern/config
github.com/knqyf263/utern/cloudwatch
github.com/knqyf263/utern
# github.com/knqyf263/utern
../../go/src/github.com/knqyf263/utern/main.go:29:17: cannot use cli.StringFlag literal (type cli.StringFlag) as type cli.Flag in slice literal:
        cli.StringFlag does not implement cli.Flag (Apply method has pointer receiver)
../../go/src/github.com/knqyf263/utern/main.go:34:17: cannot use cli.StringFlag literal (type cli.StringFlag) as type cli.Flag in slice literal:
        cli.StringFlag does not implement cli.Flag (Apply method has pointer receiver)
../../go/src/github.com/knqyf263/utern/main.go:39:17: cannot use cli.StringFlag literal (type cli.StringFlag) as type cli.Flag in slice literal:
        cli.StringFlag does not implement cli.Flag (Apply method has pointer receiver)
../../go/src/github.com/knqyf263/utern/main.go:44:17: cannot use cli.StringFlag literal (type cli.StringFlag) as type cli.Flag in slice literal:
        cli.StringFlag does not implement cli.Flag (Apply method has pointer receiver)
../../go/src/github.com/knqyf263/utern/main.go:49:17: cannot use cli.StringFlag literal (type cli.StringFlag) as type cli.Flag in slice literal:
        cli.StringFlag does not implement cli.Flag (Apply method has pointer receiver)
../../go/src/github.com/knqyf263/utern/main.go:54:17: cannot use cli.StringFlag literal (type cli.StringFlag) as type cli.Flag in slice literal:
        cli.StringFlag does not implement cli.Flag (Apply method has pointer receiver)
../../go/src/github.com/knqyf263/utern/main.go:59:17: cannot use cli.StringFlag literal (type cli.StringFlag) as type cli.Flag in slice literal:
        cli.StringFlag does not implement cli.Flag (Apply method has pointer receiver)
../../go/src/github.com/knqyf263/utern/main.go:64:17: cannot use cli.StringFlag literal (type cli.StringFlag) as type cli.Flag in slice literal:
        cli.StringFlag does not implement cli.Flag (Apply method has pointer receiver)
../../go/src/github.com/knqyf263/utern/main.go:69:15: cannot use cli.BoolFlag literal (type cli.BoolFlag) as type cli.Flag in slice literal:
        cli.BoolFlag does not implement cli.Flag (Apply method has pointer receiver)
../../go/src/github.com/knqyf263/utern/main.go:73:15: cannot use cli.BoolFlag literal (type cli.BoolFlag) as type cli.Flag in slice literal:
        cli.BoolFlag does not implement cli.Flag (Apply method has pointer receiver)
../../go/src/github.com/knqyf263/utern/main.go:73:15: too many errors


❯ go version
go version go1.15.5 linux/amd64

panic: runtime error when tailing logs

When running command as follows:

utern --timestamps --no-log-stream --since 1m "/aws/lambda/testapp-demo-dev-*"

I am ocassionally getting this:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x14f82d6]

goroutine 83 [running]:
github.com/knqyf263/utern/cloudwatch.(*Client).ListStreams.func1(0xc0002bcc00, 0xc0004fae01, 0xc0005d9d30)
	/home/travis/gopath/src/github.com/knqyf263/utern/cloudwatch/stream.go:32 +0xf6
github.com/aws/aws-sdk-go/service/cloudwatchlogs.(*CloudWatchLogs).DescribeLogStreamsPagesWithContext(0xc00000e040, 0x17b4da0, 0xc0000b3b00, 0xc0005af4a0, 0xc0005d9d10, 0x0, 0x0, 0x0, 0xc0005d9eb8, 0xc000080060)
	/home/travis/gopath/pkg/mod/github.com/aws/[email protected]/service/cloudwatchlogs/api.go:1666 +0x163
github.com/knqyf263/utern/cloudwatch.(*Client).ListStreams(0xc00005e260, 0x17b4da0, 0xc0000b3b00, 0xc0000c40c0, 0x25, 0x1709d3001b0, 0x1b48838, 0x0, 0x0, 0x0, ...)
	/home/travis/gopath/src/github.com/knqyf263/utern/cloudwatch/stream.go:66 +0x1d2
github.com/knqyf263/utern/cloudwatch.(*Client).tail(0xc00005e260, 0x17b4da0, 0xc0000b3b00, 0xc0000c40c0, 0x25, 0xc00008a0c0, 0xc00001e540, 0xc00008a120, 0x0, 0x0)
	/home/travis/gopath/src/github.com/knqyf263/utern/cloudwatch/cloudwatch.go:175 +0x3b5
github.com/knqyf263/utern/cloudwatch.(*Client).Tail.func3(0xc000304620, 0xc00005e260, 0x17b4da0, 0xc0000b3b00, 0xc00008a0c0, 0xc00001e540, 0xc00008a120, 0xc0000c40c0, 0x25)
	/home/travis/gopath/src/github.com/knqyf263/utern/cloudwatch/cloudwatch.go:116 +0xd5
created by github.com/knqyf263/utern/cloudwatch.(*Client).Tail
	/home/travis/gopath/src/github.com/knqyf263/utern/cloudwatch/cloudwatch.go:114 +0x29b

Crash when running utern "Stage"

The app crashes when running utern "Stage"

Here is the full output:

+ /aws/lambda/BasicRatesStage
+ /aws/lambda/CollectionCollectionAuditEventsStage
+ /aws/lambda/CollectionCollectionsStage
+ /aws/lambda/CollectionDeliveriesStage
+ /aws/lambda/CollectionDeliveryAuditEventsStage
+ /aws/lambda/CollectionNewCollectionsStage
+ /aws/lambda/CollectionNewDeliveriesStage
+ /aws/lambda/DriverNotifierProxyStage
+ /aws/lambda/GeocodeAddressStage
+ /aws/lambda/GetAddressStage
+ /aws/lambda/NotificationCheckSendShipmentStage
+ /aws/lambda/NotificationCheckSendShipmentStage › 2020/02/11/[$LATEST]a9150c83e4414a1f8bbd95b94d3c070e (2020-02-11T14:52:49+02:00)
+ /aws/lambda/NotificationSendEmailStage
+ /aws/lambda/NotificationSendOTPStage
⠹ Fetching log streams... panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x14f82d6]

goroutine 1 [running]:
github.com/knqyf263/utern/cloudwatch.(*Client).ListStreams.func1(0xc000253b80, 0xc0004d0401, 0x10000c0002acd20)
	/home/travis/gopath/src/github.com/knqyf263/utern/cloudwatch/stream.go:32 +0xf6
github.com/aws/aws-sdk-go/service/cloudwatchlogs.(*CloudWatchLogs).DescribeLogStreamsPagesWithContext(0xc0000a00d8, 0x17b4da0, 0xc000079b40, 0xc0001c3170, 0xc0002c54a8, 0x0, 0x0, 0x0, 0xc0002c54c8, 0x14853f4)
	/home/travis/gopath/pkg/mod/github.com/aws/[email protected]/service/cloudwatchlogs/api.go:1666 +0x163
github.com/knqyf263/utern/cloudwatch.(*Client).ListStreams(0xc0001ee640, 0x17b4da0, 0xc000079b40, 0xc000366e70, 0x24, 0x170344b6810, 0x1b48838, 0x0, 0x0, 0x0, ...)
	/home/travis/gopath/src/github.com/knqyf263/utern/cloudwatch/stream.go:66 +0x1d2
github.com/knqyf263/utern/cloudwatch.(*Client).Tail(0xc0001ee640, 0x17b4da0, 0xc000079b40, 0xc000079b40, 0xc0001ee580)
	/home/travis/gopath/src/github.com/knqyf263/utern/cloudwatch/cloudwatch.go:100 +0x782
main.run(0xc0001e2e70, 0xc0001e2e70, 0x0)
	/home/travis/gopath/src/github.com/knqyf263/utern/main.go:121 +0x8f
main.main.func1(0xc0000c86e0, 0x0, 0x0)
	/home/travis/gopath/src/github.com/knqyf263/utern/main.go:90 +0x8d
github.com/urfave/cli.HandleAction(0x1557380, 0x1631ed8, 0xc0000c86e0, 0x0, 0x0)
	/home/travis/gopath/pkg/mod/github.com/urfave/[email protected]/app.go:523 +0xbe
github.com/urfave/cli.(*App).Run(0xc0001ec000, 0xc00009c020, 0x2, 0x2, 0x0, 0x0)
	/home/travis/gopath/pkg/mod/github.com/urfave/[email protected]/app.go:285 +0x5df
main.main()
	/home/travis/gopath/src/github.com/knqyf263/utern/main.go:93 +0x907

Is it possible to rather specify each group instead of the app finding it with a regular expression?

add --profile option

Like awslogs option,

awslogs --profile myprofile

I want to specify which profile to use instead of default.

add --region option

Like awslogs get --aws-region us-east-1,
I want to specify which region to fetch AWS CloudWatch Logs.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.