isabella232 / honeytail

Honeycomb's open-source agent. Contains various parsers for extracting structured data out of common log files.

honeytail

honeytail is Honeycomb's agent for ingesting log file data into Honeycomb and making it available for exploration. Its favorite format is JSON, but understands how to parse a range of other well-known log formats.

See our documentation to read about how to configure and run honeytail, to find tips and best practices, and to download prebuilt versions.

Supported Parsers

honeytail supports reading files from STDIN as well as from a file on disk.

Our complete list of parsers can be found in the parsers/ directory, but as of this writing, honeytail will support parsing logs generated by:

• ArangoDB
• MongoDB
• MySQL
• PostgreSQL (Note: does not support quoted table or column names in queries.)
• nginx
• regex
• keyval(logfmt)
• csv
• syslog

Installation

Install from source:

go get github.com/honeycombio/honeytail

to install to a specific path:

GOPATH=/usr/local go get github.com/honeycombio/honeytail

the binary will install to /usr/local/bin/honeytail

Use a prebuilt binary: find the latest version on Honeycomb.io

Usage

honeytail --writekey=YOUR_WRITE_KEY --dataset='Best Data Ever' --parser=json --file=/var/log/api_server.log

For more advanced usage, options, and the ability to scrub or drop specific fields, see our documentation.

Related Work

In some cases, we've extracted out some generic work for a particular log format

• mongodbtools contains logic specific to parsing various versions of MongoDB logs, and a script for capturing high-level statistics on the database server itself
• mysqltools contains logic specific to normalizing MySQL queries

Contributions

Features, bug fixes and other changes to honeytail are gladly accepted. Please open issues or a pull request with your change. Remember to add your name to the CONTRIBUTORS file!

All contributions will be released under the Apache License 2.0.