tstack / lnav

Log file navigator

This is the source repository for lnav, visit https://lnav.org for a high level overview.

LNAV -- The Logfile Navigator

The Logfile Navigator is a log file viewer for the terminal. Given a set of files/directories, lnav will:

• decompress as needed;
• detect their format;
• merge the files together by time into a single view;
• tail the files, follow renames, find new files in directories;
• build an index of errors and warnings;
• pretty-print JSON-lines.

Then, in the lnav TUI, you can:

• jump quickly to the previous/next error (press e/E);
• search using regular expressions (press /);
• highlight text with a regular expression (:highlight command);
• filter messages using regular expressions or SQLite expressions;
• pretty-print structured text (press P);
• view a histogram of messages over time (press i);
• query messages using SQLite (press ;)

Screenshot

The following screenshot shows a mix of syslog and web access log files. Failed requests are shown in red. Identifiers, like IP address and PIDs are semantically highlighted.

Why not just use tail/grep/less?

The standard Unix utilities are great for processing raw text lines, however, they do not understand log messages. Tail can watch multiple files at a time, but it won't display messages in order by time and you can't scroll backwards. Grep will only find matching lines, but won't return a full multi-line log message. Less can only display a single file at a time. Also, none of these basic tools handle compressed files.

Try online before installing

You can SSH into a demo node to play with lnav before installing.

The "playground" account starts lnav with a couple of log files as an example:

$ ssh [email protected]

The "tutorial 1" account is an interactive tutorial that can teach you the basics of operation:

$ ssh [email protected]

Installation

Download a statically-linked binary for Linux/MacOS from the release page

Brew on MacOS

$ brew install lnav

Usage

Simply point lnav at the files or directories you want to monitor, it will figure out the rest:

$ lnav /path/to/file1 /path/to/dir ...

The lnav TUI will pop up right away and begin indexing the files. Progress is displayed in the "Files" panel at the bottom. Once the indexing has finished, the LOG view will display the log messages that were recognized¹. You can then use the usual hotkeys to move around the view (arrow keys or j/k/h/l to move down/up/left/right).

See the Usage section of the online documentation for more information.

Usage with systemd-journald

On systems running systemd-journald, you can use lnav as the pager:

$ journalctl | lnav

or in follow mode:

$ journalctl -f | lnav

Since journalctl's default output format omits the year, if you are viewing logs which span multiple years you will need to change the output format to include the year, otherwise lnav gets confused:

$ journalctl -o short-iso | lnav

It is also possible to use journalctl's json output format and lnav will make use of additional fields such as PRIORITY and _SYSTEMD_UNIT:

$ journalctl -o json | lnav

In case some MESSAGE fields contain special characters such as ANSI color codes which are considered as unprintable by journalctl, specifying journalctl's -a option might be preferable in order to output those messages still in a non-binary representation:

$ journalctl -a -o json | lnav

If using systemd v236 or newer, the output fields can be limited to the ones actually recognized by lnav for increased efficiency:

$ journalctl -o json --output-fields=MESSAGE,PRIORITY,_PID,SYSLOG_IDENTIFIER,_SYSTEMD_UNIT | lnav

If your system has been running for a long time, for increased efficiency you may want to limit the number of log lines fed into lnav, e.g. via journalctl's -n or --since=... options.

In case of a persistent journal, you may want to limit the number of log lines fed into lnav via journalctl's -b option.

Support

Please file issues on this repository or use the discussions section. The following alternatives are also available:

• [email protected]
• Discord
• Google Groups

Links

• Main Site
• Documentation on Read the Docs
• Internal Architecture

Contributing

• Become a Sponsor on GitHub

Building From Source

Prerequisites

The following software packages are required to build lnav:

• gcc/clang - A C++14-compatible compiler.
• libpcre2 - The Perl Compatible Regular Expression v2 (PCRE2) library.
• sqlite - The SQLite database engine. Version 3.9.0 or higher is required.
• ncurses - The ncurses text UI library.
• readline - The readline line editing library.
• zlib - The zlib compression library.
• bz2 - The bzip2 compression library.
• libcurl - The cURL library for downloading files from URLs. Version 7.23.0 or higher is required.
• libarchive - The libarchive library for opening archive files, like zip/tgz.
• wireshark - The 'tshark' program is used to interpret pcap files.
• cargo/rust - The Rust language is used to build the PRQL compiler.

Build

Lnav follows the usual GNU style for configuring and installing software:

Run ./autogen.sh if compiling from a cloned repository.

$ ./configure
$ make
$ sudo make install

See Also

Angle-grinder is a tool to slice and dice log files on the command-line. If you're familiar with the SumoLogic query language, you might find this tool more comfortable to work with.

Footnotes

1. Files that do not contain log messages can be seen in the TEXT view (reachable by pressing t). ↩

lnav's Issues

make fails on Ubuntu 12.10

Running make I get:

$ make CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/bash /home/user/dev/lnav/missing aclocal-1.14 -I . /home/user/dev/lnav/missing: line 81: aclocal-1.14: command not found WARNING: 'aclocal-1.14' is missing on your system. You should only need it if you modified 'acinclude.m4' or 'configure.ac' or m4 files included by 'configure.ac'. The 'aclocal' program is part of the GNU Automake package: <http://www.gnu.org/software/automake> It also requires GNU Autoconf, GNU m4 and Perl in order to run: <http://www.gnu.org/software/autoconf> <http://www.gnu.org/software/m4/> <http://www.perl.org/> make: *** [aclocal.m4] Error 127

Apt returns nothing for aclocal whatever version number.

I have installed automake & autoconf. I'm stuck.

Any help appreciated...

searching should move horizontally as well

Hitting 'n' will move to the next line, but will not move horizontally to display the text.

Allow the use of yajl installed on the system

We currently use an embedded version of yajl since I didn't want to add yet another dependency, but we should still use the one installed on the system if it's available.

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/10776137-allow-the-use-of-yajl-installed-on-the-system?utm_campaign=plugin&utm_content=tracker%2F449456&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F449456&utm_medium=issues&utm_source=github).

add a hotkey to clear all user bookmarks

Since there are ways to create a lot of user bookmarks now, we should add a hotkey to clear them quickly as well.

drop dependence on xclip

It would be nice to drop the dependence on xclip and just build in the functionality

Installation requires GPM dev files

Just an FYI...

On Debian Wheezy, make was throwing errors like so:

undefined reference to 'Gpm_*'

Installing the package libgpm-dev solved the problem.

gzipped files are handled poorly

We need to use a lock around gzipped files because the gzio implementation does not use pread. So, any gzseeks move the file pointer which causes problems with simultaneous searches.

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/10776152-gzipped-files-are-handled-poorly?utm_campaign=plugin&utm_content=tracker%2F449456&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F449456&utm_medium=issues&utm_source=github).

Add support for clickable fields

A good idea from Michael Permana:

Have the .py:123 link to github's source file or file:///

There are a few approaches to making this happen, not sure which might be the best:

• Display a link and let the terminal turn a click into opening a browser.
  • Display the full link
  • Display a shortened link that lnav redirects
• Require xterm-mouse mode and turn the click into:
  • a call to "open"
  • an expansion of the URL.
• ...

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/10776140-add-support-for-clickable-fields?utm_campaign=plugin&utm_content=tracker%2F449456&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F449456&utm_medium=issues&utm_source=github).

AArch64 support and licensing problem

Please take a look at : https://bugzilla.redhat.com/show_bug.cgi?id=957415

Mentioned:

1.No AUTHORS and ChangeLog file for autoreconf.

2.None of the source files mentions their license, neither do the README files. Both standard BSD 2-clause (./LICENSE) license and GPLv3 (./COPYING) are included.

Thanks

Add warning, highlights, and bookmark stats to histogram view

Right now, we only show total messages and errors in the histogram view.

From Kevin Quick:

Show counts of warnings, highlights (all), and bookmarks in regions
on the histogram, and modify the histogram color to include highlight
counts.

tune the file polling constants for friendlier interactive performance

If a log file is growing rapidly while being viewed, it can interfere with the user scrolling around.

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/10776146-tune-the-file-polling-constants-for-friendlier-interactive-performance?utm_campaign=plugin&utm_content=tracker%2F449456&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F449456&utm_medium=issues&utm_source=github).

add more default files

The set of default files to load is pretty small

files with long lines have performance issues

When viewing a file with really long lines scrolling up and down can be very slow.

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/10776149-files-with-long-lines-have-performance-issues?utm_campaign=plugin&utm_content=tracker%2F449456&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F449456&utm_medium=issues&utm_source=github).

check for replaced files

We should try to detect files that have been replaced and include their contents when automatically tailing.

add a 'force redraw' and/or do a periodic forced redraw

Sometimes, the display gets screwed up (because of kernel / background-process spew) and there's no easy way to get the display looking proper again.

add support for saving and restoring the current session

The typical workflow for lnav is to iteratively process log files: filtering out unimportant lines, highlighting session-specific data, building sql tables. So, we should probably add support for saving and restoring a session so a user can stop and pick it up again later.

Add xterm mouse support

When available, we should pay attention to mouse events. In particular:

• Scrolling with a mouse wheel/trackpad. Supporting multi-touch scrolling has become very important to me recently.
• Mouse clicks should mark lines in the log view. Not sure how this behavior should work. Do we want to support dragging? Should it only work if a modifier key is depressed?
• Clicking on the right side where the scroll bar is should move the view to the click. We might even want to support dragging the scroll bar.

Fix c11 errors/warnings

It looks like there are some c11-related warnings in lnav...

From Christopher Meng:

• make
  Making all in src
  make[1]: Entering directory /home/rpmaker/rpmbuild/BUILD/lnav-0.5.0/src' make all-am make[2]: Entering directory/home/rpmaker/rpmbuild/BUILD/lnav-0.5.0/src'
  g++ -DHAVE_CONFIG_H -I. -Wall -L/usr/lib -I/usr/local/include -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -c -o bookmarks.o bookmarks.cc
  g++ -DHAVE_CONFIG_H -I. -Wall -L/usr/lib -I/usr/local/include -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -c -o grep_proc.o grep_proc.cc
  g++ -DHAVE_CONFIG_H -I. -Wall -L/usr/lib -I/usr/local/include -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -c -o hist_source.o hist_source.cc
  g++ -DHAVE_CONFIG_H -I. -Wall -L/usr/lib -I/usr/local/include -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -c -o line_buffer.o line_buffer.cc
  g++ -DHAVE_CONFIG_H -I. -Wall -L/usr/lib -I/usr/local/include -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -c -o listview_curses.o listview_curses.cc
  g++ -DHAVE_CONFIG_H -I. -Wall -L/usr/lib -I/usr/local/include -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -c -o lnav_commands.o lnav_commands.cc
  In file included from lnav.hh:22:0,
  from lnav_commands.cc:8:
  statusview_curses.hh: In member function 'void status_field::set_value(std::string)':
  statusview_curses.hh:61:25: warning: narrowing conversion of '((status_field_)this)->status_field::sf_cylon_pos' from 'size_t {aka unsigned int}' to 'int' inside { } is ill-formed in C++11 [-Wnarrowing]
  this->sf_width };
  ^
  statusview_curses.hh:61:25: warning: narrowing conversion of '((status_field_)this)->status_field::sf_width' from 'size_t {aka unsigned int}' to 'int' inside { } is ill-formed in C++11 [-Wnarrowing]
  g++ -DHAVE_CONFIG_H -I. -Wall -L/usr/lib -I/usr/local/include -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -c -o log_format.o log_format.cc
  g++ -DHAVE_CONFIG_H -I. -Wall -L/usr/lib -I/usr/local/include -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -c -o logfile.o logfile.cc
  g++ -DHAVE_CONFIG_H -I. -Wall -L/usr/lib -I/usr/local/include -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -c -o logfile_sub_source.o logfile_sub_source.cc
  logfile_sub_source.cc: In member function 'virtual void logfile_sub_source::text_value_for_line(textview_curses&, int, std::string&, bool)':
  logfile_sub_source.cc:143:34: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
  assert(row < this->lss_index.size());
  ^
  g++ -DHAVE_CONFIG_H -I. -Wall -L/usr/lib -I/usr/local/include -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -c -o data_scanner.o data_scanner.cc
  g++ -DHAVE_CONFIG_H -I. -Wall -L/usr/lib -I/usr/local/include -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -c -o data_parser.o data_parser.cc
  g++ -DHAVE_CONFIG_H -I. -Wall -L/usr/lib -I/usr/local/include -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -c -o readline_curses.o readline_curses.cc
  g++ -DHAVE_CONFIG_H -I. -Wall -L/usr/lib -I/usr/local/include -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -c -o sequence_matcher.o sequence_matcher.cc
  g++ -DHAVE_CONFIG_H -I. -Wall -L/usr/lib -I/usr/local/include -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -c -o statusview_curses.o statusview_curses.cc
  In file included from statusview_curses.cc:7:0:
  statusview_curses.hh: In member function 'void status_field::set_value(std::string)':
  statusview_curses.hh:61:25: warning: narrowing conversion of '((status_field_)this)->status_field::sf_cylon_pos' from 'size_t {aka unsigned int}' to 'int' inside { } is ill-formed in C++11 [-Wnarrowing]
  this->sf_width };
  ^
  statusview_curses.hh:61:25: warning: narrowing conversion of '((status_field_)this)->status_field::sf_width' from 'size_t {aka unsigned int}' to 'int' inside { } is ill-formed in C++11 [-Wnarrowing]
  statusview_curses.cc: In member function 'virtual void statusview_curses::do_update()':
  statusview_curses.cc:31:43: warning: narrowing conversion of '(& sf)->status_field::get_width()' from 'size_t {aka unsigned int}' to 'int' inside { } is ill-formed in C++11 [-Wnarrowing]
  struct line_range lr = { 0, sf.get_width() };
  ^
  g++ -DHAVE_CONFIG_H -I. -Wall -L/usr/lib -I/usr/local/include -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -c -o piper_proc.o piper_proc.cc
  g++ -DHAVE_CONFIG_H -I. -Wall -L/usr/lib -I/usr/local/include -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -c -o textview_curses.o textview_curses.cc
  textview_curses.cc: In member function 'void ansi_scrubber::scrub_value(std::string&, string_attrs_t&)':
  textview_curses.cc:123:50: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
  for (int lpc = 0; lpc < range_queue.size(); lpc++) {
  ^
  g++ -DHAVE_CONFIG_H -I. -Wall -L/usr/lib -I/usr/local/include -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -c -o view_curses.o view_curses.cc
  g++ -DHAVE_CONFIG_H -I. -Wall -L/usr/lib -I/usr/local/include -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -c -o vt52_curses.o vt52_curses.cc
  vt52_curses.cc: In member function 'void vt52_curses::map_output(const char_, int)':
  vt52_curses.cc:218:28: warning: variable 'height' set but not used [-Wunused-but-set-variable]
  unsigned long width, height;
  ^
  g++ -DHAVE_CONFIG_H -I. -Wall -L/usr/lib -I/usr/local/include -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -c -o log_vtab_impl.o log_vtab_impl.cc
  log_vtab_impl.cc: In function 'int vt_create(sqlite3_, void_, int, const char_ const_, sqlite3_vtab__, char__)':
  log_vtab_impl.cc:54:9: warning: variable 'rc' set but not used [-Wunused-but-set-variable]
  int rc = SQLITE_OK;
  ^
  g++ -DHAVE_CONFIG_H -I. -Wall -L/usr/lib -I/usr/local/include -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -c -o xterm_mouse.o xterm_mouse.cc
  In file included from xterm_mouse.cc:4:0:
  xterm_mouse.hh: In member function 'void xterm_mouse::handle_mouse(int)':
  xterm_mouse.hh:116:35: warning: suggest parentheses around '-' in operand of '&' [-Wparentheses]
  int bstate = getch() - XT_MAGIC & XT_BUTTON__MASK;
  ^
  rm -f libdiag.a
  ar cru libdiag.a bookmarks.o grep_proc.o hist_source.o line_buffer.o listview_curses.o lnav_commands.o log_format.o logfile.o logfile_sub_source.o data_scanner.o data_parser.o readline_curses.o sequence_matcher.o statusview_curses.o piper_proc.o textview_curses.o view_curses.o vt52_curses.o log_vtab_impl.o xterm_mouse.o
  ranlib libdiag.a
  g++ -DHAVE_CONFIG_H -I. -Wall -L/usr/lib -I/usr/local/include -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -c -o lnav.o lnav.cc
  In file included from lnav.hh:22:0,
  from lnav.cc:45:
  statusview_curses.hh: In member function 'void status_field::set_value(std::string)':
  statusview_curses.hh:61:25: warning: narrowing conversion of '((status_field_)this)->status_field::sf_cylon_pos' from 'size_t {aka unsigned int}' to 'int' inside { } is ill-formed in C++11 [-Wnarrowing]
  this->sf_width };
  ^
  statusview_curses.hh:61:25: warning: narrowing conversion of '((status_field*)this)->status_field::sf_width' from 'size_t {aka unsigned int}' to 'int' inside { } is ill-formed in C++11 [-Wnarrowing]
  In file included from lnav.cc:68:0:
  xterm_mouse.hh: In member function 'void xterm_mouse::handle_mouse(int)':
  xterm_mouse.hh:116:35: warning: suggest parentheses around '-' in operand of '&' [-Wparentheses]
  int bstate = getch() - XT_MAGIC & XT_BUTTON__MASK;
  ^
  lnav.cc: In member function 'virtual void lnav_behavior::mouse_event(int, int, int)':
  lnav.cc:1721:57: warning: comparison between signed and unsigned integer expressions [-Wsign-compare](%28this->lb_scrollbar_y != -1%29 || %28x >= %28width - 2%29%29)) {
  ^
  gcc -DHAVE_CONFIG_H -I. -Wall -L/usr/lib -I/usr/local/include -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -c bin2c.c
  gcc -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -I/usr/include -pthread -static -Wl,-z,relro -L/usr/local/lib -o bin2c bin2c.o libdiag.a -lreadline -lncursesw -lsqlite3 -lpcrecpp -lpcre -ltinfo -ldl -lbz2 -lz -lutil
  /usr/bin/ld: cannot find -lreadline
  /usr/bin/ld: cannot find -lncursesw
  /usr/bin/ld: cannot find -lsqlite3
  /usr/bin/ld: cannot find -lpcrecpp
  /usr/bin/ld: cannot find -lpcre
  /usr/bin/ld: cannot find -ltinfo
  /usr/bin/ld: cannot find -ldl
  /usr/bin/ld: cannot find -lbz2
  /usr/bin/ld: cannot find -lz
  /usr/bin/ld: cannot find -lutil
  /usr/bin/ld: cannot find -lpthread
  /usr/bin/ld: cannot find -lc
  collect2: error: ld returned 1 exit status

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/10776143-fix-c11-errors-warnings?utm_campaign=plugin&utm_content=tracker%2F449456&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F449456&utm_medium=issues&utm_source=github).

shift-j/k should probably reset to the top line after scrolling up/down

If you start marking with shift-j and then move down below the marked region, hitting shift-j again will mark the next line instead of moving to the top line.

drop soci

Soci is nice, but we're not really using any of it's features, so just drop it.

add 'capture-into' command that captures log text and puts it in the db

If the user wants to process log data manually, they should be able to write a regex that captures log data and puts it in a db table. Something like the following:

:capture-into

The data will be placed in 'table-name' in the db and each capture in the regex will be a separate column. There will also be timestamp and sample number columns.

Add an option to jump from the sql view to the log view based on the selected line number.

The histogram view allows you to switch to the log view and move to the time that was at the top of the hist view. The same should be available for the sql view, if "line_number" is one of the columns selected for in the query, the user should be able to jump to that line in the log view.

add a headless mode

the non-viewer portions of the tool might be useful to have access to.

Fix/find a better way to link in help.txt

The help.txt file is linked into lnav using binutils, but it requires some arch-specific settings, which is annoying.

From Kevin Quick:

2. In configure.in, add:

+i686)

•   OBJ_FORMAT=elf32-i386
  

•   ;;
  

  I suspect there's a few other combinations. A better fix might be
  to see if there's any binutil that can tell you the BFD for the
  current system... maybe by checking the BFD of one of the other
  object files you've built.

virtual table performance

The current virtual table backends re-execute the regex every time a column is read instead of caching the result and reusing it.

give more pcre examples

We need to make it more clear that lnav uses pcre and all the goodness that comes with that.

add more columns to the syslog_log table

the syslog_log virtual table should have hostname, pid, ident, etc... columns

add support for can_change_color

Right now we only support the standard terminal colors. This works well enough, but it would be nice to get some other highlight colors, for example, blue is a terrible highlight color right now.

add plugin support

need to find a way to allow people to add their own log formats

make static linking optional

Static linking is hardcoded now because I like being able to copy the binary to any random machine that I'm trying to investigate without having to get all the deps on there too. But, we should make it an option in configure.

From Kevin Quick:

Make the -static link a configure option to allow shared lib linking.

Add filter-word command

The no longer functioning 'scrub' option would hide redundant parts of the log line (years in dates, log levels). Adding the ability to filter within a line would be good too.

From Kevin Quick:

Have a "filter-word" command. Very similar to filter-out, but
applies to words instead of lines. Maybe "filter-regexp" for
better flexibility, but still intra-line. Useful to eliminate
"noise" on the lines to focus on the interesting portion.

fix the build

fix various build issues on 32-bit hosts, freebsd, etc...

auto-completion for sql should be done using the sqlite_master table

The table and column names used for tab-completion are currently hardcoded. We should just pull them out of the sqlite_master table.

Add URI parsing helper functions to sqlite

When processing web access logs with sql, it might be nice to have some functions that can deal with query strings in URLs.

The following might be a good library to use:

http://uriparser.sourceforge.net

There's probably a few other places where some extra functions would help in processing logs with sql.

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/10776141-add-uri-parsing-helper-functions-to-sqlite?utm_campaign=plugin&utm_content=tracker%2F449456&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F449456&utm_medium=issues&utm_source=github).

tab-completion for search

Look into tab-completion for search terms. During the file scan we can build up a dictionary of words in the files and use that for suggesting search terms.

The automatic graphing code should ignore the sc_status field

We already ignore line_number, the same should be done for sc_status since it's not meaningful to graph.

Download link on webiste

It seems you've re designed the lnav webiste, very beautiful.

However I found that latest 0.6.1 download link is:

http://lnav.org/s/lnav-061tar.gz

Well, can you name your tarball like:

http://lnav.org/s/lnav-0.6.1-tar.gz ?

Thanks.

add a configuration screen

Add another screen that contains configuration information: files loaded, highlights, filters, stats, etc...

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/10776145-add-a-configuration-screen?utm_campaign=plugin&utm_content=tracker%2F449456&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F449456&utm_medium=issues&utm_source=github).

preserve readline history

we should preserve the readline history for each mode (search, command, sql)

Add curl support

A nice-to-have would be support for downloading logs from remote URLs.

some ansi colors are not handled correctly

I've noticed that the ansi color decoding is not working properly in some cases. Need to also add a test.

Add a hex-view for binary files

Might want to add a hex view for raw binary files. Not a high priority.

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/10776144-add-a-hex-view-for-binary-files?utm_campaign=plugin&utm_content=tracker%2F449456&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F449456&utm_medium=issues&utm_source=github).

Try out PCRE's JIT compiler

There might be some performance win here...

Allow directories on the command line

We should allow the user to specify a directory on the command-line. In this mode, lnav should monitor the directory and automatically open all text files that are found. Ideally, I would like to be able to type 'lnav .' or 'lnav /var/log/' and it should just work.

Add more information to the right-hand scroll bar

We only display spatial information in the scroll bar on the right. We could also add data for time and user marks.

From Kevin Quick:

In the scroll-bar on the right, add a '-' for each day break and
also add a '*' or something similar for each bookmark.

Autoconf should complain about missing libgpm.

Invocation of 'configure' succeeds without 'libgpm-dev' package on Ubuntu and the only indication given to the user is this:

checking for library containing Gpm_Open... no

However, it fails during the compilation stage with the following errors:

g++ -g -O2 -I/usr/include -pthread -static -L/usr/local/lib -o lnav lnav.o help.o libdiag.a -lreadline -lncurses -lsqlite3 -lpcrecpp -lpcre -ltinfo -ldl -lz -lutil
/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/libsqlite3.a(sqlite3.o): In function unixDlOpen': (.text+0x75f9): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/libreadline.a(complete.o): In functionrl_username_completion_function':
(.text+0x37f9): warning: Using 'getpwent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/libreadline.a(tilde.o): In function tilde_expand_word': (.text+0x22c): warning: Using 'getpwnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/libreadline.a(shell.o): In functionsh_get_home_dir':
(.text+0x14c): warning: Using 'getpwuid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/libreadline.a(complete.o): In function rl_username_completion_function': (.text+0x37ee): warning: Using 'setpwent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/libreadline.a(complete.o): In functionrl_username_completion_function':
(.text+0x3899): warning: Using 'endpwent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/libncurses.a(lib_mouse.o): In function _nc_mouse_event': (.text+0x61e): undefined reference toGpm_GetEvent'
/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/libncurses.a(lib_mouse.o): In function enable_gpm_mouse': (.text+0x78c): undefined reference toGpm_Close'
/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/libncurses.a(lib_mouse.o): In function enable_gpm_mouse': (.text+0x7ce): undefined reference toGpm_Open'
/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/libncurses.a(lib_mouse.o): In function enable_gpm_mouse': (.text+0x7df): undefined reference toGpm_Close'
/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/libncurses.a(lib_mouse.o): In function initialize_mousetype': (.text+0x935): undefined reference togpm_fd'
/usr/lib/gcc/x86_64-linux-gnu/4.6/../../../x86_64-linux-gnu/libncurses.a(lib_mouse.o): In function mouse_activate': (.text+0xb67): undefined reference togpm_fd'
collect2: ld returned 1 exit status
make[2]: *** [lnav] Error 1

break long lines

Currently, long lines are displayed all on one line and you have to scroll left/right to view them. Part of the reason for this is implementation details. However, sometimes it would be nice to view a single content line broken up over multiple display lines.

FR: support for wildcard in filenames on the cmdline

It might be nice to have lnav search for new files that match a pattern given on the command line.

Build fails if there is no libsqlite3 installed.

The invocation of 'configure' succeeds without any sqlite3 installed and passing in the '--with-sqlite3=no', but calling 'make' fails with the following error:

Making all in src
make[1]: Entering directory /backup/home/***/src/lnav/src' make all-am make[2]: Entering directory/backup/home/**/src/lnav/src'
g++ -DHAVE_CONFIG_H -I. -Wall -I/usr/local/include -g -O2 -MT lnav_commands.o -MD -MP -MF .deps/lnav_commands.Tpo -c -o lnav_commands.o lnav_commands.cc
In file included from lnav.hh:8:0,
from lnav_commands.cc:8:
config.h:134:0: warning: "BSD_SOURCE" redefined [enabled by default]
/usr/include/features.h:172:0: note: this is the location of the previous definition
In file included from lnav.hh:29:0,
from lnav_commands.cc:8:
log_vtab_impl.hh:5:21: fatal error: sqlite3.h: No such file or directory
compilation terminated.
make[2]: ** [lnav_commands.o] Error 1
make[2]: Leaving directory /backup/home/***/src/lnav/src' make[1]: *** [all] Error 2 make[1]: Leaving directory/backup/home/**/src/lnav/src'
make: *_* [all-recursive] Error 1

This sort of makes the '--with-sqlite3=[ARG]' option pointless in the configure file. It should either be removed and sqlite3 made into a hard dependency, like libz, pcre and readline; or sqlite3 usage should be encapsulated within the HAVE_SQLITE macro.

add support for tcpdump logs

add a log handler for tcpdump, not sure if we want to use libpcap and operate on captures or just tcpdump output

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/10776151-add-support-for-tcpdump-logs?utm_campaign=plugin&utm_content=tracker%2F449456&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F449456&utm_medium=issues&utm_source=github).