Wrong log_time in sql view when log_time is out of order about lnav

[jefft]

I just hit this too. To replicate:

cat > access.log <<EOF
localhost:443 1.2.3.4 1j0vhgd jsmith [10/Aug/2023:02:00:00 -0700] "GET /foo HTTP/2.0" 200 17163 "-" "Mozilla/5.0" 116075
localhost:443 1.2.3.4 cger1e khenry [10/Aug/2023:01:00:00 -0700] "GET /bar HTTP/2.0" 200 16517 "-" "Mozilla/5.0" 156302
EOF
lnav access.log -n -c ';select log_time from access_log;'

Expected output:

       log_time         
2023-08-10 02:00:00.000 
2023-08-10 01:00:00.000 

Actual output (lnav 0.11.2-97-g7c8d323):

       log_time         
2023-08-10 02:00:00.000 
2023-08-10 02:00:00.000 

[tstack]

lnav expects/likes to have the messages in a file be in time-order so that it doesn't have to do heavy re-sorting work frequently. So, when that isn't the case, it treats the message as out-of-time-order and uses the timestamp from the previous log message. In the log view, you can see the original timestamp and further details by pressing p. In the SQL interface, you can access the original timestamp with the log_actual_time column[1].

If you really want things to be in time-order, you can patch the log file format and set the ordered-by-time flag to false. Patching the format can be done by putting the following fragment in a .json file in your lnav dot directory (e.g. ~/.lnav/formats/installed or ~/.config/lnav/formats/installed)

{
    "access_log": {
        "ordered-by-time": false
    }
}

[1] - Reading the actual time can be costly since the log message needs to be read from disk.

[jefft]

In the SQL interface, you can access the original timestamp with the [log_actual_time (https://docs.lnav.org/en/latest/sqlext.html#log-tables) column[1].

Nice, thanks.

For me this arose when I ran lnav on <(cat access.log access.log.1 | grep ...). It was trivial to fix the order once I knew about the log_time behaviour, which is nicely documented in TFM I didn't R.