smbache / loggr

Easy and flexible logging for R

loggr - Logging for R

The aim of loggr is to provide a simple mechanism for logging events in R, without the need to wrap expressions in e.g. withCallingHandlers or similar and while keeping the possibility of having several active log outputs (files and/or console) possibly listening for different events.

Features:

• Multiple log outputs (files, console)
• Capture classic events signalled with message, warning and stop.
• Additional event levels, DEBUG, INFO, WARN, ERROR, CRITICAL.

Installation:

devtools::install_github("smbache/loggr")

Usage:

In your R script/program, activate logging by specifying a log file:

log_file("path/to/logfile.log")

Then, whenever a log event (by default, simpleMessages, simpleWarnings, and simpleErrors are considered log events) is signalled, it will be logged to the specified file.

To listen for log events to show in the console, use of

log_file("console") # or
log_file("stdout")

To signal a log event, use one of the following

log_debug(message)     # DEBUG
log_info(message)      # INFO
log_warn(message)      # WARN
log_error(message)     # ERROR  (will not break code execution)
log_critical(message)  # CRITICAL   (breaks code execution)

Specifying which events to log

It is possible to log only certain events when they are raised when specifying the log file:

log_file("/path/to/file.log", WARN, .message = FALSE, .error = FALSE)

This will only listen for WARN events, and has disabled the classic conditions simpleError and simpleMessage, which are logged by default.

NB: the current enabling/disbling of classic events may be changed to align them with the other log events.

Formatting the log entries.

By default, the entries are formatted as e.g.

2015-04-12 15:10:44.601 - WARN - Something bad happened. 

You can change the formatter by specifying it for the log file:

log_file("/path/to/file.log", .formatter = my_formatter)

Where my_formatter should accept a log_event object as argument, and return a character representation.

How it works

loggr adds a hook to warning, stop and signalCondition, so whenever these functions are executed, loggr will be notified (Note that message uses signalCondition internally and does not need its own hook). The event is sent to any log outputs that subscribe to the type of event.

This means that there is very little code needed in the functions where signalling of log events are desired, and the "consumer" only needs to specify a log_file.

Bug reports / suggestions

The package is still young and under development, so if you experience any issues or have suggestions, please file an issue.

loggr's Issues

Traceback log

Awesome library!
Would there be a way to also capture the traceback stack?

Function to parse logs

Hi,

I am currently using loggr for logging in a shiny application. The logs will be useful to analyse user behaviour and check that they are using the app as intended.

Would you consider including a function parse_logs(), which reads a log file and parses the results as a data frame?

SIMPLEERROR - there is no package called ‘variable’

I'm seeing this error show up in my logs, but it's not generated by anything I'm doing. As far as I can tell there's no R package out there called "variable" either.

Can you help me understand where this error is coming from and how I can resolve it? I'd like to keep my logs clean of extraneous stuff.

2017-04-06 21:29:54.558 - DEBUG - HELLO
2017-04-06 21:37:24.893 - SIMPLEERROR - there is no package called ‘variable’
2017-04-06 21:37:25.771 - SIMPLEERROR - there is no package called ‘variable’

logging warnings when there are multiple warnings

When a line of code returns something like

There were 11 warnings (use warnings() to see them)

Is there a way to obtain them?

use with the pipe operator

About to leave my current job and before I go I am frantically refactoring all our R code in production to use loggr (fun Sunday :D). Firstly, let me say it is a fantastic package. Congrats!

Now on to the request... do you think it would be possible to include variants of the logging functions that would work with the pipe operator? e.g.

iris %>%
  log_info("grouping by species") %>%
  group_by(Species) %>%
  log_info("summing Sepal.Length by Species") %>%
  summarise(Sum.Sepal.Length = sum(Sepal.Length)

Combined with ensurer, you have some even more knarly piping for use in production.

log_file does not work when `source`d

log_file only works in the environment from which it is called, and therefore has no effect when in a script which is sourced.

If log_file is executed outside of source it works, even if the remainder of the script is sourced.

Scripts executed with R CMD BATCH and Rscript works fine.

Not sure there is a way to fix things with source though.

features question / FR transactional logging

Hi,
A question - and future request maybe.

Any future plans for incorporating following features?

• transactional logging: log before evaluation and update log after evaluation
• parallel logging support

Asking because I'm developing similar package - with a very similar name jangorecki/logR - and I would consider to switch to other package but those two features and quite important for me.

BTW. a typo in log_file.R#L34

Prevent duplication of messages and warnings?

I like the simplicity of this package, and would like to use it just to add timestamps to my messages and warnings. However, it seems that the timestamped messages appear in addition to the normal ones rather then replacing them. Is there any way to fix this?

> library(loggr)
> log_file("console")
2017-03-17 11:15:51.149 - DEBUG - Activating logging to console
> message("This appears twice")
2017-03-17 11:22:56.405 - SIMPLEMESSAGE - This appears twice
This appears twice

(Obviously I could just replace message with log_info, etc., but I can't do that in other packages' code that uses message.)

Rename deactivate_log() to something internally consistent

Maybe there is a greater reason for this that I am blind to, but I think it's a bit odd that all the other functions in the package are prefixed with log_, only to have the function to quit logging break this syntax.

Would log_close() or log_end() not work and be much more logical given the existing functions?
Just a thought.

Dots argument to log_file

The dots argument to log_file induced a lot of repetition when generalising the interface. At the same time, I think it makes the package harder to use.

Writing:

log_file("mylog.log", INFO)

looks cool, but doesn't really add much more functionality than

log_file("mylog.log", "INFO")

and the latter has the advantage that it's easier to generate lists of subscriptions programmatically:

if (verbose) {
  subscriptions <- c("DEBUG", "INFO", "WARN", "ERROR", "CRITICAL")
} else {
  subscriptions <- c("WARN", "ERROR", "CRITICAL")
}
log_file("mylog.log", subscriptions)

I've added a subscriptions argument to allow this explicitly, but any interest in simplifying the interface?

Avoid logging handled exceptions

Hi I can't figure out how to do this, maybe this needs a pull request.
Basically what I mean to do is to source the following R script:

log_file("testloggr.log", overwrite = TRUE, INFO)
log_info("I want to log this to keep a trace")
e <- sin(pi/6)
#But I don't wish to see any traceback in the logfile for the following
try(log("e"))

#I certainly don't want to see any output from this
fun <- function() {
 a <- tryCatch(na.fail(NA),
           error = function(e) 1)
 a
}
fun()

The last function call writes a message to the logfile which I do not want.
Also for some weird reason if I change the INFO log level to DEBUG, there's no more log_info output. I find this strange since I assume DEBUG is a more verbose mode which should include INFO or maybe I'm missing something.

The use case is am running a bunch of R scripts from the command line and I wan't to log them properly for production mainly.

Ordering of log levels?

I really like the fact that this package captures messages from message(), warning() and stop(). IMO it is a potential killer feature.

However, I miss a simple way to specify that "I want to log all messages of level INFO or higher", etc. As I understand it, currently, one has to specify all levels explicitly in the log_file() call. It would be very useful if the levels were ordered, for instance DEBUG > INFO > (SIMPLE)MESSAGE > WARN = SIMPLEWARNING > ERROR > SIMPLEERROR = CRITICAL.
Other logging systems I've seen do this by assigning a numerical value to each symbolic log level, and then test for >= num_log_level. Would it be possible to implement something like this in loggr? I think it would be very useful, and it would also make it easy to add custom log levels.

Option vs environment

Any reason for using options / getOption to store the global state and not a package environment? I feel like the latter is more typical and is generally easier to work with.

This is probably just a stylistic difference, though I feel that the semantics around the two differ (options users are allowed to poke with, environments are package internals).

Again, happy to work up a PR with this, as it's a really straightforward change. There would be no user-visible differences.

Is there any functionality that enables me to only save custom messages into log file and not the "DEBUG" lines.

Hi, nice package. Ive been using it with my shiny application. Was wondering if there is a way to ensure that the following lines are not saved in my log file!

2021-05-06 20:15:16.553 - DEBUG - Activating logging to ~path_to_log/logfile.log
2021-05-06 20:15:28.352 - DEBUG - Activating logging to console

I only want my custom messages:
2021-05-06 20:15:28.358 - SIMPLEMESSAGE - Running method 1
2021-05-06 20:15:28.361 - SIMPLEMESSAGE - Successful method 1

Lock the output file to avoid issues in parallel environments

Will require a file locking package - seagull is headed to cran soonish and may work here.

Code injection / hook set up

This seems unlikely to pass muster with CRAN; already some obfuscation is required to avoid the unsafe NOTE in R CMD check. Or worse; it will pass and then get archived.

I believe it is possible to use trace for this purpose instead, with advantages:

• Less Ripley
• easy temporary disabling with tracingState
• simpler deactivation
• don't have to delete _notify_loggr from base, which is not currently possible
• simpler and more transparent

Implementation would look approximately like this (replacing body of use_logging:

catch_signal <- quote(loggr:::notify_loggr(cond))
catch_stop <- quote(loggr:::notify_loggr(..., call.=call., domain=domain, type="error"))
catch_warning <- quote(loggr:::notify_loggr(..., call.=call., immediate.=immediate., noBreaks.=noBreaks., domain=domain, type="warning"))
suppressMessages({
  trace(base::signalCondition, catch_signal, print=FALSE)
  trace(base::stop, catch_stop, print=FALSE)
  trace(base::warning, catch_warning, print=FALSE)
})

and then set_loggr_hook becomes unnecessary. The unset_loggr_hook (in PR #9) becomes really simple as it is just a set of untrace calls.

I'm happy to work up a PR with a full implementation.

Cannot open file [...] No such file or directory

Hi there,

Love the simple logging utility for R. Say, I'm currently getting a weird error--weird because I can see loggr create the log file, and I can see it writing to the log file. Yet, I'm seeing constant warning messages in the console (very annoying!) which say that it cannot open the log file and that there's no such file or directory. Clearly it can open the file because it is opening the file and writing to it.

Thoughts?

library(loggr)
# setup logging
log_file(paste0("logs/gravity_well_",Sys.Date(),".log"), overwrite = FALSE)

> log_debug("HELLO")
Warning message:
In file(file, ifelse(append, "a", "w")) :
  cannot open file '/logs/gravity_well_2017-04-06.log': No such file or directory

File permissions are:
-rw-r--r-- 1 abe-burnett abe-burnett 371 Apr 6 21:29 gravity_well_2017-04-06.log

Should overwrite be TRUE or FALSE by default?

The help file for log_file says:

#' @param overwrite logical: whether or not to overwrite the file at
#' \code{file_name} if it already exists. Set to FALSE by default.

However, the default is actually TRUE: https://github.com/smbache/loggr/blob/master/R/log_file.R#L51

Which is the preferred behavior?

Messages being truncated

I'm looking at using loggr in a scenario where I'm creating new processes using system2(command = "Rscript" , args = c('--vanilla','my_script.R'), stdout = stdout_file, stderr = stderr_file, wait = FALSE), where I use loggr inside the script my_script. However, when I compare the log file to the stderr file, the loggr file seems to have truncated the messages. For example, a stderr file might have the contents:

Warning messages:
1: There were 38 divergent transitions after warmup. Increasing adapt_delta above 0.8 may help. See
http://mc-stan.org/misc/warnings.html#divergent-transitions-after-warmup 
2: Examine the pairs() plot to diagnose sampling problems

But the log file only has:

There were 
Examine the pairs() plot to diagnose sampling problems

All I'm doing to activate loggr in my_script.R is:

library(loggr)
my_formatter <- function(event) event$message
log_file(log_file_name,.formatter = my_formatter,subscriptions=c('message', 'warning','stop'))

Any idea what I should be doing instead to avoid truncation of those messages?

loggr must be attached to work

With changes to log_info (in de5cf83), loggr no longer works when not attached:

loggr::log_file("console")
#2015-07-06 16:23:08.735 - INFO - Activating logging to console
loggr::log_info("foo")
# Error in eval(expr, envir, enclos) : 
#  could not find function "log_with_level"

Loading the package fixes this:

library(loggr)
loggr::log_info("foo")

Is this unavoidable with the changes to allow use in pipelines?

Easier message generation

Maybe this needs consideration outside the development of loggr, but it would be great if the logging functions supported string interpolation e.g.

...
user_name = "Alice"
log_info("User $user_name has logged in")

or failing that:

...
user_name = "Alice"
log_info("User $user_name has logged in", user_name = user_name)

Change level for activating log message

Would you be willing to change the level for the Activating logging to ... message to DEBUG? My reasoning is that users might not need that information in the log, and setting the level to DEBUG only includes it if the user wants all messages. If you're willing, I can send a PR.

repeat of log4r?

Not sure if you are aware of this package: https://github.com/johnmyleswhite/log4r?

It looks like there is at least some cross over.

Brief comparison with other logging options

REAME section comparing different options.

CRAN/release targets

We should settle on features that we focus on for a first release; something like

• logging of events raised with message, warning, stop, without need for the user to change code to achieve this (no need for calling handlers, tryCatch etc).
• main log events; DEBUG, INFO, WARN, ERROR, CRITICAL.
• (multiple) console and file outputs.
• default formatters (file, json, ?)
• switch to ignore/include suppressed warnings & messages
• vignette?
• string interpolation (maybe include explicitely, until a part of CRAN stringr)

Other?