eddelbuettel / nanotime

Nanosecond Resolution Time Functionality for R

nanotime: Nanosecond-Resolution Time Objects for R

Motivation

R has excellent tools for dates and times. The Date and POSIXct classes (as well as the 'wide' representation in POSIXlt) are versatile, and a lot of useful tooling has been built around them.

However, POSIXct is implemented as a double with fractional seconds since the epoch. Given the 53 bits accuracy, it leaves just a bit less than microsecond resolution. Furthermore, using floating-point arithmetic for an integer concept opens the door to painful issues of error accumulation.

More and more performance measurements, latency statistics, etc., are now measured more finely, and we need nanosecond resolution for which commonly an integer64 is used to represent nanoseconds since the epoch.

And while R does not have a native type for this, the bit64 package by Jens Oehlschlägel offers a performant one implemented as a lightweight S3 class. So this package uses the integer64 class, along with multiple helper functions for parsing and formatting at nano-second resolution from the RcppCCTZ package which wraps the CCTZ library from Google. CCTZ is a modern C++11 library extending the (C++11-native) chrono type.

In addition to the point-in-time type nanotime, this package also provides an interval type nanoival which may have open or closed start/end, a period type nanoperiod that is a human representation of time, such as day, month, etc., and a duration type nanoduration. These types are similar to what the lubridate package proposes.

Set and arithmetic operations on these types are available. All functionality is designed to correctly handle instances across different time zones. Because these temporal types are based on R built-in types, most functions have an efficient implementation and the types are suitable for use in data.frame and data.table. nanotime is also a better choice than the native POSIXct in most of the cases where fractional seconds are needed because it avoids floating point issues.

Documentation

Package documentation, help pages, a vignette, and more is available here.

Demo

See the included demo script nanosecondDelayExample.R for a (completely simulated and hence made-up) study of network latency measured in nanoseconds resulting in the figure below

Examples

Simple Parsing and Arithmetic

R> x <- as.nanotime("1970-01-01T00:00:00.000000001+00:00")
R> x
[1] "1970-01-01T00:00:00.000000001+00:00"
R> x + 1e9
[1] "1970-01-01T00:00:01.000000001+00:00"
R> as.nanotime("2020-03-21 Europe/London")
[1] 2020-03-21T00:00:00+00:00

Vectorised

R> options("width"=60)
R> v <- nanotime(Sys.time()) + 1:5
R> v
[1] 2020-03-22T03:09:20.732122001+00:00
[2] 2020-03-22T03:09:20.732122002+00:00
[3] 2020-03-22T03:09:20.732122003+00:00
[4] 2020-03-22T03:09:20.732122004+00:00
[5] 2020-03-22T03:09:20.732122005+00:00
R>

Use with zoo

R> library(zoo)
R> z <- zoo(cbind(A=1:5, B=5:1), v)
R> options(nanotimeFormat="%H:%M:%E*S")  ## override default format
R> z
R> options(nanotimeFormat=NULL)  ## go back to default format
R> z

Use with data.table

R> library(data.table)
R> dt <- data.table(v, cbind(A=1:5, B=5:1))
R> fwrite(dt, file="datatableTest.csv")  # write out
R> dtcheck <- fread("datatableTest.csv") # read back
R> dtcheck
R> dtcheck[, v:=nanotime(v)]             # read as a string, need to re-class as nanotime
R> fread("../datatableTest.csv", colClasses=c("nanotime", "integer", "integer"))

Use with data.frame

This requires version 0.0.2 or later.

R> data.frame(cbind(A=1:5, B=5:1), v=v)

Intervals

R> ival <- as.nanoival("[phone]:12:00 America/New_York -> 2009-02-01 15:11:03 America/New_York-")
R> ival
[1] [phone]T18:12:00+00:00 -> 2009-02-01T20:11:03+00:00-

R> start <- nanotime("2009-01-01 13:12:00 America/New_York")
R> end   <- nanotime("2009-02-01 15:11:00 America/New_York")
R> nanoival(start, end)                   # by default sopen=F,eopen=T
[1] [phone]T18:12:00+00:00 -> 2009-02-01T20:11:00+00:00-
R> nanoival(start, end, sopen=FALSE, eopen=TRUE)
[1] [phone]T18:12:00+00:00 -> 2009-02-01T20:11:00+00:00-

R> intersect(as.nanoival("[phone] UTC -> 2020-03-01 UTC-"),
             as.nanoival("[phone] UTC -> 2020-06-01 UTC-"))
[1] [phone]T00:00:00+00:00 -> 2020-03-01T00:00:00+00:00-

R> union(as.nanoival("[phone] UTC -> 2020-03-01 UTC-"),
         as.nanoival("[phone] UTC -> 2020-06-01 UTC-"))
[1] [phone]T00:00:00+00:00 -> 2020-06-01T00:00:00+00:00-

R> setdiff(as.nanoival("[phone] UTC -> 2020-03-01 UTC-"),
           as.nanoival("[phone] UTC -> 2020-06-01 UTC-"))
[1] [phone]T00:00:00+00:00 -> 2020-01-01T00:00:00+00:00-    

Periods

R> as.nanoperiod("1y1m1w1d/01:01:01.000_000_001")
[1] 13m8d/01:01:01.000_000_001
R> nanoperiod(months=13, days=-1, duration="01:00:00")
[1] 13m-1d/01:00:00

R> ones <- as.nanoperiod("1y1m1w1d/01:01:01.000_000_001")
R> nanoperiod.month(ones); nanoperiod.day(ones); nanoperiod.nanoduration(ones)
[1] 13
[1] 8
[1] 01:01:01.000_000_001

R> plus(v, as.nanoperiod("1y1m"), tz="UTC")
[1] 2021-04-22T03:09:20.732122001+00:00
[2] 2021-04-22T03:09:20.732122002+00:00
[3] 2021-04-22T03:09:20.732122003+00:00
[4] 2021-04-22T03:09:20.732122004+00:00
[5] 2021-04-22T03:09:20.732122005+00:00

Durations

R> nanoduration(hours=1, minutes=1, seconds=1, nanoseconds=1)
R> as.nanoduration("00:00:01")
R> as.nanoduration("-00:00:01")
R> as.nanoduration("100:00:00")
R> as.nanoduration("00:00:00.000_000_001")

Sequences

R> from <- as.nanotime("2018-09-14T12:44:00+00:00")
R> seq(from, by=as.nanoperiod("1y"), length.out=4, tz="Europe/London")
[1] 2018-09-14T12:44:00+00:00
[2] 2019-09-14T12:44:00+00:00
[3] 2020-09-14T12:44:00+00:00
[4] 2021-09-14T12:44:00+00:00

Technical Details

The bit64 package (by Jens Oehlschlägel) supplies the integer64 type used to store the nanosecond resolution time as (positive or negative) offsets to the epoch of January 1, 1970. The RcppCCTZ package supplies the formatting and parsing routines based on the (modern C++) library CCTZ from Google, when the parsing cannot be done using a fast built-in parser. integer64 is also used for the type nanoduration, whereas nanoival and nanoperiod are stored in a complex, i.e. over 128 bits.

Status

The package is by now fairly mature, has been rewritten once (to go from S3 to S4) and has recently received a sizeable feature extension. There may still be changes, though there should generally never be breaking ones. The package also has an extensive test suite, and very good code coverage.

See the issue tickets for an up to date list of potentially desirable, possibly planned, or at least discussed items.

Installation

The package is on CRAN and can be installed via a standard

install.packages("nanotime")

whereas in order to install development versions a

remotes::install_github("eddelbuettel/nanotime")  # dev version

should suffice.

Authors

Dirk Eddelbuettel and Leonardo Silvestri

License

GPL (>= 2)

nanotime's Issues

inherit integer64 class

I wonder if it would make any problem if nanotime class would also inherits integer64?
It could be useful when dispatching, for example instead of following code:

SWITCH(TYPEOF(x)) {
  case REALSXP: {
    if (inherits(x,"integer64") || inherits(x,"nanotime")) {
...
    } else { // numeric
...
    }

It would be sufficient to use:

SWITCH(TYPEOF(x)) {
  case REALSXP: {
    if (inherits(x,"integer64")) {
...
    } else { // numeric
...
    }

In data.table we use similar logic for IDate class, which inherits from Date, thus we need to make single check for inheritance, not two.

Error in callNextMethod() : error in evaluating a 'primitive' next method while doing rbind of data containing nanotime

Error in callNextMethod() : error in evaluating a 'primitive' next method: Error in as.integer64(value) :argument "value" is missing, with no default
Calls: [<- -> [<-.integer64 -> as.integer64 [<- -> new -> initialize -> initialize -> callNextMethod

Why could I be facing this issue ?

Define a shorter notation to get `nanotime-nanoival` subset index

This would allow an efficient subsetting of a data.table time-series with a nanotime column with more clarity than using intersect.idx.

Given a data.table dt with a nanotime index idx and a vector of nanoival ni, we can then write:

dt[idx %in% ni]

and get a subset of a dt.

Display of `nanotime` types incorrect in Jupyter Notebook

In Jupyter Notebook, the nanotime types do not display correctly except for nanoperiod. See what we can do about that...

format.nanotime: attribute 'tzone' and parameters 'format' and 'tz'

Having format.nanotime behave like format.POSIXct would make it easier to replace POSIXct by nanotime. One specific example is for the 'xts' package.

as.integer64 returns an error

I'm not sure if this is exactly an issue with nanotime but since as.integer64.nanotime is listed as a function in the help file for it, I wasn't expecting this behavior.

library(nanotime)
nt <- nanotime("2017-02-02T17:16:27.633531000+00:00")
as.integer64(nt)
#> Error in eval(expr, envir, enclos): could not find function "as.integer64"
suppressMessages(library(bit64))
as.integer64(nt)
#> integer64
#> [1] 1486055787633531000

Should I have to explicitly load bit64 for this to work or is there a configuration issue with nanotime? Perhaps this is just a lack of understanding on my part.

Provide aggregation functionality for use with data.table

Replicate functionality described in Rdatatable/data.table#1390

Support for abs()

This is apparently causing a problem for displaying a nanotime column in a tibble.
r-lib/pillar#378

summary fails on a nanotime vector

summary(nanotime((1:100)*3600e9))
                                 Min.                               1st Qu. 
"1970-01-01T00:00:00.000000000+00:00" "1970-01-01T00:00:00.000000000+00:00" 
                               Median                                  Mean 
"1970-01-01T00:00:00.000000000+00:00" "1970-01-01T00:00:00.000000000+00:00" 
                              3rd Qu.                                  Max. 
"1970-01-01T00:00:00.000000000+00:00" "1970-01-01T00:00:00.000000000+00:00" 
Warning message:
In qtile.integer64(x, probs = probs, na.rm = na.rm, names = names,  :
  class 'nanotime' has no 'names' slot; assigning a names attribute will create an invalid object

(Edited: You want three opening ticks followed by the language, here r. --Dirk)

strange parsing behavior

I am getting some strange behavior using the nanotime() constructor.

I hesitate to claim it's a bug because of the possibility that I am calling it incorrectly.

> nanotime("2015-11-18T05:33:03.765333000+00:00")
[1] "1970-01-17T18:10:24.783765333+00:00"

Using nanotime from CRAN on Mac (R 3.3.2)

BTW, really liking this package for filling the need for "sub-millisecond" timestamps!

Thanks!

Different format strings for parser and formatter

There is a lot to be said for ISO 8601 as well as RFC 3339 though.

Also see SO question on difference between them.

A finisher to an old issue

#39

I cannot help you !

Try this

nanotime(1531830678123456789)

You will lose your precision !

Only to prove you need to convert to character !

Incorrect subsetting with operator `%in%`

Looks like some interference with bit64. We've seen that previously and have always been able to get around it by defining and exporting S3 methods. For some reason it does not work with %in%. It passes the unit tests because it works when the package is test loaded, but it doesn't work straight out of the box.

So this does not work:

library(nanotime)
one_second <- 1e9
index <- seq(nanotime("2022-12-12 12:12:10+00:00"), length.out=10, by=one_second)
ival <- c(as.nanoival("-2022-12-12 12:12:10+00:00 -> 2022-12-12 12:12:14+00:00-"),
          as.nanoival("[phone]:12:18+00:00 -> 2022-12-12 12:12:20+00:00+"))
index %in% ival
##  [1] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE

But this works:

library(bit64)
library(nanotime)
one_second <- 1e9
index <- seq(nanotime("2022-12-12 12:12:10+00:00"), length.out=10, by=one_second)
ival <- c(as.nanoival("-2022-12-12 12:12:10+00:00 -> 2022-12-12 12:12:14+00:00-"),
          as.nanoival("[phone]:12:18+00:00 -> 2022-12-12 12:12:20+00:00+"))
index %in% ival
##  [1] FALSE  TRUE  TRUE  TRUE FALSE FALSE FALSE FALSE  TRUE  TRUE

I have tried to fix this, but for the moment I haven't been successful with whatever type of export I've used in NAMESPACE. Will have to dig into this a bit more.

Fix windows behaviour in tests

Dang. win-builder and R Hub fall over:

     Running 'data.frame.R'
     Running 'runTests.R'
   Warning message:
   running command '"C:/PROGRA~1/R/R-devel/bin/x64/R" CMD BATCH --vanilla  "runTests.R" "runTests.Rout"' had status 1 
    ERROR
   Running the tests in 'tests/runTests.R' failed.
   Last 13 lines of output:
         still.identical
     
     The following objects are masked from 'package:base':
     
         %in%, :, is.double, match, order, rank
     
     Error in runTestSuite(ts, useOwnErrorHandler = useOwnErrorHandler, verbose = verbose) : 
       invalid test suite supplied.
     Calls: doTest -> runTestFile -> runTestSuite
     In addition: Warning messages:
     1: In isValidTestSuite(testSuites) :
       specified directory ../nanotime/unitTests not found.
     2: In isValidTestSuite(testSuites[[1]]) :
       'testSuite' object is not of class 'RUnitTestSuite'.
     Execution halted

Timestamp parsing error

Hi there!

I am facing the following error while using nanotime with my timestamps.
R version: 3.2.3
Platform: Ubuntu 16.04.6 LTS (x86_64)

> nanotime("2020-08-20 03:45:30.511400598", format ="%Y-%m-%d %H:%M:%E9S",  tz ="GMT")
[1] "20200820 03:45:30.511400598"

> nanotime("20200820 03:45:30.511400598", format ="%Y%m%d %H:%M:%E9S",  tz ="GMT")
Error in RcppCCTZ::parseDouble(x, fmt = format, tz = tz) : 
  Parse error on 20200820 03:45:30.511400598

As you see, it works with date in %Y-%m-%d format, but doesn't work for %Y%m%d. Is there any issue in the way I'm calling nanotime?

Thanks in advance!

[[ method for nanotime

[[ is not defined for nanotime:

library(nanotime)

times <- c(nanotime('2011-12-05 08:30:00.000',format ="%Y-%m-%d %H:%M:%E9S",  tz ="GMT"),
  nanotime('2011-12-05 08:30:00.100',format ="%Y-%m-%d %H:%M:%E9S",  tz ="GMT"),
  nanotime('2011-12-05 08:30:00.825',format ="%Y-%m-%d %H:%M:%E9S",  tz ="GMT"))

times[1]
#> [1] "2011-12-05T08:30:00.000000000+00:00"
times[[1]]
#> integer64
#> [1] 1323073800000000000

related to tidyverse/dplyr#4381

%in%.nanotime works only if bit64 is explicitly loaded

Here is an example:

a <- seq(nanotime("2012-12-12 12:12:12+00:00"), length.out=10, by=1e9)
idx <- as.nanoival("[phone]:12:14+00:00 -> 2012-12-12 12:12:19+00:00+")
r <- list(x=c(3,4,5,6,7,8), y=c(1,1,1,1,1,1))
expect_identical(a %in% idx, 1:10 %in% c(3,4,5,6,7,8))  ## fails when `bit64` is not loaded

as.character(nanotime) returns the underlying integer

Thank you for nanotime! Just came across this :

x = nanotime(1)
x
# [1] 1970-01-01T00:00:00.000000001+00:00
as.character(x)
# [1] "1"                                      # observed
# [1] "1970-01-01T00:00:00.000000001+00:00"    # expected

expectation arises from Date and POSIXct :

x = as.Date("1970-01-01")
as.character(x)
# [1] "1970-01-01"   # as expected

x = as.POSIXct("1970-01-01")
as.character(x)
# [1] "1970-01-01"  # as expected

> sessionInfo()
R version 4.1.1 (2021-08-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Pop!_OS 21.04

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.13.so

Random number generation:
 RNG:     Mersenne-Twister 
 Normal:  Inversion 
 Sample:  Rounding 
 
locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8      
 [8] LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] nanotime_0.3.3

loaded via a namespace (and not attached):
[1] zoo_1.8-9       bit_4.0.4       compiler_4.1.1  RcppCCTZ_0.2.9  Rcpp_1.0.7      bit64_4.0.5     grid_4.1.1      lattice_0.20-45
> 

Include trailing 0 (always 9 digits) for easier scanning

For example, this looks like 7 nanoseconds on first glance :

> format(nanotime(cooked$rdsent[1]))
[1] "2016-09-28T15:30:00.00000007+00:00"

but you have to look closely to realize there's just 8 subsecond digits there. So it's actually 70 nanoseconds.

Current behaviour :

format(nanotime(as.integer64("1475076600000000070")))
[1] "2016-09-28T15:30:00.00000007+00:00"
format(nanotime(as.integer64("1475076600000000071")))
[1] "2016-09-28T15:30:00.000000071+00:00"

Desired behaviour :

format(nanotime(as.integer64("1475076600000000070")))
[1] "2016-09-28T15:30:00.000000070+00:00"
format(nanotime(as.integer64("1475076600000000071")))
[1] "2016-09-28T15:30:00.000000071+00:00"

Could you implement comparaison on nanoduration and Character

Hello @eddelbuettel and @lsilvest
Would you consider implementing comparaison for nanoduration as you did for nanotime ?

library(nanotime)
options(nanotimeFormat = "%Y-%m-%dT%H:%M:%E9S")
# this is a nanoduration
as.nanotime("2020-11-30T16:11:39.860999936") - as.nanotime("2020-11-30T16:11:38.960999936")
[1] 00:00:00.900
# you can compare nanotime to Character
as.nanotime("2020-11-30T16:11:39.860999936") < "2020-11-30T16:11:38.960999936"
[1] FALSE
# but you cannot compare nanoduration to Character
(as.nanotime("2020-11-30T16:11:39.860999936") - as.nanotime("2020-11-30T16:11:38.960999936")) < "00:00:00.999"
[1] NA
# the expected output would be the same as
(as.nanotime("2020-11-30T16:11:39.860999936") - as.nanotime("2020-11-30T16:11:38.960999936")) < as.nanoduration("00:00:00.999")
[1] TRUE
attr(,".S3Class")
[1] "integer64"

I think that would be very useful, I would personally use nanoduration as replacement of hms
Kind regards

PS: ran in R 4.0.3 --vanilla

subsetting with NA incorrect

> as.nanotime(1)[NA]
[1] 2262-02-18T20:47:17.227407266+00:00

For nanotime that's because integer64 is not handling NA correctly:

> as.integer64(1)[NA]
integer64
[1] 9218868437227407266

As opposed to:

> as.integer(1)[NA]
[1] NA

So for nanotime and nanoduration we could ask Jens or go ahead and implement our own subsetting functions.

But unfortunately we are also handling our non-integer64-based types incorrecty:

> as.nanoival("[phone] UTC -> 2012-02-02 UTC-")[NA]
[1] [phone]T22:23:38.613703633+00:00 -> 2116-01-25T22:23:38.613703633+00:00+
> as.nanoperiod("1d")[NA]
[1] 1954m2146435072d/2560796:47:17.227_407_266

Test failure on some non-x86 systems

When running tests on non-x86 systems, some of them will fail. This build runs on a variety of systems and fails with tests on some of them. If you check the ppc64, s390x, or armv7hl builds, they all fail a single test. Unfortunately, I don't have interactive access to s390x or armv7hl systems, but I can check the result on ppc64.

There seems to be some problem in test_format_default:

test_format_default: FAILURE !! (check number 2)
Error in checkEquals(format(nanotime("1680-07-17T00:00:01.000000000+00:00")),  : 
  1 string mismatch

If I try writing it out manually, the result is:

> format(nanotime("2120-01-01T00:00:59.987654321+00:00"))
[1] "2120-01-01T00:00:59.987654833+00:00"

so there seems to be some kind of floating point issue in parsing things.

Fix 0-length vectors for ops handled at C++ level

> nanoival(nanotime(1), nanotime(2)) + integer()

Process R floating point exception at Fri Mar 27 15:51:29 2020

This is not so fabulous...

format.nanotime loses nanotime precision

Can be reproduced like this:

n <- nanotime("2120-01-01T00:00:59.987654321+00:00")
format(n)
## [1] "2120-01-01T00:00:59.987653809+00:00"

Convenience constructor for seconds + nanoseconds

Due to R's data type restrictions (and the usual implementation of struct timespec) it would be nice to have a shortcut way to instantiate a nanotime from two integer values: seconds since epoch, nanoseconds since last second).

Not critical, just convenience.

Vectorize

Requires corresponding change in RcppCCTZ first

Constructor bug?

This works:

R> nanotime(c(0,1,2))
[1] "1970-01-01T00:00:00.000000000+00:00" "1970-01-01T00:00:00.000000001+00:00"
[3] "1970-01-01T00:00:00.000000002+00:00"
R> 

This doesn't, but should methinks:

R> nn <- c(0,1,2)
R> oldClass(nn) <- "integer64"
R> new("nanotime", nn)
[1] "1970-01-01T00:00:00.000000000+00:00" "2115-12-30T20:53:38.800017408+00:00"
[3] "2116-02-20T23:53:38.427387904+00:00"
R> 

duration.cpp:96:7: error: no viable overloaded '-=' with nanotime 0.3.0 on Travis MacOS

Just seen this from Travis MacOS. (Travis Linux seems ok, and it's ok on Linux locally for me).
https://travis-ci.org/github/Rdatatable/data.table/jobs/715689511

trying URL 'https://cloud.r-project.org/src/contrib/nanotime_0.3.0.tar.gz'
Content type 'application/x-gzip' length 535610 bytes (523 KB)
==================================================
downloaded 523 KB
* installing *source* package ‘nanotime’ ...
** package ‘nanotime’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
clang++ -mmacosx-version-min=10.13 -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Users/travis/R/Library/Rcpp/include' -I'/Users/travis/R/Library/RcppCCTZ/include' -I'/Users/travis/R/Library/RcppDate/include' -I/usr/local/include  -I../inst/include -fPIC  -Wall -g -O2  -c RcppExports.cpp -o RcppExports.o
clang++ -mmacosx-version-min=10.13 -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Users/travis/R/Library/Rcpp/include' -I'/Users/travis/R/Library/RcppCCTZ/include' -I'/Users/travis/R/Library/RcppDate/include' -I/usr/local/include  -I../inst/include -fPIC  -Wall -g -O2  -c duration.cpp -o duration.o
duration.cpp:96:7: error: no viable overloaded '-='
    d -= ns * std::chrono::nanoseconds(1);
    ~ ^  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Applications/Xcode-9.4.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/chrono:578:71: note: candidate function not viable: no known conversion from 'duration<[...], ratio<[...], 1000000000>>' to 'const duration<[...], ratio<[...], 1000000>>' for 1st argument
    _LIBCPP_INLINE_VISIBILITY _LIBCPP_CONSTEXPR_AFTER_CXX14 duration& operator-=(const duration& __d) {__rep_ -= __d.count(); return *this;}
                                                                      ^
1 error generated.
make: *** [duration.o] Error 1
ERROR: compilation failed for package ‘nanotime’
* removing ‘/Users/travis/R/Library/nanotime’
* restoring previous ‘/Users/travis/R/Library/nanotime’

Add to awesome-vctrs?

This package looks like a good fit for https://github.com/krlmlr/awesome-vctrs#date-and-time, a list of vector classes I curate. Would you like to add it to that list? Happy to review a pull request.

nanotime(NA) should not return a nanotime

> nanotime(NA)
[1] "1677-09-21T00:12:43.145224192+00:00"

TZ support

Should be simple matter of setting attribute and passing to formatting helper from RcppCCTZ

The 'parse at' one is always UTC for consistency with POSIXct, the 'formatted' one from the user or local time?

all.equal on NA nanotime incorrect

issue also affects bit64

library(bit64)
all.equal(as.integer64(c(1L,NA)), as.integer64(1:2))
#[1] TRUE

library(nanotime)
all.equal(nanotime(c(1L,NA)), nanotime(1:2))
#[1] TRUE

is there anything we can do about that?

using unclass on each input shows that the problem is about floating point comparison tolerance.

all.equal(unclass(as.integer64(c(1L,NA))), unclass(as.integer64(1:2)), tolerance=0)
#[1] "Mean absolute difference: 9.881313e-324"

Use namespace at C++ level

interval and period at C++ level are not in a namespace: that's potentially problematic when using the nanotime header files in other packages. Also, put header files in inst/include/ztsdb/ instead of inst/include: helpful because the include in other packages then clearly identifies where they come from.

Windows support

Maybe a little tricky to get at std::chrono without CCTZ and Rcpp but doable. Would forgo the formatting, parsing, TZ help.

At a strict minimum someone driven-enough could extend Simon's fasttime to return two doubles (in UTC time) as we do -- which would cover parsing. Formating could be done as a kludge with strftime and sprintf as we already do for fractional seconds. Likely brittle though.

Maybe someone should just port CCTZ to Windows? Or maybe just build it and have it skip TZ operations for lack of zoneinfo and still parse / format at UTC?

nanotime arithmetic?

Another point that it blocking in order for xts to work with nanotime is arithmetic. In particular:

nanotime("2017-02-02T17:16:27.633531000+00:00") + as.integer64(1e9)
[1] "2017-02-02T17:16:27.633531000+00:00"
Warning message:
Incompatible methods ("Ops.nanotime", "+.integer64") for "+" 

The above works with double but not with integer64. I don't know how to solve this...

The fact that nanotime subtraction gives back a nanotime might be also problematic:

> nanotime("2017-02-02T17:16:27.633531000+00:00") - nanotime("2017-02-02T17:16:26.633531000+00:00")
[1] "1970-01-01T00:00:01.000000000+00:00"

Did you have a specific idea in mind for arithmetic?

Comparing nanotime with a character

Hello, I am using nanotime in conjuction with data.table a lot now and I would have thought being able to compare nanotime with character would be good syntactic suggar.
I mean

options(nanotimeFormat = "%Y-%m-%dT%H:%M:%E9S")
nanotime("2018-12-28T16:34:59.649943448") > "2018-12-28T16:34:59.000000000"
[1] NA

would be equivalent to

options(nanotimeFormat = "%Y-%m-%dT%H:%M:%E9S")
nanotime("2018-12-28T16:34:59.649943448") > nanotime("2018-12-28T16:34:59.000000000")
[1] TRUE

I data.table this get very appreciable I think and this links well with #51
I am happy to try to do a PR but wanted to check first that I am not missing something and/or/hence the PR would be rejected

printing tibbles with nanotime

Hi,

I suspect that this issue falls in a grey area between nanotime and tibble.

I cannot get a tibble to print, but I can get a data.frame to print:

> tmp <- tibble::tibble(x = nanotime::nanotime(Sys.time()))
> tmp
Error in callNextMethod() : 
  error in evaluating a 'primitive' next method: Error in FUN(x = x, i = i) : could not find function "FUN"
> as.data.frame(tmp)
                                    x
1 2017-07-21T19:57:13.419241000+00:00

Is there something that nanotime can provide to be more "tibble-friendly"? My cursory google-search did not turn up anything.

As well, I have a problem to display in the RStudio viewer, but I'll take this one issue at a time ;)

FWIW, I am using CRAN versions of tibble and nanotime on R 3.4.1, using Linux.

Export `nanoperiod` C interface for other packages to use

Duration since yesterday's midnight in some timezone

Hello nanotime authors.
When using nanotime in conjunction with data.table one often needs to extract the time of day portion of a nanotime to compute quantities across dates (say how many entries did a process record on average, per date, between 08:00 and 09:00 in 2020 in some TZ).
I understand that nanoduration is not meant to be a time class but it is fairly close.

How about a function that would return the duration of a nanotime since previous date midnight (hence expressed in a given timezone) ?
Going back to string is pretty costly and could likely be instant

library(nanotime)         
library(data.table)
options(nanotimeFormat = "%Y-%m-%dD%H:%M:%E3S%z")
options(nanotimeTz = "Europe/Paris")
set.seed(1L)
start_ts <- nanotime("2020-08-03 UTC")                                                                                
one_hour <- 3600e9                                                                                                     
N <- 1e7                                                                                                               
DT <- data.table(ts = start_ts + runif(N, max = 12 * one_hour))                                                        
# takes                                                                                                                
# - 3 secs on my laptop for "%H:00:00"                                                                                 
# - 9 secs on my laptop for "%H:%M:%E3S"                                                                               
system.time({                                                                                                          
    DT[, time_str := format(ts, format = "%H:%M:%E3S")]                                                                
})                                                                                                                     
# pretty much instantaneous but is the offset since midnight in UTC                                                                                        
system.time({                                                                                                          
    DT[, time_nd := as.nanoduration(as.integer64(ts) %% 86400e9)]                                                      
})                                                                                                                     

Is there already an efficient way to do this that I am missing ?
If not and if you do not have strong opinion against it I could provide a PR for this.

incorrect parsing of timestamp?

Hello there!

I apologize in advance if this is a silly question, but I was not able to use nanotime with my timestamps.

• R version 3.5.1 (2018-07-02)
• Platform: x86_64-ubuntu18-linux-gnu (64-bit)

> nanotime('2018-01-01 15:30:00.000', format ="%Y%m%d %H:%M:%S",  tz ="GMT")
Error in RcppCCTZ::parseDouble(x, fmt = format, tz = tz) : 
  Parse error on 2018-01-01 15:30:00.000
> nanotime('2018-01-01 15:30:00.000', format ="%Y%m%d %H:%M:%S",  tz ="UTC")
Error in RcppCCTZ::parseDouble(x, fmt = format, tz = tz) : 
  Parse error on 2018-01-01 15:30:00.000
> nanotime('2018-01-01 15:30:00.000', tz ="UTC")
Error in RcppCCTZ::parseDouble(x, fmt = format, tz = tz) : 
  Parse error on 2018-01-01 15:30:00.000

Am I calling nanotime incorrectly?
Thanks!

Allow dropping of fractional time parts and relax a bit parsing syntax

It would be nice to allow, amongst others:

nanotime("2019-09-10")   # like POSIXct
nanotime("2019-09-10 America/New_York") # often more natural to specify a time zone than an offset
nanotime("2019-09-10 20:00:00") # forgive the missing 'T'!

In output, the fractional time and parts could be omitted when no element of the vector to be printed has fractional part. Again this would be like POSIXct.

Finally, in output, the default prints out the time zone offset, but the timezone abbreviation is shorter and would be the same as POSIXct:

[1] 2019-09-10T20:00:00.000000001 CST

This saves one to two characters per printout.

Finally, besides the missing 'T', there could also be forgiveness for using '/' or '' instead of '-' as date separator?

create and export as.nanotime

It might be sometimes useful to operate on arbitrary type objects. For example if fread's colClasses arg will be supplied with "nanotime" type, then we could flexibly dispatch to arbitrary class by using

convert = function(x, type) {
  as.call(c(as.name(paste("as", type, sep=".")), as.name("x")))
}
convert(x, type="nanotime")
#as.nanotime(x)

as.* functions are kind of standard in R.

Use new RcppCCTZ header

See eddelbuettel/rcppcctz#38

Incorrect parsing for negative `nanoperiod`

> as.nanoperiod("0d/-00:00:01")
[1] 0m0d/-00:00:01
> as.nanoperiod("00:00:01")
[1] 0m0d/00:00:01
> as.nanoperiod("-00:00:01")
Error in period_from_string_impl(x) : cannot parse nanoperiod

This last one should actually parse exactly like a nanoduration:

> as.nanoduration("-00:00:01")
[1] -00:00:01

nanotime and ggplot2?

Hi @eddelbuettel, thanks for this amazing package!

Nanotime support is clearly what we need at the moment. I am wondering if your package is compatible with ggplot2.

Can I create charts that have an x-axis with timestamps up to nanoseconds? Are there any particular issues?

Thanks!

Comparing nanotime with a POSIXct

Hello,
in addition to #52 can I suggest to add comparator for POSIXct ?

nanotime(Sys.time()) > (Sys.time() - 10)
[1] FALSE
Warning messages:
1: In if (class(e2) == "nanotime") { :
the condition has length > 1 and only the first element will be used
2: In eval(call, callEnv) :
Incompatible methods (">.integer64", "Ops.POSIXt") for ">"

Issue with nanotime min/which.min functions

Hello, there is an ungoing issue with integer64 that I think affect nanotime

min(nanotime(as.integer64(7300)), nanotime(as.integer64(7400)))
[1] "1970-01-01T01:00:00.000007400"
which.min(nanotime(c(as.integer64(7400)), as.integer64(7300)))
[1] 1

It looks very much like the same thing and referred to in this SO question

nanotime 0.3.3 release

Life got busy and somehow away from work here, so I never made a release after the 0.3.2 release last fall.

I have been running 0.3.2.3 (== current 'master') everywhere and have no issue so I am thinking about labeling this as 0.3.3 and sending it to CRAN. Any objections / issues / concerns / things I forgot?

Tagging @lsilvest, and also @statquant re PR #88 which would then (belatedly) make it to CRAN.

Example plots

Guys, could someone please post some simple plot examples on how to use nanotime to display nanosec timestamps on x axis. Thank you!

integer64 interaction

Error in if (n2%%n1) warning("length(e2) not a multiple length(e1)") :
argument is not interpretable as logical

Ideally when empty array is passed to nanotime, like other functions in R, it should return empty array of type nanotime only