jsonlite and dates about jsonlite

[jeroen]

The json format unfortunately does not natively support dates or timestamps. BSON does, but that is another story. jsonlite gives you several options on how to encode dates and times:

> now <- Sys.time()
> toJSON(now)
[1] "[ \"2014-02-23 14:14:40\" ]"
> toJSON(now, POSIXt="ISO8601")
[1] "[ \"2014-02-23T14:14:40\" ]"
> toJSON(now, POSIXt="epoch")
[1] "[ 1393193680926 ]"
> toJSON(now, POSIXt="mongo")
[1] "[ { \"$date\" : 1393193680926 } ]"

And for Date:

> today <- as.Date(now)
> toJSON(today)
[1] "[ \"2014-02-23\" ]"
> toJSON(today, Date="epoch")
[1] "[ 16124 ]"

The fromJSON parser will not automatically try to detect dates. This excel like hacking is too dangerous and can lead to unexpected behavior. However the base R functions as.Date and as.POSIXct are actually very good at coersing strings into dates and timestamps:

as.Date(fromJSON(toJSON(today)))
as.POSIXct(fromJSON(toJSON(now)))

[jetpad]

Thought I'd post this here for anyone looking to make that final step of getting the json'd date into an R date format. http://stackoverflow.com/questions/15838548/parsing-iso8601-date-and-time-format-in-r

[johnwebbcole]

I agree with the notion that excel like date coercion is pretty dangerous, except for the POSIXt="mongo" case. There you have a clear type definition that you have a date object and what the value is. It would be very nice for a fromJSON call to allow a POSIXt="mongo" and handle the special mongodb dates.

[jeroen]

@johnwebbcole that sounds reasonable. I tried to implement this. Would you mind testing if this is doing what you would expect?

x <- Sys.time() + 1:3
fromJSON(toJSON(x, POSIX="mongo")) 

[johnwebbcole]

@jeroenooms I tried loading from github:

> dev_mode(on=T)
Dev mode: ON
d> install_github("jsonlite")
Installing github repo jsonlite/master from hadley
Downloading master.zip from https://github.com/hadley/jsonlite/archive/master.zip
Error in (function (url, name = NULL, subdir = NULL, config = list(),  : 
  client error: (404) Not Found

How should I test that?

Looking at it, I would think I need to pass in an option to fromJSON to let it know that it's mongo flavor.

query <- fromJSON('{
    "ts": {
            "$gt": { "$date": 1394598852736 }
        }
    }', POSIX="mongo")

[rubenarslan]

You would do devtools::install_github("jeroenooms/jsonlite”), otherwise
it defaults to hadley’s repository.

On 16 Apr 2014, at 16:49, johnwebbcole [email protected] wrote:

@jeroenooms I tried loading from github:

dev_mode(on=T)
Dev mode: ON
d> install_github("jsonlite")
Installing github repo jsonlite/master from hadley
Downloading master.zip from https://github.com/hadley/jsonlite/archive/master.zip
Error in (function (url, name = NULL, subdir = NULL, config = list(), :
client error: (404) Not Found
How should I test that?

Looking at it, I would think I need to pass in an option to fromJSON to let it know that it's mongo flavor.

query <- fromJSON('{
"ts": {
"$gt": { "$date": 1394598852736 }
}
}', POSIX="mongo")
—
Reply to this email directly or view it on GitHub.

[johnwebbcole]

@rubenarslan Thanks, that worked.

Ok running this:

x <- Sys.time() + 1:3
print(toJSON(x, POSIX="mongo"))
print(toJSON(x))

produces

[1] "[ { \"$date\" : 1397660463360 }, { \"$date\" : 1397660464360 }, { \"$date\" : 1397660465360 } ]"
[1] "[ \"2014-04-16 10:01:03\", \"2014-04-16 10:01:04\", \"2014-04-16 10:01:05\" ]"

This seems ok to me. However

query <- fromJSON('{
    "ts": {
            "$gt": { "$date": 1397661867620 }
        }
    }')
print(query)
print(toJSON(query, POSIX="mongo"))
print(toJSON(query))

produces

         $date
1 1.394599e+12
[1] "[ { \"$gt\" : { \"$date\" : { \"ts\" : [ 1397661867620 ] } } } ]"
[1] "[ { \"$gt\" : { \"$date\" : { \"ts\" : [ 1397661867620 ] } } } ]"

I expected

ts : 3   
        $gt : 9      1397661867620
[1] "[ { \"$ts\" : { \"$gt\" : { \"$date\" : 1397661867620  } } } ]"
[1] "[ { \"$ts\" : { \"$gt\" : \"2014-04-16 10:24:27\" } } ]"

[jeroen]

Yeah it only works for mongo style data which should appear within a json array

myjson <- '[
   { "ts": { "$gt": { "$date": 1397661867620 } } },
   { "ts": { "$gt": { "$date": 1397661867621 } } },
   { "ts": { "$gt": { "$date": 1397661867622 } } }
 ]'

 fromJSON(myjson)

[jeroen]

I pushed another update. Can you try again?

[johnwebbcole]

query <- fromJSON('{
    "ts": {
            "$gt": { "$date": 1394598852736 }
        }
    }')

print(query)
print(toJSON(query, POSIX="mongo"))
print(toJSON(query))

produces

$ts
$ts$`$gt`
[1] "2014-03-11 23:34:12 CDT"


[1] "{ \"ts\" : { \"$gt\" : [ { \"$date\" : 1394598852736 } ] } }"
[1] "{ \"ts\" : { \"$gt\" : [ \"2014-03-11 23:34:12\" ] } }"

which is pretty close to what I expect, except for $gt being inside an array.

[jeroen]

That is actually expected because R does not have scalars, so the $gt value is a vector. You can disable this using either the unbox function, or the auto_unbox argument.