Missing values about readstat

[evanmiller]

Missing strings are the zero-length string but the treatment of missing numeric values is currently inconsistent. The DTA parser returns NULL but the others return a NaN. The SAS parser returns the data representation in the file (a NaN, but it might be a tagged NaN?) while the others return a system NaN.

[hadley]

What about integers? Also seems a bit dangerous to encode a missing strings as zero-length

[evanmiller]

Only the DTA parser returns integers which is why it uses the NULL convention rather than NaNs.

I think using NULL in all cases might make sense. READSTAT_TYPE_MISSING loses information about the underlying type, which puts more of a burden on the client to keep track of the column types.

[hadley]

Using NULL seems reasonable to me. Another option would be to provide readstat_value_is_missing() or similar so the representation could be changed in the future.

[evanmiller]

In the meantime we could do both. A new readstat_value_t type could be a possibly NULL pointer, which we could change to a struct later.

[evanmiller]

Ok, the callbacks now all receive NULL for missing numeric values:

737b816

String are trickier since I believe none of these file formats distinguish between zero-length strings and missing strings. (RData might be an exception.)

[hadley]

Oh interesting. In that case, leaving as empty strings seems reasonable to me.

[evanmiller]

Closing