read_sas cannot load my .sas7bcat file about haven

[evanmiller]

If you can point me to the sas7bcat file I'll take a look.

Unfortunately all of the sas files I work with are HIPPA protected data located on our servers, so I'm not able to do that. In order to have data transfer over to sas with the value labels, do I need to reference the catalog file, or am I doing something wrong since when I only reference the sas7bdat file nothing includes the labels?

I apologize for not having a reproducible example; I can close the issue since I am unable to provide one if that would be appropriate (it's why I closed it originally).

[evanmiller]

Thanks for the update. @hadley might be able to shed light on the issue of importing into a data frame that already has value labels applied.

[hadley]

Yes, you need the catalog file if you want labels. Could you try creating an example with dummy data?

[cullenjd]

I'm having the same problem. When I reference the catalog file using read_sas, I get:

"Error: Failed to parse K:...\formats.sas7bcat: Invalid file, or file has unsupported features."

My data contains personal health information and I don't have SAS to create dummy data.

I can send the catalog file if that would be any benefit.

[evanmiller]

@cullenjd Yes, a sample catalog file that demonstrates the problem would help.

[cullenjd]

Here's a link to the sas7bcat file:

https://www.dropbox.com/s/o6xwioy72xs4g3i/formats.sas7bcat?dl=0

[evanmiller]

Thanks @cullenjd. I believe I've tracked down the issue, but without an accompanying data set I cannot fully test. I've committed code to ReadStat here:

WizardMac/ReadStat@c732495

You can test this code with the instructions here:

https://github.com/hadley/haven#updating-readstat

Let me know if correctly reads and applies your value labels. @hadley please re-open this issue until @cullenjd confirms it is resolved.

For posterity: It looks like catalog files can contain both an 8-character short name and a 32-character long name for each variable format (e.g. CLAVICLE and CLAVICLE_FORMAT). The presence of the long name appears to be determined by a bit-field. Without more sample data files I'm not sure whether to return the short name or the long name to the client code, i.e. whether the data file refers to formats using their short name or long name.

[cullenjd]

I can't get the code to update ReadStat to work. I think it might be something to do with the path. I get the following error:

file.copy(src, "src/", overwrite = TRUE)
Error in file.copy(src, "src/", overwrite = TRUE) :
more 'from' files than 'to' files

[evanmiller]

Should be fixed by PR #114.

[cullenjd]

I've tried updating ReadStat using the code given on Hadley's Haven page but still get the same error. Is there a way to explicitly specify the 'to' directory? I'm not sure which src directory is referenced with the "src/" parameter.

[evanmiller]

Hi @cullenjd -- for now, just try this:

devtools::install_github("evanmiller/haven")

That will install my fork of haven, which has the latest ReadStat code.

[evanmiller]

If that doesn't fix it, I am working on another bugfix here: WizardMac/ReadStat#38

[cullenjd]

Hi @evanmiller - your fork installed fine and I was able to pass the name of the catalog without error, however, no value labels from the catalog file were applied. For example;

table(mydata$gender)

returned 0 and 1 not Male and Female as specified in the catalog file.

[evanmiller]

Hi @cullenjd,

Please try the latest hadley/haven master, which includes all of the latest fixes. If you still experience the same problem, please file a new issue against haven.

[evanmiller]

@cullenjd I suspect the issue has to do with short vs long names of formats mentioned earlier (e.g. CLAVICLE vs CLAVICLE_FORMAT). Right now I am returning the short format to haven but I'm not sure if that's correct. Let's continue the discussion over here: #121

[BenWiseman]

I'm still having this issue with
http://www.cdc.gov/brfss/annual_data/2014/files/formats14.sas7bcat

I can read in the data from the respective .sas7bdat

Error: Failed to parse C:\Users\WisemBH\AppData\Local\Temp\Rtmpi2Z4F3\file14f8582e3cc6: Invalid file, or file has unsupported features.

[evanmiller]

Hi @BenWiseman, are you using the development version of haven?

devtools::install_github("hadley/haven")

[Z-02]

@evanmiller
I use the devtools to install but still encounter the same problem

Error in df_parse_sas_file(spec_data, spec_cat, encoding = encoding, cols_only = cols_only) : Failed to parse /home/s/Documents/health/2005.sas7bdat: File has an unsupported character set.

[evanmiller]

@2YC Can you share the sas7bdat file that you're working with?

[cimentadaj]

Hi @evanmiller, I'm still encountering the same problem.

It's coming from a publicly available dataset from the PISA study.

Data can be downloaded from here: http://vs-web-fs-1.oecd.org/pisa/PUF_SAS_COMBINED_CMB_STU_QQQ.zip

The download can take some time (~ 5-10 mins) as the dataset is about ~ 2GB

library(haven)
pisa_2015 <- read_sas("cy6_ms_cmb_stu_qqq.sas7bdat", "CY6_MS_CMB_STU_QQQ.sas7bdat.format.sas")

Error in df_parse_sas_file(spec_data, spec_cat, encoding = encoding, cols_only = cols_only) : 
  Failed to parse /dir//CY6_MS_CMB_STU_QQQ.sas7bdat.format.sas: Invalid file, or file has unsupported features.

I also tried changing the extension from sas7bdat.format.sas to sas7bcat and the result is the same. Do note that: pisa_2015 <- read_sas("cy6_ms_cmb_stu_qqq.sas7bdat") reads the data correctly.

Any ideas?