GithubHelp home page GithubHelp logo

sas7bdat-js's Introduction

sas7bdat-js Build Status

Read SAS files in JavaScript. Because you always wanted to do that, right?

Ported from the sas7bdat Python package. All functionality should be the same, except sas7bdat-js supports fewer compression methods.

Install

npm install sas7bdat

Use

First load the module:

const SAS7BDAT = require('sas7bdat');

SAS7BDAT.createReadStream returns a stream that emits individual rows, one at a time:

const stream = SAS7BDAT.createReadStream('test.sas7bdat');
stream.on('data', row => console.log(row));
stream.on('end', () => console.log('Done!'));
stream.on('error', err => console.log(err));

SAS7BDAT.parse returns a promise that resolves to an array containing all the rows:

SAS7BDAT.parse('test.sas7bdat')
    .then(rows => console.log(rows))
    .catch(err => console.log(err));

SAS7BDAT.toCsv converts a sas7bdat file to CSV format:

SAS7BDAT.toCsv('test.sas7bdat', 'test.csv', {
        sasOptions: {
            // See below, same as second argument to other SAS7BDAT functions
        }
        csvOptions: {
            // These are passed to http://csv.adaltas.com/stringify/
            quotedEmpty: false,
            quotedString: true
        }
    })
    .then(rows => console.log('Done!'))
    .catch(err => console.log(err));

Options

Pass an options object as the second parameter to SAS7BDAT.createReadStream or SAS7BDAT.parse:

const options = {};

const stream = SAS7BDAT.createReadStream('test.sas7bdat', options);

options.rowFormat

A string equal to 'array' (default) or 'object' which controls whether rows come back as arrays:

['Col1', 'Col2', 'Col3']
[1, 'a', 'whatever']
[2, 'b', 'whatever']
...

or objects:

{Col1: 1, Col2: 'a', Col3: 'whatever'}
{Col1: 2, Col2: 'b', Col3: 'whatever'}
...

options.dateFormatter

This lets you customize the output format of date/time variables. For example, the default is:

options.dateFormatter = (d, outputFormat) => {
    if (outputFormat === 'date') {
        return d.toISOString().slice(0, 10);
    }
    if (outputFormat === 'time') {
        return d.toISOString().slice(11, 23);
    }
    return d.toISOString();
}

The two arguments to the callback function are d (a JavaScript Date object) and outputFormat (a string containing 'date', 'time', or 'datetime').

options.skipHeader

By default, the first row emitted contains column names, like:

['Col1', 'Col2', 'Col3']
[1, 'a', 'whatever']
[2, 'b', 'whatever']
...

When options.skipHeader is true, the row containing column names will be skipped:

[1, 'a', 'whatever']
[2, 'b', 'whatever']
...

If options.rowFormat is 'object', then options.skipHeader has no effect.

options.extraDateFormatStrings, options.extraTimeFormatStrings, and options.extraDatetimeFormatStrings

Date/time/datetime columns are identified by a string attatched to them, like "YYMMDD" means a date column and "DATETIME" means a datetime column. The default identifiers used here are:

  • date: ['YYMMDD', 'MMDDYY', 'DDMMYY', 'DATE', 'JULIAN', 'MONYY', 'WEEKDATE']
  • time: ['TIME']
  • datetime: ['DATETIME']

Some files might have some other strings used to identify these columns, in which case you can use options.extraDateFormatStrings, options.extraTimeFormatStrings, and options.extraDatetimeFormatStrings as needed. For example:

// Add another date format string options.extraDateFormatStrings = 'whatever';

// Add more than one time format string by using an array options.extraTimeFormatStrings = ['foo', 'bar'];

options.encoding

A string containing the character encoding of strings in the file, default is 'utf8'. Other available options are whatever is supported by Node.js, currently:

  • 'ascii' - for 7-bit ASCII data only. This encoding method is very fast and will strip the high bit if set.
  • 'utf8' - Multibyte encoded Unicode characters. Many web pages and other document formats use UTF-8.
  • 'utf16le' - 2 or 4 bytes, little-endian encoded Unicode characters. Surrogate pairs (U+10000 to U+10FFFF) are supported.
  • 'ucs2' - Alias of 'utf16le'.
  • 'base64' - Base64 string encoding. When creating a buffer from a string, this encoding will also correctly accept "URL * `Filename Safe Alphabet" as specified in RFC 4648, Section 5.
  • 'binary' - A way of encoding the buffer into a one-byte (latin-1) encoded string. The string 'latin-1' is not supported. Instead, pass 'binary' to use 'latin-1' encoding.
  • 'hex' - Encode each byte as two hexadecimal characters.

options.alignCorrection

Boolean, default true. I'm not totally sure what this does, it came along with the port from Python. If it's needed, it'll hopefully produce an error message telling you that.

options.logLevel

A string containing one of the following options:

  • 'critical' - Log important error messages to the console.
  • 'error' - Everything above, plus messages about less important errors.
  • 'warning' - Everything above, plus messages about even less important warnings.
  • 'info' - Everything above, plus messages about normal behavior.
  • 'debug' - Everything above, plus more verbose debugging information.

The default value is 'warning' which will usually result in no logged messages. This is meant to by like Python's logging module.

Tests

npm test

Similar open source projects for other languages

sas7bdat-js's People

Contributors

dumbmatter avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

sas7bdat-js's Issues

Doensn't install for version 12.13.1 of node.js

It seems the package fs-ext it's not building for version 12.x for current version

../fs-ext.cc: In function ‘Nan::NAN_METHOD_RETURN_TYPE Fcntl(Nan::NAN_METHOD_ARGS_TYPE)’:
../fs-ext.cc:369:32: error: no matching function for call to ‘v8::Value::Int32Value()’
int fd = info[0]->Int32Value();
^
In file included from /home/alxolr/.cache/node-gyp/12.13.1/include/node/node.h:63:0,
from ../fs-ext.cc:20:
/home/alxolr/.cache/node-gyp/12.13.1/include/node/v8.h:2613:40: note: candidate: v8::Maybe v8::Value::Int32Value(v8::Localv8::Context) const
V8_WARN_UNUSED_RESULT Maybe<int32_t> Int32Value(Local context) const;
^~~~~~~~~~
/home/alxolr/.cache/node-gyp/12.13.1/include/node/v8.h:2613:40: note: candidate expects 1 argument, 0 provided
../fs-ext.cc:370:33: error: no matching function for call to ‘v8::Value::Int32Value()’
int cmd = info[1]->Int32Value();

It seems it fails for fs-ext module, ideea to upgrade the fs-ext module version.

Reading file with '‹' character.

When trying to read a file with the '‹' character it converts it to something like:

On Windows: (a new line, followed by) ï¿ ½"
On Mac: �

Any idea if I'm doing something wrong? I tried utf-8, ascii, ucs2 and binary encoding.

Big endian not supported

In the struct_unpack function, an error is thrown if the endianness is BE. I tried swapping the endianness of the buffer as suggested in the corresponding comment (using the native buf.swap64()), but didn't have much success — the script just exits after printing one row as an empty array.

Do you have a sense of what it would take to support big endian encodings? fwiw, the python version of this library seems to work fine, though I'd prefer to leverage the JS implementation here.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.