pradeep-mishra / csvjson Goto Github PK

Hello,
Is there a way to filter which fields/keys to export when converting json to csv?

I'm using csvjson in a React project.
When i try to use toColumnArray function with a CSV with a comma inside a string (eg "foo,bar"), i get the following error:
index.js:34 Uncaught TypeError: Cannot read property 'push' of undefined.
It seems the algorythim identifies comma as CSV separator instead part of the string

./node_modules/csvjson/src/stream.js:17:15-32 - Error: Module not found: Error: Can't resolve 'stream' in .../node_modules/csvjson/src'
Did you mean './stream'?
Requests that should resolve in the current directory need to start with './'.
Requests that start with a name are treated as module requests and resolve within module directories

If changing the source code is not an option there is also a resolve options called 'preferRelative' which tries to resolve these kind of requests in the current directory too.

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
- add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
- install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
resolve.fallback: { "stream": false }

using angular 14 and node 18.

version 4.1.2

var data = "
A;B;C\n
;\n
"
csvjson.toObject(data, {delimiter:';'})

result:

TypeError: Cannot read property 'trim' of undefined
       at _trimQuote (./node_modules/csvjson/index.js:397:15)

expected:
not sure...

• throw your own descriptive parse exception?
• add callback / promise interface to pass back errors nicely?

Please allow the character used as delimiter between CSV fields to be configured. Common characters are not only comma (,) but also semi-colon (;) and tab (/t)

As u know JSON sometimes asks us strict rules which are that we have to use double quotations in JSON. In your examples, it looks fine but it generates only single quotations or no quotations. Would you check it once? I've tried it several times to get exact same result with your examples. I couldn't make it. If I'm doing wrong, leave a comment. Thx.

Since node.js rely on tasks being short, it would be good if csvjson offered an event-driven incremental parsing mode (similar to https://github.com/koles/ya-csv) in order to support very large CSV files (but also because it's just very convenient in the context of node.js).

For example, it would be great to have

id,title,contents[].name,contents[].title,contents[].type

{"id": "XXX", "title": "YYY", "contents": [ { "name": "AAA", "title": "BBB", "type": "CCC"} ]

Similarly, something like "contents[1].title" and "contents[2].title" could be used for multiple objects within an array

When used in streaming mode with a reasonable size file, it seems to choke at regular points - I'm guessing its at chunk boundaries. Each time it stops part way through a record and inserts a closing and opening bracket ][ and then continues from the next record.

The csv file I'm using has about 5000 lines and the breaks occur at roughly every 560 lines.

Line lengths are variable but less than 150 characters.

There are 12 columns in the data.

Not all CSV files have its header/schema in the same file as the CSV data itself. Please allow use of header/schema information from a different source, such a separate file, an object, or string. (either is fine for my immediate needs)

In the example: convert csv data to json object, a comma is missing after the 'delimeter' key-value pair.
The code doesn't run due to this.

I am trying to use csvjson in my Angular application. I did npm i csvjson --save

But I am getting the following error:

ERROR in ./node_modules/csvjson/src/stream.js Module not found: Error: Can't resolve 'stream' in 'D:\Projects\MyProject\node_modules\csvjson\src'

When I looked into this src folder in node_modules\csvjson\src -> I could find all 3 files app.js, helper.js and stream.js

We are really stuck on this. Please help...

If there are line breaks in a double quoted .csv field the parser splits the output on the line breaks as well as the commas resulting in really messed up objects.

e.g. "Invest in Tinder, Buzzfeed or Airbnb" will get detected as ""Invest in Tinder," and "Buzzfeed or Airbnb"

When using in the browser like this:

const pcsToImport = csvjson.toObject(data).output

I get this error:

Uncaught TypeError: fs.existsSync is not a function

If you don't want to make it work in the browser please state this in the readme.

JSON to CSV creates header rows with object and array notation. Great. I need to get rid of them.
The objectDenote and arrayDenote are optional and default to their respective defaults. But what is the option to get rid of them without modifying the file directly? No documentation for what the options are.

This is a duplicate of unsolved #12.

When running this code:

const csvjson = require('csvjson');

const options = {
  delimiter: ',',
  quote: true
};

const data = `one,two
"some data", "some ""other"" data"`;

const obj = csvjson.toObject(data, options);

console.log(obj);

the output is:

[ { one: 'some data', two: 'data' } ]

and should be:

[ { one: 'some data', two: 'some "other" data' } ]

This also occurs with the quote option set to '"'.

Hoping to do what the title says. I am pulling in answers from a form, and then turning them from JSON into CSV. Sometimes the answers are arrays, sometimes not (multiple checkboxes). When that happens it adds the non-arrays and arrays as separate headers (for the same question).

For example, "question_2" and "question_2[]" become two headers, but I would just like "question_2" as the header. I changed the source code locally, and it worked when I removed the brackets from the string, but setting them as empty strings does not work.

@pradeep-mishra

When I'm converting JSON to CSV using the options:
var options = { headers : "key", delimiter : ";" }

One of the keys in my object is "Asset Sub-Type". The full CSV is rendered fine, but the header for the field associated with "Asset Sub-Type" is renamed to just "Type".

When parsing CSV, escaped double quotes cause error.
TypeError: Cannot read property '0' of null at node_modules/csvjson/index.js:74:55 at Array.forEach (native) at node_modules/csvjson/index.js:73:21 at Array.forEach (native) at Object.toObject (node_modules/csvjson/index.js:67:13)

Double quotes in CSV are escaped with double quotes, so it looks looks like this:
"this string contains ""double quotes"""
But using toObject to parse CSV files which includes such escaped quotes seems to invoke this error.

How do I disable quotes for bool column value? And conditionally leave it on for string column values?

Actually i am getting CSV (here Tab-separated) data from client through HTTP on server side, now i want to convert that data to json. How can i?

Example:
JS =>

let sheet = (await axios.get(config.docs.technicalAnalysisIndicators)).data;
let result = csvjson.toObject(sheet, { encoding: 'utf8', delimiter: ',', quote: /['",]+/ });
return result;

CSV =>

func,name,shortname,caption,variable,predefined,min,max,step,values,type,category,groupCaption,description,notes
generic,Validation Size,generic,Validation Size,validation_size,0.2,0.1,0.5,0.1,,float,Machine Learning,It's a set of data used to access the strength and utility of a predictive relationship,,
generic,Seed,generic,Seed,seed,7,1,20,1,,integer,Machine Learning,description,,
generic,n Fold,generic,n Fold,num_fold,10,2,20,1,,integer,Machine Learning,description,,
generic,Scoring,generic,Scoring,scoring,accuracy,,,,"accuracy,recall",string,Machine Learning,"A ""score function"" or ""scoring rule"" measures the accuracy of probabilistic predictions",,Per ora l'utente non può selezionare nulla ma in futuro potremmo aggiungere l'opzione di scelta
LogisticRegression,Logistic Regression,LR,Regularization,lr_regularization,l2,,,,"l1, l2",string,Machine Learning,It's the go-to method for binary classification problems,penalty,
LogisticRegression,Logistic Regression,LR,C,lr_c,1.0,0.1,10.0,0.1,,float,Machine Learning,It's the go-to method for binary classification problems,C,
LogisticRegression,Logistic Regression,LR,Solver,lr_solver,liblinear,,,,"newton-cg, lbfgs, liblinear",string,Machine Learning,It's the go-to method for binary classification problems,solver,"Se seleziona lr_regularization = L1 allora il solver = Linear, se seleziona lr_regularization = L2 allora il solver = Linear, Newton-CG, L-BFGS"
LinearDiscriminantAnalysis,Linear Discriminant Analysis,LDA,Solver,lda_solver,svd,,,,"cvd, lsqr, eingen",string,Machine Learning,It's the go-to linear method for multiclass classification problems,solver,

Result is wrong, from "values" column everything gets messy =>

{
        "func": "generic",
        "name": "Scoring",
        "shortname": "generic",
        "caption": "Scoring",
        "variable": "scoring",
        "predefined": "accuracy",
        "min": "",
        "max": "",
        "step": "",
        "values": "\"accuracy",
        "type": "recall\"",
        "category": "string",
        "groupCaption": "Machine Learning",
        "description": "A \"\"score function\"\" or \"\"scoring rule\"\" measures the accuracy of probabilistic predictions",
        "notes": ""
    },