nickewing / line-reader Goto Github PK

Hello, it seems that the example using promises is not working as expected. Using the code presented on the README file, the promise is never resolved. The problem looks to happen because four parameters are passed to lineReader.eachLine function.

If a pass only the following params:

lineReader.eachLine(filename, options, function(err){ ...})
the promise is resolved. Can you please check the code on the readme file?

In the following, the file is never closed when file.txt size is >= 1024 (using default buffer size of 1024).

var lineReader = require('line-reader')
var fs = require('fs')

var lines = ""
//var N = 511 // File is closed
var N = 512 // File is not closed
//var N = 513 // File is not closed
for (var i=0;i<N;i++) {
    lines = lines + " " + "\n"
}
fs.writeFileSync("file.txt", lines)

var i = 0
lineReader.eachLine('file.txt', function(line, last) {
  console.log(i + " " + line)
  if (i == 1) return false
  i = i+1
  return true
})

If I modify line_reader.js as below (and replace finish() with finish(reader) elsewhere)

   function finish(reader) {
      reader.close()
...

the file is closed. (I have not run extensive tests on the fix.)

I need to stop reading a file once it gets to certain line & just stop whatever else is going on in the eachLine part. How can I do that? I've read through the docs and searched on the internet but found nothing that works. Please tell me it's possible to do

Alternative to arguments of type String in open(), it would be really handy to support arguments of type Stream as well.

Hello,

I have this case where I need to read lines from several flat files, and I loop over them.
I would like to be able to wait the processing of all files before I go on.

Of course, sync reading is needed but I can't get it to work.
In the method below, controlFiles mean reading the 1st line of all files.
I want this step to be completed before I move anything.

try {
    // Controlling all files => Reading the 1st line of each 
    await controlFiles.controlMediapostFiles();

    // Moving files to files_out directory
    await moveFiles.moveFilesToDir(filePath, files_out, files_done, false);
} catch (error) {
    // what ever
}

No here is how I TRY to get this done sync :

controlMediapostFiles: async function () {
    return new Promise(async (resolve, reject) => {
	try {

	    let numberOfFiles = 0;

            // FOR EACH FILES IN DIRECTORY
	    await fs.readdirSync(files_out).forEach(async function (file) {
				
	    	numberOfFiles++;

	    	let lineNumber = 0;

            	// TRY AND WAIT FOR FINISHED EVENT (EOF or return false)
	    	await lineReader.eachLine(files_out + path.sep + file, await function(line) {
	        	lineNumber++;
	        	if (lineNumber === 1) {
		    		// somthing
		    		console.log(line)
	        	} else {
		    		return false;
	        	}
	    	});
    	    }
	});

	if (numberOfFiles === 0){
		reject({done:false, step: 'CONTROLE', message: 'Aucun fichier fourni'});
	}


	resolve({done:true, message: 'Control OK'});

   } catch (error) {
	reject(error);
   }
});
}

No matter how I try, the operations (including moving files and logging) in the method AFTER are done before the reading of the file.

Note that I can't use the method here after, since it's a for each files loop:

lineReader.eachLine(bufferStream, function(line) {
    //do stuff
}, async function finished (err) {
     // called when reading is finished
});

Calling eachLine on an empty file executes the error call back function (errCb) with an undefined error. I think the correct behaviour would be to not execute any of the call-back functions in the case of an empty file since there are no lines and there is no error.

Hello.I just use the demo after installed the module,but I got the following error:
[10745:1021/162854:ERROR:nw_shell.cc(336)] TypeError: Cannot read property 'length' of undefined

How to save output to file. Please help

The following code:

var linereader = require('line-reader');

linereader.eachLine('test_data', function(line, last) {
    console.log(line);
});

with test_data:

aa
bb
cc
dd

only returns dd in the console.

bug.txt (one \n after test; 10 bytes total)

test
lines

bug.js:

var lineReader = require('line_reader');

lineReader.eachLine('bug.txt', function(line) { console.log(line); }, undefined, undefined, 5);
lineReader.eachLine('bug.txt', function(line) { console.log(line); }, undefined, undefined, 11);

Outputs:

test
lines
test

lines

Hi,

The version on npm does not work, however the code on master does.
I am looking at adding this library as a dependency to my project, any chance the npm package could be updated?

I want get the err when open the file.

Help... I have the right file.

there is a bug for version 0.2.4.
0.2.3 is ok.
stack:
node_modules/line-reader/lib/line_reader.js:130
setImmediate(readNext);
ReferenceError: setImmediate is not defined
at newRead (node_modules/line-reader/lib/line_reader.js:130:11)
at then.finalFn (node_modules/line-reader/lib/line_reader.js:162:7)
at node_modules/line-reader/lib/line_reader.js:112:9
at node_modules/line-reader/lib/line_reader.js:56:13
at Object.wrapper as oncomplete

I have no idea how it stops working when I load my module in unit tests while it works directly. I'm not spying fa or path modules.

I'm using jasmine for unit testing

console.log(filePath);//working
console.log(fs.readFileSync(filePath).toString());//working
lineReader.eachLine(filePath, function(line, last) {
    console.log(line);//not reaching here
});

When reading a large file with many lines, the maximum call stack size exceeded error occur. I was trying something like the following:

lineReader.eachLine('file.txt', function(line, last) {
...

}

(node) warning: Recursive process.nextTick detected. This will break in the next version of node. Please use setImmediate for recursive deferral.

RangeError: Maximum call stack size exceeded

Is there typescript support?

Sorry if my question is dumb, but I believe the whole point of reading a file line-by-line is not having to load it entirely into RAM in order to read it. So if I have, for example, an 100 MB file, will it be loaded into RAM before the code can read it?

IMHO, hardcoded \n should be replaced with os.EOL. Further, it would be really handy if line-separator could automatically make a decision between \n and \r\n by looking at the very first line.

The eachLine function is documented to return an object with one property, then. It is documented as:

"If a callback is provided to then, it will be called once all lines have been read."

The problem is that this callback function is called when reading of the file is stopped prematurely by returning (synchronous version) or passing (asynchronous version) false to the eachLine callback function. In this case, not all of the lines in the file have been read so the final callback function should not be called.

My file‘name is "city.txt",

·58453 beijing
58665 shanghai
58559 china
58652 mmm·

just return the last line "58652 mmm". Tell me why!
My code is
·lineReader.open('./city.txt', function(reader) {
if (reader.hasNextLine()) {
reader.nextLine(function(line) {
console.log(line);
var result = line.split(" ");
//console.log(result[0]+"\t"+result[1]);
});
}
});·

Looking at the code - it's not safe to decode buffers to strings except at separator boundaries - they may end part-way through a multi-byte character, hence the issue.

var callback = function() {}
lineReader.eachLine(file, function(line, last) {
     if(last) callback()
})

in this case, if file is empty, callback won't excute

I think there should be a library to do this stuff. Please help.

They way the internal open function is implemented, it simply throws any error it receives when attempting to open a file. This is a major problem since that error is in a OS-level callback and therefore can't be handled by the caller. Basically, a missing file can crash your entire app.

Here's a brief code example which reproduces the problem:

var lineReader = require("line-reader");

try {
    lineReader.eachLine('missing file', function(line, isLast) {
        console.log("got a line: " + line);
    }).then(function () {
        console.log("all done!");
    });
} catch (e) {
    console.log("got an exception: " + e);
}

What happens on an open or read error? Using the node standard would be nice. Throwing is not nice in node.

When i try to use this code sample...

var lineReader = require('line-reader');

// read all lines:
lineReader.eachLine('file.txt', function(line) {
  console.log(line);
}).then(function (err) {
  if (err) throw err;
  console.log("I'm done!!");
});

... i get this error:

}).then(function (err) {
^
TypeError: lineReader.eachLine(...).then is not a function

Do you have any idea why?

Could you add the offset as a parameter to eachLine?
This would make overwriting a line a lot easier.
I think that's what the variable separatorIndex is in line_reader.js.

Hello, can I use line-reader to read binary with custom separator? It's a binary file with framed data.
Thanks

The first, most basic example in the readme works on a Mac but not on Windows. Tried a file with one line (with line ending) on both platforms. Tried different line endings (\r\n and \n) but no dice. Running the example does nothing on Windows but prints the line on Mac.

Its not really much of an issue

I was just wondering if there was a way to get the value of the previous line

Is it possible that there is a bug in the new version? I have problems in reading bigger files which I have processed successfully in the past. They all stop reading after 900 - 2100 lines by returning reader.hasNextLine() === false. The files are > 100.000 lines long. Each file stops at a different line, but per file the stopping line is the same.

Here is my code:

lineReader.open('bigfile.txt', function(reader) {
    var process_line, i = 0;

    process_line = function(line) {
        console.log(i, line);
        i += 1;
        if( reader.hasNextLine() ) {
            reader.nextLine( process_line );
        }
    };
    reader.nextLine( process_line );
});

If I remove the if like so

        // if( reader.hasNextLine() ) {
            reader.nextLine( process_line );
        // }

the whole file is process, but the function crashes (of course) when the end of file is reached.

Any ideas?

Thanks and best regards,
jochenonline

When multiple workers use the same reader in parallel, with hasNextLine() and nextLine(), some lines are read multiple times.

see the title!

Hello!

Very nice package, really! How can I call an async funtion (i.e. writing the line data to a database) during line processing? The problem for me is, that I want to call nextLine() in the callback of my async function. This does not work, because code is running forth in the meantime.

Thanks for your help.

Hi,
I am using electron.js and I have found line-reader is not working after a page refresh. It reads no lines and throws no errors. Not sure what is happening. It says trying to read the file (plus file name) and then nothing. The first line of File.parseLine is not getting hit. File.parseLine returns false after completion before the page refresh (which I assume allows line-reader to close the file). File.parseLine works properly before the page refresh.

if (fs.existsSync(fileName[0])){ clearData(); //clear the old data before reading the file console.log("trying to read file: " + fileName[0]); lineReader.eachLine(fileName[0], File.parseLine); }

Hey there - I forked your project and created a reverse line reader that starts reading from bottom to top with more or less the same api. Maybe you could add a link to it in the description if people are searching for something like that. https://github.com/paul-em/reverse-line-reader

i run the lineReader.eachLine() method first, then after i changed the txt file (add several lines) and run the function ag, the file seem to be cached forever, cannot read the new added lines even when i restart the server.