js-devtools / readdir-enhanced Goto Github PK

I want to build a stream and only listen to files. I'm building it like so:

const files = readdirEnhanced.readdirStreamStat('/tmp/test')

files.on('file', (file) => console.log(file))
files.on('end', () => console.log('end'))

If I run the above code, no files pass through the stream. If I add a data listener (even an empty function), things start moving:

const files = readdirEnhanced.readdirStreamStat('/tmp/test')

files.on('data', () => {
  // no-op
})

files.on('file', (file) => console.log(file))
files.on('end', () => console.log('end'))

Is this intended behaviour? If so, I'm happy to update the documentation a bit to make it clearer why the data handler is needed.

Cheers

I use the following under Windows

const readdir = require('readdir-enhanced')
const result = readdir.sync.stat('C:/')

Under my system this fails because some files are protected (for instance C:\pagefile.sys). Should there be an option to either skip those errors and return the rest of the file stats, or to return the other files as part of the exception payload?

Why both readdir-enhanced and @jsdevtools/readdir-enhanced exist?

Hello, @BigstickCarpet,

Firstly, very thanks you for this package. This package works faster analogues and covers all my needs. 🌮

Do you not think about how to prevent the reading of the directory, if it has been filtered? I'm not mean filter of results, I talk about skip the contents of the directory, if it has been filtered.

For example,

filter

function filter(stat) {
  console.log('stat: ', stat.path);
  if (stat.isDirectory() && stat.path === 'a') {
    console.log('Hello from the "a/" directory!');
    return false;
  }
  return true;
}

current behaviour

stat:  a
Hello from the "a/" directory!
stat:  a.txt
stat:  a\b                  // <--- but "a" is filtered
stat:  a\b\b.no-txt    // <--- but "a" is filtered
stat:  a\b\c               // <--- but "a" is filtered
stat:  a\b\c\d           // <--- but "a" is filtered
stat:  a\b\c\d\e.txt   // <--- but "a" is filtered

expected behaviour

stat:  a
Hello from the "a/" directory!
stat:  a.txt

I ask because this may severely degrade performance on very large projects.

when i use filter: '*txt in my code it always return empty array
readdir(path1, { basePath: path1, filter: "*.txt", deep: true}, function(err, files) {

if i remove the filter i will show all the files in the directory including the subdirectories

Hello,

And it's me again :)

We have a small problem. When working with large nested directory, we often apply the filter. For example, built-in filter by patterns:

const re = require('readdir-enhanced');

let files = null;

try {
    files = re.sync('./node_modules', {
        deep: true,
        filter: '**/test-*'
    });
} catch (error) {
    console.log('MEMORY: ' + (process.memoryUsage().heapUsed / 1e6) + ' MB');

    console.dir(error, { colors: true });
}

console.dir(files, { colors: true });

In the example above, I call the reading of the directory that contains 20320 entries:

$ npm i jest ava babel-core standard eslint typescript tslint monaco xterm readdir-enhanced

After running the script we get the following:

MEMORY: 9.37484 MB

node_modules/readdir-enhanced/lib/call.js:51
      throw err;
      ^

RangeError: Maximum call stack size exceeded
    at Object.fs.lstatSync (fs.js:839:18)
    at exports.lstat (node_modules/readdir-enhanced/lib/sync/fs.js:58:20)
    at Object.safeCall [as safe] (node_modules/readdir-enhanced/lib/call.js:24:8)
    at stat (node_modules/readdir-enhanced/lib/stat.js:19:8)
    at DirectoryReader.processItem (node_modules/readdir-enhanced/lib/directory-reader.js:171:5)
    at node_modules/readdir-enhanced/lib/sync/for-each.js:14:5
    at Array.forEach (native)
    at Object.syncForEach [as forEach] (node_modules/readdir-enhanced/lib/sync/for-each.js:13:9)
    at node_modules/readdir-enhanced/lib/directory-reader.js:80:16
    at onceWrapper (node_modules/readdir-enhanced/lib/call.js:45:17)

If I remove any filter (pattern, function), everything works fine.

Should add something that says

Installation

npm install --save readdir-enhanced

This would help identifying the npm repository as genuine.

Well, I want to use your package with my own FS facade functions, for example, scandir-native. It seems that it would be useful not only me. For example, if you want to use this package for walk directories in the remote drive.

Links

• mrmlnc/fast-glob#47

The motivation

The native instruments (ls, find, tree) do it by default.

Is there a way with readdir.stream() to know when it is finished? I couldn't see any appropriate event for this or anything in the docs.

Note, I did try .on('finish', function () { }), but this did not get called.

If we use deep option as custom function we cannot control depth. May be we must provide number of depth to function arguments?

For example:

const options = {
	deep: (entry: IReaddirEntry, depth: number) => ...
};