micromatch / glob-fs Goto Github PK

All the examples show scanning the default folder. The docs vaguely indicate that I can modify glob.options.cwd, but it doesn't seem to do the trick. Specifying an absolute folder like /users/Kristian/** doesn't work either. Am I supposed to run process.chdir() to scan a specific folder?

@doowb / @tunnckoCore

I just pushed up a first pass at this. there is still a lot of work to be done but I think the basic conventions are a good start. I'd love feedback on the following:

• multiple pattern support. suggestions for approach - obviously there are a few existing solutions that do this: globby, micromatch, multimatch etc. but I wanted to see if you guys had any ideas for how we could make it more dynamic. I decided to use a file object in glob-fs, since it really simplifies options handling, etc. I'm thinking that we might be able to implement some interesting ignore/unignore features that leverage this. maybe even something like the stash in jade/css/stylus.
• iterators: what can we do to make iterators smarter without making them more complicated? e.g. iterators should be able to passively include, passively exclude, actively include and actively exclude files based on configuration options, pattern matching, etc.

any other thoughts would be great!

Im working on tunnckoCore/benz, which i think is awesome.
Because lies on top of great libs - async-done, now-and-later and co. Allows you to pass async, callback, generator functions, passing context through them; returning and yielding promises, thunks streams; handles errors and completion of all of them.

You can look at the tests for now. Also want to mention that it is almost like bach, but bach dont have options and flexibility that i need and want. You cant pass context to all of the functions with it. With benz you can pass context and can pass result of one middleware to next if you want. Support running on parallel and on series. Just look the tests, please :)

I will try to implement also to accept sync functions. It is little bit more work cuz promises and streams looks like sync functions, so it is lil' bit tricky to determine which is stream, which is sync, which is promise. But I believe it is possible.

You can try benz now (v0.2.0) here in glob-fs instead of custom calling middlewares - its ready, just need more tests and docs that I will add today.

With the title i mean like this

var glob = require('glob-fs')()

glob.use(function notemp(file, next) {
  if (/temp/.test(file.path)) {
    file.exclude = true
  }
  next(null, file)
})

or to be possible through option like async: true?

When I try this

require('glob-fs')().readdirPromise('**/*.yaml').then((_)=>console.log(_))

I see just just all files in dir

I suppose that error is somewhere here:

    readdirPromise: function(pattern, options) {
      this.emit('read');
      this.setPattern(pattern, options);
      var res = this.iteratorPromise(this.pattern.base);  
      this.emit('end', this.files);
      return res;
    }

because it's strange to me that end is emitted before promise is resolved.

but change from this:

      this.emit('end', this.files);

to this:

      res.then(() => this.emit('end', this.files));

didn't help

Did I used this method properly? Or am I missing somethig? Or is there some error?

It would be really great if the README made it clear that this package requires git to be on the path as well as carrying the "experimental" and "doesn't work on Windows" warnings which are shown on github. This would help make sure that devs who discover the package on www.npmjs.org are aware of these limitations before they spend time trying to try out the package.

Thanks.

Hi.
I was expecting glob to recursively list all files and (internally) reject all that don't match provided pattern. However, given following actions taken:

git clone https://github.com/ama-team/voxengine-sdk.git
cd voxengine-sdk
git checkout f2a0af
npm i glob-fs -D # installed version: 0.1.7
cat <<EOF > index.js   
var glob = require('glob-fs')
glob().readdirPromise('test/**/*.spec.js')
  .then(console.log.bind(console));
EOF
node index.js

I have following output:

[
  '/tmp/voxengine-sdk/test/mocha.opts', // unexpected
  '/tmp/voxengine-sdk/test/spec/http/_common.spec.js',
  '/tmp/voxengine-sdk/test/spec/http/basic.spec.js',
  '/tmp/voxengine-sdk/test/spec/http/rest.spec.js',
  '/tmp/voxengine-sdk/test/spec/logger/_common.spec.js',
  '/tmp/voxengine-sdk/test/spec/logger/slf4j.spec.js',
  '/tmp/voxengine-sdk/test/support/mocha-multi-reporters.json', // unexpected
  '/tmp/voxengine-sdk/test/support/setup.js' // unexpected
]

Looks like it's either me misinterpreting how this package is designed to work (and all of the above is valid output) or a bug

Could you please replace the micromatch github- by a npm-dependency?
This leads to trouble by firewall restrictions or deployments through inhouse npm caches. Thx!

The TODO section of the readme says that async iterator support is done, but I don't see any examples in the readme...is it possible to for await with the current API?

Anyone is free to create these. Middleware should be extremely specific. this keeps them very fast, more composable, and more likely to be interesting to more users

Please add a link to your middleware in the readme if you create one!

• glob-fs-tilde: expand a leading tilde to an absolute file path (using expand-tilde)
• glob-fs-bower-ignore: ignore files or directories specified in bower.json
• glob-fs-ignore-tests: ignore test files/fixtures automatically.

Any more ideas?

$ node -v
v4.8.3
$ npm -v
5.0.1
$ npm i -g glob-fs
npm ERR! code ETARGET
npm ERR! notarget No matching version found for [email protected]
npm ERR! notarget In most cases you or one of your dependencies are requesting
npm ERR! notarget a package version that doesn't exist.
npm ERR! notarget 
npm ERR! notarget It was specified as a dependency of 'glob-fs'
npm ERR! notarget 

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/user/.npm/_logs/2017-06-02T20_41_06_057Z-debug.log

Node.js was installed via nvm, and I upgraded npm to v5.

Wishlist

Some user features that would be nice to have in a node.js file globbing lib:

• define patterns as a string or array
• allow ignore patterns to be passed on the options
• ignore node_modules by default. IMO it makes more sense for a user to explicitly define node_modules to include it.

API

• streams
• async
• sync

Code

• how much flexibility to we want to provide for negation patterns?
• is it important to allow users to un-negate previously negated patterns? If not, things become simpler, but I don't think we'll need to compromise on speed either way

I hope this is just me not understanding glob syntax. I seem to be able to read test and src separately, but not together, joined in braces.

> var glob = require('glob-fs')();
undefined

> glob.readdirStream('../foopath/test/**').on('data', function(file) { console.log(file.path); }), null;
null
> /Users/steveluscher/.../foopath/test/testOneFile.js
/Users/steveluscher/.../foopath/test/testTwoFile.js

> glob.readdirStream('../foopath/src/**').on('data', function(file) { console.log(file.path); }), null;
null
> /Users/steveluscher/.../foopath/src/srcOneFile.js
/Users/steveluscher/.../foopath/src/srcTwoFile.js

> glob.readdirStream('{../foopath/src,../foopath/test}/**').on('data', function(file) { console.log(file.path); }), null;
null

Note that the same glob pattern used with ls lists all four files:

ls -al {../foopath/src,../foopath/test}/**

The dependency for: "micromatch": "jonschlinkert/micromatch#2.2.0", pulls from github repo instead of npm. This breaks on firewalled networks that do not have access to public github. Can this be changed to pull from npm instead?

package: "glob-fs": "^0.1.6"

dot files (.git .gitignore ...) are included in my array results.

I get the below when using readdirPromise:
/Users/scott/Dropbox/Docs/Scripts/mail-restore-js/node_modules/glob-fs/lib/symlinks.js:38
self.realpath(file, cb);
^

TypeError: Cannot read property 'realpath' of undefined
    at /Users/scott/Dropbox/Docs/Scripts/mail-restore-js/node_modules/glob-fs/lib/symlinks.js:38:15
    at LOOP (fs.js:1609:14)
    at _combinedTickCallback (node.js:376:9)
    at process._tickCallback (node.js:407:11)

I'm using the Promise-based API which doesn't seem to respect the file.exclude property. The excluded files are still returned.

It does work with the other readdir methods

running npm audit with latest glob-fs gives:

High Prototype Pollution
Package set-value
Patched in >=2.0.1 <3.0.0 || >=3.0.1

Would you be updating package.json to use latest set-value (currently it is ^0.2.0).

When trying to run the sync example from the website, I get the following error:

    return fp.split('\\').join('/');
             ^

TypeError: Cannot read property 'split' of undefined

For reference, here is the code:

var glob = require('glob-fs')({ gitignore: true });

var files = glob.readdirSync('**/*.js');
console.log(files);

My hunch is it involves line 74 of glob-fs/lib/pattern.js: this.base = path.join(this.cwd, this.parent);. Running the debugger, the path.join returns a nonsense path (the c:/ drive is in both this.cwd and this.parent and path.join just smashes them together).

Even if I fix the issue of this.pattern.re vs this.pattern.regex, I still get the same 'fp is undefined' error.

Facing issue on package installation.

npm ERR! code ENOLOCAL
npm ERR! Could not install from "node_modules/glob-fs/micromatch@github:jonschlinkert/micromatch#5017fd78202e04c684cc31d3c2fb1f469ea222ff" as it does not contain a package.json file.

Getting this Error on npm install
pacakge.json contains
"glob-fs": "^0.1.7",

node - 8.10.0
npm - 6.14.9

Please help with this issue.
@jonschlinkert

Why does the module prepend it's own path to the passed path in readdirSync?

Surely this is an oversight?

given

``` debug(processing files at path ${process.cwd()} );

let fileProcessor = new FileProcessor(this._config);
fileProcessor.rootPath = process.cwd();
let files = glob.readdirSync(this._config.globPattern);
files.forEach( (file) => {```

where the process.cwd() is /Users/georgecook/Documents/h7ci/hope/opensource/navSpike/build/.roku-deploy-staging

and a globPattern of **/*.brs

I find the returned files have a path of .roku-deploy-staging/components/Framework/Core/BaseAggregateView.brs - this is wrong! that folder doesn't even exist - it's as if readdirSync is running the glob against build instead of build.roku-deploy-staging

I'll have to stop using this package if I can't get around this, as it's breaking my tools.

error: /Users/craigcosmo/Desktop/electron skeleton/webpack.config.babel.js:49
var files = _globFs2.default.readdirSync('*.js');
^

TypeError: _globFs2.default.readdirSync is not a function

Reproduce

import fs from 'fs'
import glob from 'glob-fs'
let files = glob.readdirSync('*.js');

console.log(files)

I have used glob-fs in my program for a faster glob experience. The full pattern recognition was not available. I ran many attempts with patterns similar to *!(.js) and every result included .js. This is unfortunate as this looks like a promising library.

When trying to use glob-fs 0.1.6 just right out of the box I had to do the following modification in index.js, to get it running:

    // if middleware are registered, use the glob, otherwise regex
    var glob = this.fns.length
      ? this.pattern.glob
      //: this.pattern.re; //<-- error
      : this.pattern.regex; //<-- fix

If I call globFs.readdirSync(path, {cwd: '/'}) multiple times I get a different array each time. The file list is being added to the last response each time.

Note I'm using the cwd param because I'm passing in absolute paths.

I work in Windows.

How would I go about looking for files like, for example, both:
*.txt and *.TXT

glob.readdir returns either the one or the other, but not both.

For example I have 2 files:
file1.txt
file2.TXT

Now, I want to find all files with extension txt (Upper and Lower case). How would I do that without calling 2 functions (*.txt and *.TXT) ?