sindresorhus / gulp-filter

Filter files in a `vinyl` stream

gulp-filter

Filter files in a vinyl stream

Enables you to work on a subset of the original files by filtering them using glob patterns. When you're done and want all the original files back, you just use the restore stream.

Install

npm install --save-dev gulp-filter

Usage

Filter only

You may want to just filter the stream content:

import gulp from 'gulp';
import uglify from 'gulp-uglify';
import filter from 'gulp-filter';

export default () => {
	// Create filter instance inside task function
	const f = filter(['**', '!*src/vendor']);

	return gulp.src('src/**/*.js')
		// Filter a subset of the files
		.pipe(f)
		// Run them through a plugin
		.pipe(uglify())
		.pipe(gulp.dest('dist'));
};

Restoring filtered files

import gulp 'gulp';
import uglify 'gulp-uglify';
import filter 'gulp-filter';

export default () => {
	// Create filter instance inside task function
	const f = filter(['**', '!*src/vendor'], {restore: true});

	return gulp.src('src/**/*.js')
		// Filter a subset of the files
		.pipe(f)
		// Run them through a plugin
		.pipe(uglify())
		// Bring back the previously filtered out files (optional)
		.pipe(f.restore)
		.pipe(gulp.dest('dist'));
};

Multiple filters

By combining and restoring different filters you can process different sets of files with a single pipeline.

import gulp from 'gulp';
import less from 'gulp-less';
import concat from 'gulp-concat';
import filter from 'gulp-filter';

export default () => {
	const jsFilter = filter('**/*.js', {restore: true});
	const lessFilter = filter('**/*.less', {restore: true});

	return gulp.src('assets/**')
		.pipe(jsFilter)
		.pipe(concat('bundle.js'))
		.pipe(jsFilter.restore)
		.pipe(lessFilter)
		.pipe(less())
		.pipe(lessFilter.restore)
		.pipe(gulp.dest('out/'));
};

Restore as a file source

You can restore filtered files in a different place and use it as a standalone source of files (ReadableStream). Setting the passthrough option to false allows you to do so.

import gulp 'gulp';
import uglify 'gulp-uglify';
import filter 'gulp-filter';

export default () => {
	const f = filter(['**', '!*src/vendor'], {restore: true, passthrough: false});

	const stream = gulp.src('src/**/*.js')
		// Filter a subset of the files
		.pipe(f)
		// Run them through a plugin
		.pipe(uglify())
		.pipe(gulp.dest('dist'));

	// Use filtered files as a gulp file source
	f.restore.pipe(gulp.dest('vendor-dist'));

	return stream;
};

API

filter(pattern, options?)

Returns a transform stream with a .restore property.

pattern

Type: string | string[] | Function

Accepts a string/array with globbing patterns which are run through multimatch.

If you supply a function, you'll get a vinyl file object as the first argument and you're expected to return a boolean of whether to include the file:

filter(file => /unicorns/.test(file.path));

options

Type: object

Accepts minimatch options.

Note: Set dot: true if you need to match files prefixed with a dot, for example, .gitignore.

restore

Type: boolean
Default: false

Restore filtered files.

passthrough

Type: boolean
Default: true

When set to true, filtered files are restored with a stream.PassThrough, otherwise, when set to false, filtered files are restored as a stram.Readable.

When the stream is a stream.Readable, it ends by itself, but when it's stream.PassThrough, you are responsible of ending the stream.

gulp-filter's Issues

Glob returns an empty stream

filter(glob) returns an empty stream (no result) in this function (current glob doesn't work):

gulp.task('views', function() {
  return gulp.src('source/views/**/*.jade')
    .pipe(filter([ '*.jade', '!{_*.jade,includes/**,_backup/**}' ]))
    .pipe(jade())
    .pipe(gulp.dest('public'))

while a nearly identical function (with a same glob) works well:

gulp.task('views', function() {
  return gulp.src('source/views/**/*.jade')
    .pipe(filter(function(file) {
      var match = multimatch([file.relative], [ '*.jade', '!{_*.jade,includes/**,_backup/**}' ])
      if (match.length)
        return file
    }))
    .pipe(jade())
    .pipe(gulp.dest('public'))

Am I doing something wrong?

"gulp": "^3.9.1"
"gulp-jade": "^1.1.0"
"gulp-filter": "^4.0.0"

The first usage example is incorrect and gives the wrong impression about how gulp-filter matches files.

I'm referring to this

var gulp = require('gulp');
var jscs = require('gulp-jscs');
var gulpFilter = require('gulp-filter');

gulp.task('default', function () {
    // create filter instance inside task function
    var filter = gulpFilter(['*', '!src/vendor']);

    return gulp.src('src/*.js')
        // filter a subset of the files
        .pipe(filter)
        // run them through a plugin
        .pipe(jscs())
        .pipe(gulp.dest('dist'));
});

The intent seems to process everything in src but to exclude the files in src/vendor. Problems:

1. The filter does no work because src/*.js won't go into subdirectories in the first place. You could remove the filter and you'd still not get anything from src/vendor into dest.
2. If we amend to src/**/*.js then this pattern will cover every .js file in src, including subdirectories but the filter still gives the wrong impression. It does the work of excluding files in src/vendor not because of !src/vendor but because of *, which matches only files that are not in a subdirectory. You could reduce the filter to gulpFilter('*') and it would still exclude src/vendor and everything in it.
3. If we amend the filter so that we have ** instead of *, then the filter no longer works because !src/vendor does not match anything. Why? Because gulp-filter matches against the file's relative path.

Ultimately, the code I see doing what the example seems to be illustrating should be:

var gulp = require('gulp');
var jscs = require('gulp-jscs');
var gulpFilter = require('gulp-filter');

gulp.task('default', function () {
    // create filter instance inside task function
    var filter = gulpFilter(['**', '!vendor/**']);

    return gulp.src('src/**/*.js')
        // filter a subset of the files
        .pipe(filter)
        // run them through a plugin
        .pipe(jscs())
        .pipe(gulp.dest('dist'));
});

The examples based off of this one should also be fixed.

Restore is messing up file order

I have a gulp task that runs multiple tasks on my styles and afterwards I'd like to concat some vendor styles to everything that's been processed. The thing is that I want sourcemap support of the styles I'm writing and right now restore does a weird thing where it puts the filtered out files BEFORE everything else (in my case 'boards.less') which messes up the sourcemaps.

So before I had filePatterns.styles, filePatterns.vendorStyles but after restore it turns into filePatterns.vendorStyles, filePatterns.styles. Any idea on how to fix this?

var path = require('path'),
    gulp = require('gulp'),
    concat = require('gulp-concat'),
    less = require('gulp-less'),
    sourcemaps = require('gulp-sourcemaps'),
    gulpFilter = require('gulp-filter');

var filePatterns = {
  styles : 'src/styles/**/*.{css,less}',
  vendorStyles: 'vendor/styles/**/*.css'
}


module.exports = function(wrapperType, environment) {
  var filterOutVendor = gulpFilter(function(file) {
    return !/vendor\/styles/.test(file.path);
  });

  return gulp.src([filePatterns.styles, filePatterns.vendorStyles])
    // filter out vendor styles
    .pipe(filterOutVendor)
    .pipe(sourcemaps.init())
    .pipe(concat('boards.less'))
    .pipe(less())
    .pipe(sourcemaps.write())
    .pipe(filterOutVendor.restore())
    .pipe(concat('boards.css'))
    .pipe(gulp.dest(path.join(filePaths.build, wrapperType)));
};

Extending filter behaviour

Before I start to create new plugin, I want to ask a question here.

Here is my use case:

• Created filter: var ctplFilter = filter('**/*.ctpl');.
• Matched some files: ['file1.js', 'file2.ctpl', 'file3.js', 'file4.ctpl', 'file5.js'].
• Applied filter, called some plugin, restored filter: .pipe(ctplFilter).pipe(compile()).pipe(ctplFilter.restore());.

Let's say that this compile() plugin, eats all files and emits ['compiled1.js', 'compiled2.js'] to stream.

After filter is restored my stream looks like ['file1.js', 'file3.js', 'file5.js', 'compiled1.js', 'compiled2.js'] but what I need is ['file1.js', 'compiled1.js', 'compiled2.js', 'file3.js', 'file5.js'] or ['file1.js', 'file3.js', 'compiled1.js', 'compiled2.js', 'file5.js'] (new files on first occurrence or on last occurrence of filtered subset).

Do you think we can do it inside gulp-filter plugin?

gulp-filter is not letting any files pass.

Hi, I'm new to the plugin but this definitely seems like a bug.

gulp.task('js', function(cb){
const jsFiles = ['./client/js//.js'];
pump([
gulp.src(jsFiles),
filter('.js'),
concat('main.js'),
babel({
presets:['es2015']
}).on('error', gutil.log),
uglify(),
gulp.dest(dest + 'js')
],
cb
);
});

The problem is that no files are passing through the filter. When I take the filter out, my .js files are processed correctly.

Filter without restoration exits gulp before task finishes

My gulp task:

gulp.task('useref', 'Bundle CSS and JS based on build tags and copy to `dist/` folder', function () {
  // run useref only in build
  if (build.isBuild()) {
    var assets = useref.assets(config.useref.assetsCfg);

    var jadeFilesOnly = filter(['**/*.jade'], {restore: true});
    var excludeJade = filter(['**','!**/*.jade'], {restore: true});

    return gulp.src(config.useref.src)
      .pipe(assets)
      .pipe(gulpif('*.js', gulpif(config.uglifyJs, uglify()))) // uglify JS
      .pipe(gulpif('*.css', gulpif(config.minifyCss, minifyCss()))) // minify CSS
      .pipe(gulpif(config.cacheBust, rev()))
      .pipe(assets.restore())
      .pipe(useref())
      .pipe(gulpif(config.cacheBust, revReplace({replaceInExtensions: ['.jade', '.css', '.js']})))
      .pipe(jadeFilesOnly)
      .pipe(gulp.dest(config.useref.destJade))
      .pipe(jadeFilesOnly.restore)
      .pipe(excludeJade) 
      .pipe(gulp.dest(config.useref.dest))
      .pipe(gulpif(config.cacheBust, rev.manifest(config.useref.revManifestCfg))) // create rev-manifest.json 
      .pipe(gulp.dest(config.useref.dest));
  } else {
    return;
  }
});

This .pipe(excludeJade) silently exits gulp flow without error. If I add.pipe(excludeJade.restore)` in the end of stream, everything works as expected. (OSX 10.11/Windows 7, Node 0.12.7/4.2.0)

Restoration is optional according the docs.

Matching problem on Windows

Some filters are not working on Windows.

This works:

var scssfilter = filter(["**/*.scss"], { restore: true });

But this does not (everything is filtered out):

var scssfilter = filter(["Components/**/*.scss"], { restore: true });

It seems anything more advanced is not working. This does not work either:

var scssfilter = filter(["*", "!**/*.scss"], { restore: true });

It works with other libs utilizing minimatch, such as gulp.src.

How to use with Yeoman Transform Stream

From the "Transform output files through streams" section of the Yeoman docs, it says:

"Note that every file of any type will be passed through this stream. Make sure any transform stream will passthrough the files it doesn't support. Tools like gulp-if or gulp-filter will help filter invalid types and pass them through."

There is no further documentation or examples and the following gives an error of TypeError: object is not a function:

var gulpFilter = require('gulp-filter');

...

var beautifyFilter = gulpFilter([
  '_package.json'
]);

this.registerTransformStream(beautifyFilter(beautify({
  indentSize: 2,
  indentChar : ' ',
  preserveNewlines : false
})));

Can't filter based on folder name

Prior to upgrading to 2.0.2 (from 0.3.0) I was able to filter like so:

gulp.src('./**')
       .pipe(gulpFilter('**/fonts/*')
       .pipe(doStuff());

Once I upgraded that filter no longer returns data. I believe this behavior is an unintended consequence of df53fe3 which compares the glob to the relative path of the file rather than the full path as expected. Is there a way to filter as specified in the example in gulp-filter 2.0?

Filter seems to change order of src files (e.g. for concatenation)

Let's say I have the following code:

var sass_filter = $.filter('*.scss');
var less_filter = $.filter('*.less');
src.styles = [
    "src/styles/style1.scss",
    "src/styles/style2.less"
  ];
  return gulp.src(src.styles)
    .pipe(less_filter)
    .pipe($.less())
.pipe(less_filter.restore())
.pipe(sass_filter)
  .pipe($.sass())
.pipe(sass_filter.restore())
.pipe($.concat('style.css')) // Due to the filters, file order is mixed up
.pipe(gulp.dest(DEST + '/css'))
.pipe($.size({title: 'style'}));

Without using the filters, style1 and style2 are concatenated in the correct order, however, when using the filters they are not. Is this correct?

After applying `[email protected]`, following pipes not working

Hello, after applying [email protected], following plugin like vinyl-map - not working, in older version work fine.

Thanks

Example

var stream = gulp.src(files)
      .pipe(gulp.dest(path))
      .on('error', plugins.util.log)
      .pipe(plugins.filter('*.css'))
      .pipe(map(function (code, filename) {
        plugins.util.log('CSS ' +
        plugins.util.colors.green(filename))
      }))
      .pipe(browserSync.reload({ stream: true }))
    return stream

Filtered stream never ending

First of all, thanks so much for this contribution, I <3 the project.

My issue is I have a filtered stream that i'm running async tasks on, and because it never calls done on those tasks, so the gulp task never finishes. I see in the docs that this is expected, but would it be possible to turn the filter.restore into a stream that automatically ends like gulp.src streams?

I have a universal js app that I'm trying to file rev all the files that are going to a CDN, then spilt off the server files and put them into a dist folder, then upload the rest of the assets to a CDN. The CDN assets are the ones being filtered out, and that stream is never ending. To finish the task I re-merge the streams and return that so gulp will wait until both tasks are completed before it finishes the task. Any other thoughts or suggestions on how to fix this are more than welcome.

Consider adding .npmignore file

Hey,

I am a happy user of your library, but as my project grows we have a lot of dependencies.
We could save a lot of space if you added .npmignore file to your project.

I even found tool that can help you with it: https://github.com/inikulin/dmn

Thanks!

Restore does not save order of files

Hi!

We've had a bug with modules order because of the issue in your restore function. It is not saving the order of files that was in gulp.src before.

Example:

return gulp.src(["file1.js", "file2.coffee", "file3.js"])
           .pipe(filterCoffee)
           .pipe(compileCoffeeFiles())
           .pipe(filterCoffee.restore)
           .pipe(concat(fileName))
           .pipe(gulp.dest(fileDir));

In this example, on some CPUs you'll have order file1, file3, file2 in the result.

But concat save order from gulp.src

Adding a filter.restore options to end the stream when not used as a through stream

Hi,

Thanks for this plugin.

I ran into an issue will using it with StreamQueue for changing the files order in my builds.

Here is my task:

// Bower
gulp.task('build_bower', function() {
  var jsFilter = g.filter('**/*.js');
  var jqFilter = g.filter(['!**/dist/jquery.js']);
  var ngFilter = g.filter(['!**/angular.js', '!**/angular-mocks.js']);
  var ngStrictFilter = g.filter(['**/ui-utils.js']);
  var cssFilter = g.filter('**/*.css');
  return new StreamQueue({objectMode: true},
    jqFilter.restore(), // Never ended
    ngFilter.restore(), //never ended
    g.bowerFiles({
      paths: {
        bowerDirectory: src.vendors
      },
      includeDev: true || !prod // We currently depend on faker, so let it be always true for now..
    })
      .pipe(ngStrictFilter)
      .pipe(g.insert.wrap(
        ';\n(function(window, angular, undefined) {\n',
        '\n})(window, window.angular);\n'
      ))
      .pipe(ngStrictFilter.restore())
      .pipe(jqFilter) 
      .pipe(ngFilter)
      .pipe(jsFilter)
    ).pipe(g.cond(prod, g.streamify(g.concat.bind(null, 'libs.js'))))
    .pipe(getPathesOfStream(scripts, src.html))
    .pipe(jsFilter.restore())
    .pipe(cssFilter)
    .pipe(g.cond(prod, g.minifyCss))
    .pipe(g.cond(prod, g.concat.bind(null, 'vendors.css')))
    .pipe(getPathesOfStream(styles, src.html))
    .pipe(cssFilter.restore())
    .pipe(g.cond(prod, gulp.dest.bind(null, build.vendors)));
});

The task never completes since the restore stream are never ended. So my idea is to provide a "end" parameter allowing to end the restore stream when you do not use it as a PassTrough but as a simple Readable.

Something like:

jsFilter.restore({end: true})

What about it ? I can PR you if you agree on its API.

Not working anymore since 4.0.0

I have this gulp task that works perfectly with [email protected]:

gulp.task('build:sources', ['inject'], function() {
  var htmlFilter = $.filter('*.html', {restore: true});
  var jsFilter = $.filter('**/*.js', {restore: true});
  var cssFilter = $.filter('**/*.css', {restore: true});

  return gulp.src(path.join(conf.paths.tmp, 'index.html'))
    .pipe($.replace(/<html/g, '<html ng-strict-di'))
    .pipe($.useref())
    .pipe($.if('**/app*.js', $.intercept(setEnvironment)))
    .pipe(jsFilter)
    .pipe($.ngAnnotate())
    .pipe($.uglify({preserveComments: $.uglifySaveLicense})).on('error', conf.errorHandler('Uglify'))
    .pipe($.rev())
    .pipe(jsFilter.restore)
    .pipe(cssFilter)
    .pipe($.cleanCss({processImport: false}))
    .pipe($.rev())
    .pipe(cssFilter.restore)
    .pipe($.revReplace())
    .pipe(htmlFilter)
    .pipe($.htmlmin({
      removeComments: true,
      collapseWhitespace: true
    }))
    .pipe(htmlFilter.restore)
    .pipe(gulp.dest(path.join(conf.paths.dist, '/')))
    .pipe($.size({title: path.join(conf.paths.dist, '/'), showFiles: true}));
});

Once I upgrade gulp-filter to 4.x, files do not get filtered anymore... I've read through the docs and I could not find any breaking change, what could I be missing?

prevents gulp.dest to work

This gist has a commented out line
.pipe($.filter(function(f) { console.log(f); }))

If this line is added, gulp.dest doesn't work and files are not written to destination.

buffering all files and the `.end()` event could easily cause excessive memory use

This also seems to break the "plugin should do one thing" rule.

cc: @contra, @phated

TypeError: undefined is not a function

gulp.task('build', ['inject', 'images', 'fonts'], function(){
log('*** Building Production - Optimizing Assets - HTML,CSS,JS ***');

var templateCache = config.temp + config.templateCache.file;
var assets = $.useref.assets({ searchPath: './' });
var cssFilter = $.filter('**/*.css');
//var jsFilter = $.ignore.include('/*.js');

return gulp
    .src(config.index)
    .pipe($.plumber())
    .pipe($.inject(gulp.src(templateCache, {read: false}),{
        starttag: '<!-- inject:templates:js -->'
    }))
    .pipe(assets)
    //.pipe(cssFilter)
    //.pipe($.csso())
    //.pipe(cssFilter.restore())
    .pipe(jsFilter)
    .pipe($.uglify())
    .pipe(jsFilter.restore())
    .pipe(assets.restore())
    .pipe($.useref())
    .pipe(gulp.dest(config.build))

});

Problem to restore the filtered files

I'm trying the following:

  return gulp
  .src('./bower.json')
  .pipe(bower())

  .pipe(filters.js)
  .pipe(uglify())
  .pipe(rename({
      suffix: '.min'
  }))
  .pipe(filters.js.restore)
  .pipe(gulp.dest(vendor + '/js/'));

and I get the following error:

TypeError: Cannot read property 'on' of undefined
    at Transform.Readable.pipe (_stream_readable.js:495:7)
    at Gulp.<anonymous> (/Users/breno/Documents/nativo-landing/gulpfile.js:62:4)
    at module.exports (/Users/breno/Documents/node_modules/gulp/node_modules/orchestrator/lib/runTask.js:34:7)
    at Gulp.Orchestrator._runTask (/Users/breno/Documents/node_modules/gulp/node_modules/orchestrator/index.js:273:3)
    at Gulp.Orchestrator._runStep (/Users/breno/Documents/node_modules/gulp/node_modules/orchestrator/index.js:214:10)
    at Gulp.Orchestrator.start (/Users/breno/Documents/node_modules/gulp/node_modules/orchestrator/index.js:134:8)
    at /usr/local/lib/node_modules/gulp/bin/gulp.js:129:20
    at process._tickCallback (node.js:355:11)
    at Function.Module.runMain (module.js:503:11)
    at startup (node.js:129:16)

If I try filters.js.restore() instead, then I get the following error:

TypeError: undefined is not a function
    at Gulp.<anonymous> (/Users/breno/Documents/nativo-landing/gulpfile.js:62:20)
    at module.exports (/Users/breno/Documents/node_modules/gulp/node_modules/orchestrator/lib/runTask.js:34:7)
    at Gulp.Orchestrator._runTask (/Users/breno/Documents/node_modules/gulp/node_modules/orchestrator/index.js:273:3)
    at Gulp.Orchestrator._runStep (/Users/breno/Documents/node_modules/gulp/node_modules/orchestrator/index.js:214:10)
    at Gulp.Orchestrator.start (/Users/breno/Documents/node_modules/gulp/node_modules/orchestrator/index.js:134:8)
    at /usr/local/lib/node_modules/gulp/bin/gulp.js:129:20
    at process._tickCallback (node.js:355:11)
    at Function.Module.runMain (module.js:503:11)
    at startup (node.js:129:16)
    at node.js:814:3

I'm using an object to handle all my filters:

var filters = {
    js: filter('**/*.js'),
    css: filter('*.css'),
    font: filter(['*.eot', '*.woff', '*.svg', '*.ttf']),
    img: filter(['*.png', '*.gif'])
  };

log only summary instead of each file

currently running

//handle assets
gulp.task('assets', function () {
    //copy regular assets
    gulp.src(paths.origin.assets)
        .pipe(imageFilter)
        .pipe(imagemin())
        .pipe(imageFilter.restore())
        .pipe(gulp.dest(paths.build));
});

I have around 100 images, this causes a flood of message of:

...
[gulp] gulp-imagemin: ✔ img\dials\plus.jpg (already optimized)
[gulp] gulp-imagemin: ✔ img\dials\mnitro.jpg (saved 570 B)
[gulp] gulp-imagemin: ✔ img\dials\fiverr.jpg (saved 3.2 kB)
[gulp] gulp-imagemin: ✔ img\dials\coupons.com.jpg (saved 3.2 kB)
[gulp] gulp-imagemin: ✔ img\dials\nana10.jpg (saved 3.2 kB)
[gulp] gulp-imagemin: ✔ img\dials\npr.org.jpg (saved 3 B)
[gulp] gulp-imagemin: ✔ img\dials\Quora.jpg (already optimized)
[gulp] gulp-imagemin: ✔ img\dials\theweatherchannel.png (saved 630 B)
...

It's great for debug purposes but annoying for production/watch.

Is it possible to just get a concise summary of the output:

[gulp] gulp-imagemin: processed 100 files (50 already optimized, saved 1.2MB)

Can't create patterns that work both on Windows and Linux, because of path separator "\\"

I've spent some time trying to make gulp-filter work with a vary basic setup.
I discovered it is not doing what it is supposed to because I'm on a Windows machine and using "/" in paterns.

My solution was to use a function pattern like this one:

gulpFilter(function (o) {
    var rel = o.relative.replace(/\\/g, '/');
    return pattern.test(rel);
});

instead of

gulpFilter(pattern);

and suddenly it worked.

I think it should be done internally by gulp-filter and let people write cross-platform gulp tasks easily.

2.0.1 restore() faulty

filter.restore() is not working properly, where it does with 2.0.0
files passing filter are gone after filter.restore()

Unable match on a per-directory basis

Sample gulpfile:

var gulp = require('gulp'),
    debug = require('gulp-debug'),
    filter = require('gulp-filter');

gulp.src('./build/css/**/*.css')
    .pipe(debug({ title: 'base-pre' }))
    .pipe(filter('build/css/**/*.css'))
    .pipe(debug({ title: 'base-post' }));

gulp.src('./build/css/**/*.css')
    .pipe(debug({ title: 'glob-pre' }))
    .pipe(filter('**/*.css'))
    .pipe(debug({ title: 'glob-post' }));

Output:

[14:29:32] base-pre ~/path/to/project/build/css/site.css
[14:29:32] base-pre ~/path/to/project/build/css/dummy.css
[14:29:32] base-pre 2 items
[14:29:32] base-post 0 items
[14:29:32] glob-pre ~/path/to/project/build/css/site.css
[14:29:32] glob-pre ~/path/to/project/build/css/dummy.css
[14:29:32] glob-post ~/path/to/project/build/css/site.css
[14:29:32] glob-post ~/path/to/project/build/css/dummy.css
[14:29:32] glob-pre 2 items
[14:29:32] glob-post 2 items

I'm not quite sure as to what's causing this issue; whether it has to do with streamfilter, multimatch or minimatch, non-glob patterns are clearly not being applied correctly.

Related issue: #52

---

Upon further analysis, I have come to the conclusion that this really only has to do with the way Gulp handles streams works (as opposed to actually being a bug; see gulpjs/gulp#699).

All things considered, I ended up setting the base directory to ./ to keep it from interpreting the build directory as the base: gulp.src('./build/css/**/*.css', { base: './' })

It could be a good idea to include a similar example in the README.md so to help out other users facing this issue.

Allow streaming mode

Since gulp-filter never deals with file contents, i think it shouldn't disable the streaming mode https://github.com/sindresorhus/gulp-filter/blob/master/index.js#L17

Can miss files in subfolders

var filter = gulpFilter(["**/*.html"]);
gulp.src(["**"],{cwd:"public/",cwdbase:true})
.pipe(filter)
.pipe(someHandler)
.pipe(filter.restore())
.pipe(gulp.dest("build/public/")

Some files missing in subfolders that does not pass into filter

Not working with gulp-git

const f = filter(['!*.min.js']);

return gulp.src(src)
    .pipe(f)
    .pipe(git.rm({args: '-f'}));

--->

Error: Command failed: git rm  -f

If I remove the filter gulp-filter it is working like a charme.

Somehow gulp-git is no longer getting path informations when gulp-filter is used.

Grab minified files

Using gulp filter with main-bower-files. I want to grab the minified files. Is this possible? I tried

var jsFilter  = gulpFilter('*.min.js', {restore: true});

return gulp.src(bowerMain())                                                    │
  .pipe(jsFilter)                                                              
  .pipe(gulp.dest('./temp/js'))
  .pipe(jsFilter.restore)

When I run it nothing gets copied. If I change filter to *.js it pulls all except minified files

multimatch file.relative vs file.path

Hi,

Since you change recently multimatch(file.path for multimatch(file.relative filtering is now very limited

here:
https://github.com/sindresorhus/gulp-filter/blob/master/index.js#L20

For example, this pattern seems logic, but does not work anymore with relative:

gulp.watch(["./src/js/**/*.js"], function(e) {
    gulp.src(e.path)
      .pipe(filter('!**/vendor/**'))
      .pipe(jshint())
      .pipe(jshint.reporter(stylish))
  });

Are you sure is the best solution?

filter.restore is not defined

filter.restore is not defined in 3.x.
This appears to be a regression as in version 2.0.2 it works fine, in the 3.x branch I get

TypeError: undefined is not a function

This is probably related to #42

Minor mistake in readme example

For the current “Restore as a file source” section in the readme, the code will not work as expected as there is a return statement before the last block of code. You should probably just save a reference to the stream somewhere and return it after the 2nd stream.

Failure after 16 false matches.

I'm running filter on a list of files (originally obtained from gulp-bower-files) and it seems to fail after a certain number of files.

I dug a bit and by echoing console.log(restoreStream._readableState.length) i figured this happens everytime that counter hits 16.

I'm running latest versions of everything.

    return gulp.src([ ... ])
        .pipe(filter('**/*.js'))
        .pipe(size({ showFiles: true }))

Any ideas?

wont return anything if there is a "." in directory name

I already have created and issue in multimach repo.
if there is a "." in the path like ".tmp/styles.css" multimatch then the pattern "*.css" wont work.
as a workaround i just renamed my ".tmp" to tmp

Passthrough restoration causing debug count message to show 0

This same issue happens with gulp-filter:

ops.copy = function(taskName, src, dest) {
    var bowerComponents = $.filter(['**/*', '!' + paths.bower.src + '**/*'], {restore: true, passthrough: false}),
        stream;

    stream = gulp.src(src, {base: process.cwd()})
        .pipe($.plumber({errorHandler: ops.showError}))
        .pipe(bowerComponents)
        .pipe($.rename(function (filePath) {
            filePath.dirname = '.';
        }))
        .pipe($.newer(dest))
        .pipe($.debug({title: taskName}))
        .pipe(gulp.dest(dest));

    bowerComponents.restore
        .pipe($.rename(function (filePath) {
            filePath.dirname = filePath.dirname.split(path.sep)[1];
        }))
        .pipe($.newer(dest))
        .pipe($.debug({title: taskName}))
        .pipe(gulp.dest(dest));

    return stream;
};

Shows:

$ gulp font.build
Gulp loaded in 1.582 seconds
[02:23:23] Using gulpfile gulpfile.js
[02:23:23] Starting 'font.build'...
[02:23:23] font.build font-awesome\fontawesome-webfont.eot
[02:23:23] font.build font-awesome\fontawesome-webfont.svg
[02:23:23] font.build font-awesome\fontawesome-webfont.ttf
[02:23:23] font.build font-awesome\fontawesome-webfont.woff
[02:23:23] font.build font-awesome\fontawesome-webfont.woff2
[02:23:23] font.build font-awesome\FontAwesome.otf
[02:23:23] font.build slick.js\slick.eot
[02:23:23] font.build slick.js\slick.svg
[02:23:23] font.build slick.js\slick.ttf
[02:23:23] font.build slick.js\slick.woff
[02:23:23] font.build 0 items
[02:23:23] Finished 'font.build' after 51 ms

It's showing 0 items on the filtered restore even though the debug messages are still showing up. Does it have to do with what's being returned?

why isn't gulp-filter filtering out less files in my bower packages

i'm trying to get all the files in my bower components using main bower files and filtering them according type using gulp-filter. it works fine when i filter out the js files like so:

gulp.task('bower_js', function() {
  var jsFilter = gulpFilter('**/*.js');
  gulp.src(mainBowerFiles(), {base: './bower_components'})
    .pipe(jsFilter)
    .pipe(concat('vendor.js'))
    .pipe(gulp.dest('./_public/js'));
});

however running a similar task for less doesn't work (ie nothing goes through the filter):

gulp.task('bower_css', function() {
  var lessFilter = gulpFilter('**/*.less');
  gulp.src(mainBowerFiles(), {base: './bower_components'})    
    .pipe(lessFilter)
     // .pipe(less()) commented to narrow down the problem
    .pipe(concat('app.css'))
    .pipe(gulp.dest('./_public/css'));
});

although if I run mainBowerFiles without any filter.. it dumps .less files content into my destination folder..

to give a specific example.. this is the relevant part of bower.json of one of my bower packages: bootstrap:

  "main": [
    "less/bootstrap.less",
    "dist/css/bootstrap.css",
    "dist/js/bootstrap.js",
    "dist/fonts/glyphicons-halflings-regular.eot",
    "dist/fonts/glyphicons-halflings-regular.svg",
    "dist/fonts/glyphicons-halflings-regular.ttf",
    "dist/fonts/glyphicons-halflings-regular.woff"
  ],

bootstrap.less in turn imports other files:

// Core variables and mixins
@import "variables.less";
@import "mixins.less";

// Reset and dependencies
@import "normalize.less";
@import "print.less";
..

and so on.

any ideas?

Can't exclude a file from SASS

I have a .scss file that breaks gulp-autoprefixer (scss/animation.scss), that's because it is a special file parsed by javascript using skrollr.js (https://github.com/Prinzhorn/skrollr-stylesheets). I would like to compile it to SASS, then exclude the file with gulp-filter and include it back after the autoprefixer has done his job. Could you explain why my task below it is not working?

var gulp = require('gulp'),
    // include plugins
    filters     = require('gulp-filter'),
    sass       = require('gulp-sass'),
    prefix      = require('gulp-autoprefixer');

// sass and css
gulp.task('styles', function() {
    var filter = filters('!scss/animation.scss');

    gulp.src('scss/**/*.scss')
    .pipe(sass())
    .pipe(filter)
    .pipe(prefix('last 30 version', '> 1%', 'ie 8', 'ie 9'))
    .pipe(filter.restore())
    .pipe(gulp.dest('dist/css/'));
});

// Default Task
gulp.task('default', ['styles']);

I have also tried with...

var filter = filters(['*', '!scss/animation.scss']);

Is there a simple way to debug which file is being excluded/included after I apply the filter?

Negative Filter

Hey there!

I was looking for a way to recursively filter folders based on a name and then merge it back in with gulp-rename.
Unfortunately, it seems like I'm terrible with globs or I miss some details.

This is my setup:

gulp.task('gen', () => {
    return gulp.src('src/**/*')
        .pipe(filter(['*', '!**/_wkc']))
        .pipe(gulp.dest('./dist'));
})

This looks easy, but I never get the right setup.

• ['*','!**/_wkc'] only copies over the first level of folders - containing no files - but the _wkc folder is gone.
• ['**','!src/_wkc'] copies over everything including files but also _wkc
• ['**','!**/_wkc'] copies over everything (also files) but _wkc is now missing from the second folder of the second level of folders. (wat!?)
• ['**','!_wkc'] same as ['**','!src/_wkc']

I'm fiddling around with this for hours now.

Thanks a lot in advance

How to filter the files which have underscore ("_") as prefix?

This is a question but not an issue.

I am using gulp. Similar to sass, I am trying to ignore some html files which have _ as prefix. This works fine:

gulp.task('default', function () {
    return gulp
        .src('templates/!(_).html')
        .pipe(/* nunjucks compile code */)
        .dest('dist/')
}

The above code is actually ignoring the underscore prefix files from going through a compiling system (nunjucks). Because of this, I am unable to use compiling code inside the _ prefixed html files.

That is why I am planning to do the compile on all the html files but in dest, I am trying to filter out the _ prefixed html files.

Please help.

Does not seems to work with node 5.10

Hi,

I have just updated to node v5.10.1 and gulp-filter always return empty.

Is it only me?

Thanks.

Glob isn't matching any files.

Maybe this is a problem with my understanding of how multimatch is matching by globs, but for some reason I can't get this snippet to work.

I'm using the gulp-load-plugins plugin, which is assigned to the variable $, in case you're wondering. I've tried directly requiring gulp-filter too.

gulp.task("default", function() {
    var jsFilter = $.filter("**/*.js", {restore: true});
    var cssFilter = $.filter("**/*.css", {restore: true});
    var assets;

    return gulp.src("**/_Layout.cshtml")
        .pipe($.rename("_Layout.build.cshtml"))
        // gulp-useref: concatenates scripts/styles found within build tags in the Layout page.
        .pipe(assets = $.useref.assets({ searchPath: [config.wwwroot] }))
        // This filter will match nothing.
        .pipe(cssFilter)
        // No files in the pipe, so this won't run.
        .pipe($.csso())
        // Restores everything back.
        .pipe(cssFilter.restore)
        // Same again - matches nothing.
        .pipe(jsFilter)
        // Doesn't run.
        .pipe($.uglify())
        // Restores everything back.
        .pipe(jsFilter.restore)
        .pipe(assets.restore())
        .pipe($.useref())
        .pipe($.replace("../../wwwroot", ""))
        .pipe(gulp.dest("./tmp/"));
});

The relative paths to the files in my stream are as follows:

1. ..\..\wwwroot\css\app.css
2. ..\..\wwwroot\js\app.js
3. ..\..\wwwroot\css\lib.css
4. ..\..\wwwroot\js\lib.js

The filters work fine if the relative path to the file doesn't have ..\..\wwwroot\css\ in it.

Some files are skipped during gulp.dest after restore

Many folders stays empty when using 1.0.2. 1.0.1 works fine.

My code:

gulp.task('demo-sources', [/*'composer', */'demo-sources-cleanup'], function(cb) {
    var helperFilter = filter('**/Helper/Data.php');
    return gulp.src(['code/**/*'])
        .pipe(helperFilter)
        .pipe(helperFilter.restore())
        .pipe(gulp.dest('demo-sources/raw'));
});

Sorry that I didn't test at all, maybe it's my fault. I will test it it later.

Is passing File.relative to multimatch correct?

Hi,

It seems to me that as it is, the example on the home page of this module won't work, as it is similar to an issue I had which I've described here: http://stackoverflow.com/questions/24687468/gulp-filter-filters-out-all-files/24708315#24708315

In any case, I'm wondering if it wouldn't be better to pass File.path to the multimatch function instead? When passing File.relative, as is done now, only the filename is passed, which makes it impossible to use filter patterns that filter based on a directory name. An even better approach might be to pass File.path minus File.cwd (leaving the relative path in relation to the gulpfile). It's 2:30 A.M, so I admit I haven't spent a lot of time thinking this through, but thought I'd make the suggestion. I could make a pull request, but I'm not sure whether this is something that's desired by other people as well ;)

Using multiple outputs with filters

For example, I'd like to put all files through certain pipes, but output some of the files to one destination, and others to a separate destination. Possible?

Here's what I'm currently using for a bump task:

var rootFilter = gulpFilter('!./src');
var srcFilter = gulpFilter('./src');
gulp.src(['package.json', 'bower.json', 'src/manifest.json'])
  .pipe(bump({type: 'patch'}))
  .pipe(srcFilter)
  .pipe(gulp.dest('./src'))
  .pipe(srcFilter.restore())
  .pipe(rootFilter)
  .pipe(gulp.dest('./'));
});

Can't reuse filter in the same stream

Hello,
I want to use a filter twice in a stream but I have an error the second time it's called :

stream.js:94
      throw er; // Unhandled stream error in pipe.
            ^
Error: write after end
    at writeAfterEnd (/home/jgx/Projects/javascript/cqst2/node_modules/gulp-filter/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:144:12)
    at DestroyableTransform.Writable.write (/home/jgx/Projects/javascript/cqst2/node_modules/gulp-filter/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:192:5)
    at DestroyableTransform.ondata (stream.js:51:26)
    at DestroyableTransform.emit (events.js:95:17)
    at DestroyableTransform.<anonymous> (/home/jgx/Projects/javascript/cqst2/node_modules/gulp/node_modules/vinyl-fs/node_modules/through2/node_modules/readable-stream/lib/_stream_readable.js:786:14)
    at DestroyableTransform.emit (events.js:92:17)
    at emitReadable_ (/home/jgx/Projects/javascript/cqst2/node_modules/gulp/node_modules/vinyl-fs/node_modules/through2/node_modules/readable-stream/lib/_stream_readable.js:448:10)
    at emitReadable (/home/jgx/Projects/javascript/cqst2/node_modules/gulp/node_modules/vinyl-fs/node_modules/through2/node_modules/readable-stream/lib/_stream_readable.js:444:5)
    at readableAddChunk (/home/jgx/Projects/javascript/cqst2/node_modules/gulp/node_modules/vinyl-fs/node_modules/through2/node_modules/readable-stream/lib/_stream_readable.js:187:9)
    at DestroyableTransform.Readable.push (/home/jgx/Projects/javascript/cqst2/node_modules/gulp/node_modules/vinyl-fs/node_modules/through2/node_modules/readable-stream/lib/_stream_readable.js:149:10)

And here is my stream :

return gulp.src(config.src)
    .pipe(rename({suffix: '.min'}))
    .pipe(sass(
      {
        sourcemapPath: config.sourcemapDest,
        style: 'compressed'
      }
    ))
    .pipe(cssFilter)
    .pipe(autoprefixer('last 1 version'))
    .pipe(csso())
    .pipe(cssFilter.restore())
    .pipe(gulp.dest(config.dest))
    // Can't reuse cssFilter, filter('**/*.css') works
    .pipe(cssFilter)
    .pipe(browserSync.reload({stream: true}));

As I said in the comment, if I replace the second cssFilter with filter('**/*.css') it works well.

Fix filtering

re #1 (comment)

Move filters creation inside tasks in README

Examples in README tell to create filters outside actual tasks. And it works on the first run, but on the second run (when you rerun task with gulp.watch) it throws write after end exception. It took me some time to realize that my plugin isn't the one who caused the problem.

In conclusion, I guess everything like var jsFilter = gulpFilter('**/*.js'); should be moved inside task functions in README examples.

filter.end() does not end the filter

 gulp.task('build-client', function () {
    gulp.src(dist + '{,*/}/**')
   .pipe(filter('!index.html'))
   .pipe(filter.end())   
   .pipe(gulp.dest('./client'));
})

the file is removed from the stream, does not return to it.

Cannot filter->restore several times with the same filter

Hi,

I originally encountered this problem when I tried to declare all my filters at the beginning of the gulpfile. Imagine something like this:

filters = {
    js = gulpFilter("*.js", {restore:true}),
    css = gulpFilter("*.css", {restore:true})
}

While the gulp ran fine the first time, as soon as the gulp.watch fired I got an error (pretty new to gulp, I don't really know how to get useful error messages. If someone links a howto I'll gladly provide logs). However, through trial and error I'm almost certain that the error is with trying to either save or retrieve a restore-stream more than once.

In my situation, this can be avoided by putting the filter definition in the function call, creating a new instance each time (which feels kinda bad to me tbh)
However, there is another case where this can not be avoided (pseudocode):

.pipe(filterJS)
.pipe(doA)
.pipe(filterJS.restore)
.pipe(doB)
.pipe(filterJS)
.pipe(doC)
.pipe(filterJS.restore)

I understand that this is an edgecase. At the same time, it seems pretty fixable (as fixable as code can be anyway) by clearing out a potentially used restore-cache before saving to it.

0.5.0 cause task fail without any message

my code below

gulpBowerFiles()
  .pipe(gulpFilter(function(it){
    return /\.js$/.exec(it.path);   }))
 .pipe(gulp.dest('public/js'));

0.4.1 Result

0.5.0 Result

task will not finish.
and when checking stream in gulpFilter will have weird output.

concat restored files does not work

I've updated to 3.0.0 and tried to change to the new restore API. A basic example works (pipe the restored files to dest) but fails when I pipe to gulp-concat.

gulp.task('test', function () {
  var testFilter = filter(['**', '!**/*-test.js'], { restore: true, passthrough: false });

  var stream = gulp.src('src/**/*.js',)
    .pipe(testFilter)
    .pipe(concat('a.js'))
    .pipe(gulp.dest(path.build + 'test1'));

  testFilter.restore
    .pipe(concat('b.js'))  // <---
    .pipe(gulp.dest(path.build + 'test2'));

  return stream;
});

The filtered files are copied to test2 if I remove the line. With the line, nothing happens in test2.