FilterIterator implementation that filters files based on a list of suffixes, prefixes, and other exclusion criteria.
License: BSD 3-Clause "New" or "Revised" License
You can add this library as a local, per-project dependency to your project using Composer:
composer require phpunit/php-file-iterator
If you only need this library during development, for instance to run your project's test suite, then you should add it as a development-time dependency:
composer require --dev phpunit/php-file-iterator
Perhaps strictly not a bug in that you intended it that way (that's how it looks at least), but it's certainly unwanted behavior, at least for me. I'd say filter hidden directories within the dir you're scanning, but to skip everything because the directory you asked to scan is in a hidden directory seems odd.
On a side note phpcpd gets stuck in an infinite loop when this occurs. If you decide to make it optional please propagate the option. Thanks!
if i execute phpcpd in a directory up of the source-code, i get the following error:
Warning: strpos(): Empty delimiter in /usr/share/php/File/Iterator.php on line 142
i detect the problem on line 89 of file Iterator.php. On each exclude directory will execute realtpath. but realpath return false, because realpath is not in the right directory.
Two Solutions:
Release 1.3.2 requires PHP 5.2. Release 1.3.3 requires PHP 5.3. Making such a dramatic change in a point release is confusing to end users, such as myself. It would have been nicer to call the release 1.4.0.
This would make it clear for people trying to use older versions of PHPUnit. Using older versions is necessary due to the need to test my publicly released applications on PHP 5.2 for the MANY people still stuck on outdated releases.
Unfortunately the File_Iterator 1.3.3 package breaks PHP 5.2 compatiblity, because it includes an anonymous function in the autoloader but doesn't bump the required PHP version in package.xml.
I just upgraded phpunit in one of my projects, and found an annoying warning about LoggerInterface being defined twice.
It looks like the culprit is tools/composer file in this library, that was not present with phpunit 8.5 while it is with phpunit 9.0
I think that such file should be added to .gitattributes with an export-ignore flag.
I can provide a PR.
Currently branch 1.4 is not compatible with PHP 8.1.
The PEAR channel (pear.phpunit.de) that is used to distribute File_Iterator is no more.
#47 introduced another regression. This was uncovered when I was excluding a directory that didn't exist in my PHPUnit tests, where all of my tests ended up being excluded. I tracked it down to the constructor on SebastianBergmann\FileIterator\Iterator. The order of array_map and array_filter operations needs to be reversed. realpath(false) returns the current working directory, which means the array_filter will not filter out the false entry any longer.
Test Code:
$iterator = new SebastianBergmann\FileIterator\Iterator('C:\my-code-test', new ArrayIterator([]), [], [], [false]);
Expected Result:
class SebastianBergmann\FileIterator\Iterator#224 (4) {
private $basePath =>
string(47) "C:\my-code-test"
private $suffixes =>
array(0) {
}
private $prefixes =>
array(0) {
}
private $exclude =>
array(0) {
}
}
Actual Result:
class SebastianBergmann\FileIterator\Iterator#224 (4) {
private $basePath =>
string(47) "C:\my-code-test"
private $suffixes =>
array(0) {
}
private $prefixes =>
array(0) {
}
private $exclude =>
array(1) {
[0] =>
string(47) "C:\my-code-test"
}
}
{
"require": {
"illuminate/validation": "4.2.*",
"phalcon/incubator": "^2.0",
"phpunit/phpunit": "~4.5"
}
}
tv@DESKTOP-GE3CB0C MINGW64 /c/wamp64/www/bluetv/dev/cztv_hudong_dev/cztv
$ composer update
Loading composer repositories with package information
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.
Potential causes:
Read https://getcomposer.org/doc/articles/troubleshooting.md for further common problems.
File_Iterator's package.xml requires 5.2.7 but it seems to work well with 5.2.6.
I forced the install to run phpcpd and running it worked.
I've just updated PHPUnit to 7.2.4 and our TeamCity CI-Server is not running any tests anymore. The reason imho that the TeamCity agents are using hidden work directories. So it seem, we're back to square #1.
I just update my composer and It update
Now I get the following error:
PHP Fatal error: Uncaught Error: Class 'File_Iterator_Facade' not found in /var/www/html/vendor/phpunit/phpunit/src/Util/Configuration.php:926
Stack trace:
#0 /var/www/html/vendor/phpunit/phpunit/src/Util/Configuration.php(868): PHPUnit_Util_Configuration->getTestSuite(Object(DOMElement), NULL)
#1 /var/www/html/vendor/phpunit/phpunit/src/TextUI/Command.php(783): PHPUnit_Util_Configuration->getTestSuiteConfiguration(NULL)
#2 /var/www/html/vendor/phpunit/phpunit/src/TextUI/Command.php(129): PHPUnit_TextUI_Command->handleArguments(Array)
#3 /var/www/html/vendor/phpunit/phpunit/src/TextUI/Command.php(118): PHPUnit_TextUI_Command->run(Array, true)
#4 /var/www/html/vendor/phpunit/phpunit/phpunit(52): PHPUnit_TextUI_Command::main()
#5 {main}
thrown in /var/www/html/vendor/phpunit/phpunit/src/Util/Configuration.php on line 926
Any idea ?
From wrongly created PEAR bug: https://pear.php.net/bugs/bug.php?id=19133
Calling to File_Iterator_Facade->getFilesAsArray() with
$commonPath enabled, can lead to and endless loop if the
Iterator returns 0 matching files.
100% reproducible against empty directory and/or 0 matching conditions.
File_Iterator_Facade->getFilesAsArray() to return empty
array().
Changing:
if ($commonPath) {
by
if ($files and $commonPath) {
prevents the getCommonPath() method to be called
Note I've created one pull request against phpcpd to workaround this: https://pear.php.net/bugs/bug.php?id=19133
Fixing this will make the pull request unnecesary.
Ciao (and apologises by the confusion @ PEAR) :-)
Wouldn't be better to keep Facade.php and Factory.php under a "Iterator" directory ?
At least it will make simpler for downstream distribution to recreate the previous PSR-0 compliant tree (even if we still provide, and mostly rely on the classmap autoloader ).
Hey Sebastian,
first of all, thank you for your amazing work you've done for the community!
I wondered, if theres a reason why the Facade::getFilesAsArray(...) only accepts a single path as string?
https://github.com/sebastianbergmann/php-file-iterator/blob/main/src/Facade.php#L29
The Factory::getFileIterator(...) can already handle an array of directories, so this would be only a minor change.
I'll be happy to provide a PR for this, if that's OK with you.
/Matthias
Last entry is about 1.4.0
As my test files increasing, I found some of the test case was not able to run. I got the message like this:
Time: 1.7 seconds, Memory: 6.00MB
No tests executed!
Process finished with exit code 0
After running some tests, I figure out that the problem occurs at /src/Facade.php::getFilesAsArray.
When using foreach to run the iterartor created by File_Iterator_Factory, it seems to be ok in normal situation. But when the files in the directory up to more than 60, some of the files seems to be missing.
It may be a problem with php's iterator, And I suggest that you can call $iterator->rewind() before using foreach. Or you can use $iterator->next() and other funtions to iterate.
You can use the file below to reproduce the problem of RecursiveDirectoryIterator, which is used in File_Iterator_Factory
reproduce_test_code.txt
When using phpcpd the excludes (which are handled by File_Iterator) are ignored.
reproduce script:
http://gist.github.com/504453
a patch that fixes this particular issue but probably introduces many more:
http://gist.github.com/500438
Affected version: ^9
Fatal error: Uncaught TypeError: Argument 1 passed to SebastianBergmann\FileIterator\Iterator::acceptPath() must be of the type string, bool given, called in /Users/roquie/google_drive/projects/spacetab-io/calcifer/human-resources/vendor/phpunit/php-file-iterator/src/Iterator.php on line 64 and defined in /Users/roquie/google_drive/projects/spacetab-io/calcifer/human-resources/vendor/phpunit/php-file-iterator/src/Iterator.php on line 69With ^8 version html coverage works fine.
PHP 7.4.5 (cli) (built: Apr 23 2020 02:25:56) ( NTS )
Copyright (c) The PHP Group
Zend Engine v3.4.0, Copyright (c) Zend Technologies
with Xdebug v2.9.0, Copyright (c) 2002-2019, by Derick Rethans
with Zend OPcache v7.4.5, Copyright (c), by Zend TechnologiesThat happened because realpath fails to resolve symbolic link in amphp/windows-registry package.
PHPUnit 9.2.5 by Sebastian Bergmann and contributors.
Runtime: PHP 7.4.5 with Xdebug 2.9.0
Configuration: /Users/roquie/google_drive/projects/spacetab-io/calcifer/project/phpunit.xml
false
SplFileInfo {#80
path: "/Users/roquie/google_drive/projects/spacetab-io/calcifer/project/vendor/amphp/windows-registry/docs"
filename: "asset"
basename: "asset"
pathname: "/Users/roquie/google_drive/projects/spacetab-io/calcifer/project/vendor/amphp/windows-registry/docs/asset"
extension: ""
realPath: false
type: "link"
writable: false
readable: false
executable: false
file: false
dir: false
link: true
linkTarget: ".shared/asset"
}
Possible solutions:
What do you think?
With --exclude you define one or multiple dirs which are exluded from parsing. We use the phpcpd to detect copied code in TYPO3 extensions at ci.typo3.org. There are several directorys we want exclude from parsing because they contain 3rd party code and/or librarys. The existence of such a directory is not mandatory. So, if we exclude the res/ dir in the call, but it doesn't exist, the error
"PHP Warning: strpos(): Empty delimiter in /usr/share/php/File/Iterator.php on line 142"
orrcurs.
This error related to http://github.com/sebastianbergmann/php-file-iterator/issues#issue/2
Hello,
we noticed that our phpunit tests started to freeze before execution for up to 1 minute. After debugging, I found that the bottclneck is in the $fileIteratorFacade->getFilesAsArray(...) method.
It foreaches over every file in the folders recursevely, even if the folders are excluded. Let me explain it with the example:
Consider we have the following structure:
first
├── exclude
│ ├── exclude1.php
│ └── exclude2.php
├── first1.php
└── second
├── second1.php
└── second2.phpand the following simple script:
<?php
require 'vendor/autoload.php';
use SebastianBergmann\FileIterator\Factory;
$factory = new Factory();
$iterator = $factory->getFileIterator('first', '', '', ['first/exclude']);
foreach ($iterator as $file) {
echo $file->getRealPath(), "\n\n";
}When I add a debugging code to the vendor here:
php-file-iterator/src/Iterator.php
Lines 73 to 77 in 7f0f297
I get the following result:
/tmp/phpunit-filter/first/first1.php
>> Excluding /tmp/phpunit-filter/first/exclude/exclude2.php
>> Excluding /tmp/phpunit-filter/first/exclude/exclude1.php
/tmp/phpunit-filter/first/second/second2.php
/Users/maksrafalko/tmp/phpunit-filter/first/second/second1.phpPlease note that while first/exclude folder is excluded, Iterator keeps traversing over all files from excluded folder.
Imagine, we have hundred of thousands of files (vendor folders) inside excluded folder, but each file is anyway processed.
From my opinion, folders that are excluded must not be traversed. Let's see how symfony/finder does it. Again, create a simple script for the same folder structure:
<?php
require 'vendor/autoload.php';
use Symfony\Component\Finder\Finder;
$finder = new Finder();
$finder->files()->in('first')->exclude('exclude');
foreach ($finder as $file) {
echo $file->getRealPath(), "\n\n";
}and let's see the result:
/tmp/phpunit-filter/first/first1.php
>> Excluding exclude
/tmp/phpunit-filter/first/second/second2.php
/tmp/phpunit-filter/first/second/second1.phpAs you can see, symfony/finder meets first/excluded folder, sees that it is excluded and does not traverse it farther.
This is done thanks to FilterIterator and its implementation in symfony/finder: ExcludeDirectoryFilterIterator
php-file-iterator?Thank you.
I am trying to exclude some dirs from consideration for a particular testsuite. At first I thought that any entries under the testsuite->exclude node would be relative to the testsuite->directory value, but after glancing at the source code, it is evident they are not relative.
I'd like to put in the request to make these testsuite->exclude dirs relative to testsuite->directory so I don't have to hardcode full paths within my phpunit configuration file.
The easy fix would be to remove the array_map('realpath', $exclude) code, however I get why you are doing it, to ensure that the exclude paths are indeed legitimate. So I would suggest something akin to: http://pastebin.com/ZtfXXz9r
Also, for the record, if basepath is NOT null, it is actually being stripped out via the str_replace code, so the fullpath is only needed for the validation of whether it is legitimate directory.
Thanks.
Psalm currently reports:
INFO: MissingTemplateParam - src/Iterator.php:27:13 - SebastianBergmann\FileIterator\Iterator
has missing template params when extending FilterIterator, expecting 3 (see https://psalm.dev/182)
final class Iterator extends FilterIterator
However, I do not (yet) know how to use @template-extends to document the fact that SebastianBergmann\FileIterator\Iterator extends \FilterIterator to iterate over \AppendIterator(\RecursiveIteratorIterator(\RecursiveDirectoryIterator)) and returns a string for each iteration step.
Currently, when requiring PHPUnit with Composer as phpunit/phpunit: >=3.6.10,<3.8, the installation fails because there is no stable, tagged version that is known to packagist.org:
Your requirements could not be resolved to an installable set of packages.
Problem 1
- phpunit/phpunit 3.7.1 requires phpunit/php-file-iterator >=1.3.1@stable -> no matching package found.
- phpunit/phpunit 3.7.0 requires phpunit/php-file-iterator >=1.3.1@stable -> no matching package found.
- Installation request for phpunit/phpunit >=3.6.10,<3.8 -> satisfiable by phpunit/phpunit 3.7.0, phpunit/phpunit 3.7.1.
It sucks that packagist does not scan for older releases, but that's unfortunately nothing to be fixed in the near time. So it would be great if you could tag a new release (1.3.1 is already 9 months old anyway).
Alternatively, one could restore the branch alias 1.3.x-dev and update phpunit to not require a stable release, but this would in turn require a new phpunit release.
@sebastianbergmann It's good to provide some "Getting started" section in README.md, with some example codes about how to use this library. 👍
cztv@DESKTOP-GE3CB0C MINGW64 /c/wamp64/www/bluetv/dev/cztv_hudong_dev/cztv
$ composer require --dev phpunit/php-file-iterator
[InvalidArgumentException]
Could not find a version of package phpunit/php-file-iterator matching your minimum-stability (stable). Require it
with an explicit version constraint allowing its desired stability.
require [--dev] [--prefer-source] [--prefer-dist] [--fixed] [--no-progress] [--no-suggest] [--no-update] [--no-scripts] [--update-no-dev] [--update-with-dependencies] [--update-with-all-dependencies] [--ignore-platform-reqs] [--prefer-stable] [--prefer-lowest] [--sort-packages] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--] []...
tools/composer and managed through composer self-update (see update-tools target in build.xmltools/psalm (phive install --copy psalm)tools/php-cs-fixer (phive install --copy php-cs-fixer)Hi, I tried the exact same build with php-file-iterator 1.4.3:
https://travis-ci.org/pug-php/pug-symfony/jobs/307889898
Execute all tests.
And with 1.4.4:
https://travis-ci.org/pug-php/pug-symfony/jobs/307872780
Output No tests executed!
Something with the new exclude system seems too restrictive.
This method could be improved as for each folder definitions, you're doing an array_merge, while you could do a single one no matter how much folders there's in the list:
private function pathsAfterResolvingWildcards(array $paths): array
{
$_paths = [[]];
foreach ($paths as $path) {
if ($locals = glob($path, GLOB_ONLYDIR)) {
$_paths[] = array_map('\realpath', $locals);
} else {
$_paths[] = [realpath($path)];
}
}
return array_filter(array_merge(...$_paths));
}Got this from kalessil's best practices: https://kalessil.github.io/phpinspectionsea/docs/performance.html
Originally posted by @niconoe- in sebastianbergmann/phpcpd#193 (comment)
I had installed that and phpunit dies with
till@dev:~$ phpunit
PHP Fatal error: Class 'File_Iterator' not found in /usr/share/php/File/Iterator/Factory.php on line 105
This was the list of installed packages:
till@dev:~$ pear list -c pear.phpunit.de
Installed packages, channel pear.phpunit.de:
============================================
Package Version State
DbUnit 1.0.3 stable
File_Iterator 1.3.0RC1 beta
PHPUnit 3.5.15 stable
PHPUnit_MockObject 1.1.0RC2 beta
PHPUnit_Selenium 1.1.0RC1 beta
PHP_CodeCoverage 1.0.5 stable
PHP_Timer 1.0.2 stable
PHP_TokenStream 1.1.0RC2 beta
Text_Template 1.1.1RC1 beta
phpUnderControl 0.5.0 beta
phpcpd 1.3.2 stable
phploc 1.6.1 stable
A downgrade to 1.2.6 fixed it.
Hello,
Inside my phpunit.xml.dist I have the following lines
<testsuites>
<testsuite name="UnitTests">
<directory>tests/Unit</directory>
<directory>src/Component/*/tests</directory>
<exclude>src/Component/*/tests/*Integration*</exclude>
<exclude>src/Component/*/tests/*/*Integration*</exclude>
</testsuite>
<testsuite name="FunctionalTests">
<directory>tests/Functional</directory>
</testsuite>
<testsuite name="IntegrationTests">
<directory>tests/Integration</directory>
</testsuite>
</testsuites>When I run this command: php vendor/bin/phpunit --colors=always --no-coverage --testsuite UnitTests --stop-on-failure
I get:
PHPUnit 4.8.36 by Sebastian Bergmann and contributors.
....
....
Time: 163 ms, Memory: 5.25MB
No tests executed!
This used to work before updating.
This issue is reproducible with PHPUnit 7.3.5 installed via composer.
Sample configuration file:
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
>
<testsuites>
<testsuite name="PHPUnit">
<directory>tests</directory>
<exclude>tests/NonExistingFile.php</exclude>
</testsuite>
</testsuites>
</phpunit>Given that the tests/ directory is not empty and contains valid tests, running phpunit produces:
$ phpunit
PHPUnit 7.3.5 by Sebastian Bergmann and contributors.
Time: 12 ms, Memory: 2.00MB
No tests executed!
Removing the <exclude> fixes the issue:
$ phpunit
PHPUnit 7.3.5 by Sebastian Bergmann and contributors.
. 1 / 1 (100%)
Time: 20 ms, Memory: 4.00MB
OK (1 test, 1 assertion)
The issue seems to be introduced by #35 which added realpath() filtering for excluded paths. For a non-existing path, Factory::getPathsAfterResolvingWildcards() returns false which later are interpreted somewhere in PHPUnit as current directory:
$path = '/non/existing/file';
var_dump($path);
$path = realpath($path);
var_dump($path);
$path = realpath($path);
var_dump($path);
// string(18) "/non/existing/file"
// bool(false)
// string(21) "/home/morozov"This seems to be the cause of sebastianbergmann/phpunit#2815.
When trying to make a single "exclude" relative, it strips base path from it which doesn't work if an "exclude" is a parent of base path, which (supposedly) must exclude all the contents of base path.
While #27 tries to address this issue, I'm not sure whether this would work on Windows so probably it needs another take.
I had a performance issue with PHPUnit today. It was related to code coverage and folders that are excluded. This is the relevant section from my phpunit.xml file:
<filter>
<whitelist>
<directory>.</directory>
<exclude>
<directory>vendor</directory>
</exclude>
</whitelist>
</filter>Changing vendor to ./vendor causes a considerable performance increase.
Factory should use canonical names so it can properly exclude folders. Alternatively, the documentation could be improved to account for this scenario.
Thanks
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.