bcremer / linereader

Read large files line by line in a memory efficient (constant) way.

LineReader

LineReader is a library to read large files line by line in a memory efficient (constant) way.

Install

Via Composer

$ composer require bcremer/line-reader

Usage

Given we have a textfile (some/file.txt) with lines like:

Line 1
Line 2
Line 3
Line 4
Line 5
Line 6
Line 7
Line 8
Line 9
Line 10

Also let's assume the namespace is imported to keep the examples dense:

use Bcremer\LineReader\LineReader;

Read forwards

foreach (LineReader::readLines('some/file.txt') as $line) {
    echo $line . "\n"
}

The output will be:

Line 1
Line 2
Line 3
Line 4
Line 5
...

To set an offset or a limit use the \LimitIterator:

$lineGenerator = LineReader::readLines('some/file.txt');
$lineGenerator = new \LimitIterator($lineGenerator, 2, 5);
foreach ($lineGenerator as $line) {
    echo $line . "\n"
}

Will output line 3 to 7

Line 3
Line 4
Line 5
Line 6
Line 7

Read backwards

foreach (LineReader::readLinesBackwards('some/file.txt') as $line) {
    echo $line;
}

Line 10
Line 9
Line 8
Line 7
Line 6
...

Example: Read the last 5 lines in forward order:

$lineGenerator = LineReader::readLinesBackwards('some/file.txt');
$lineGenerator = new \LimitIterator($lineGenerator, 0, 5);

$lines = array_reverse(iterator_to_array($lineGenerator));
foreach ($line as $line) {
    echo $line;
}

Line 6
Line 7
Line 8
Line 9
Line 10

Testing

$ composer test

$ TEST_MAX_LINES=200000 composer test

License

The MIT License (MIT). Please see License File for more information.

linereader's Issues

PHP 8.2 compatibility

just to let you know, I am working on that 🙂

spoiler: it is already compatible, due to the probably wrong version constraint ^8.1 😁

but I will add some pipeline adjustments, cs-fixer, phpstan and so on 👍

Use of mutation testing in LineReader - Help needed

Hello there!

My name is Ana. I noted that you use the mutation testing tool Infection in the project.
I am a postdoctoral researcher at the University of Seville (Spain), and my colleagues and I are studying how mutation testing tools are used in practice. With this aim in mind, we have analysed over 3,500 public GitHub repositories using mutation testing tools, including yours! This work has recently been published in a journal paper available at https://link.springer.com/content/pdf/10.1007/s10664-022-10177-8.pdf.

To complete this study, we are asking for your help to understand better how mutation testing is used in practice, please! We would be extremely grateful if you could contribute to this study by answering a brief survey of 21 simple questions (no more than 6 minutes). This is the link to the questionnaire https://forms.gle/FvXNrimWAsJYC1zB9.

We apologize if you have already received message multiple times or if you have already had the opportunity to complete the survey. If you have already shared your feedback, we want to convey our appreciation, kindly disregard this message, and please accept our apologies for any inconvenience.

Drop me an e-mail if you have any questions or comments ([email protected]). Thank you very much in advance!!