GithubHelp home page GithubHelp logo

archivestream-php's Introduction

ArchiveStream 1.0.7

Code Climate

A library for dynamically streaming dynamic tar or zip files without the need to have the complete file stored on the server. You can specify if you want a tar or a zip; or if you want to have the library figure out the best option based on the user agent string.

Options

/**
 * Construct Parameters:
 *
 *   $name          - Name of output file (optional).
 *   $opt           - Hash of archive options (optional, see "Archive Options"
 *                    below).
 *   $output_stream - Output stream for archive (optional - defaults to php://output)
 *
 * Archive Options:
 *
 *   comment             - Comment for this archive. (zip only)
 *   content_type        - HTTP Content-Type.  Defaults to 'application/x-zip'.
 *   content_disposition - HTTP Content-Disposition.  Defaults to
 *                         'attachment; filename=\"FILENAME\"', where
 *                         FILENAME is the specified filename.
 *   large_file_size     - Size, in bytes, of the largest file to try
 *                         and load into memory (used by
 *                         add_file_from_path()).  Large files may also
 *                         be compressed differently; see the
 *                         'large_file_method' option.
 *   send_http_headers   - Boolean indicating whether or not to send
 *                         the HTTP headers for this file.
 *   large_files_only    - Boolean indicating whether or not to assume
 *                         that all files we are sending are large.
 *
 * File Options:
 *  time     - Last-modified timestamp (seconds since the epoch) of
 *             this file.  Defaults to the current time.
 *  comment  - Comment related to this file. (zip only)
 *  type     - Type of file object. (tar only)
 *
 *
 * Note that content_type and content_disposition do nothing if you are
 * not sending HTTP headers.
 *
 * Large File Support:
 *
 * By default, the method add_file_from_path() will send send files
 * larger than 20 megabytes along raw rather than attempting to
 * compress them.  You can change both the maximum size and the
 * compression behavior using the large_file_* options above, with the
 * following caveats:
 *
 * * For "small" files (e.g. files smaller than large_file_size), the
 *   memory use can be up to twice that of the actual file.  In other
 *   words, adding a 10 megabyte file to the archive could potentially
 *   occupty 20 megabytes of memory.
 *
 * * For "large" files we use the store method, meaning that the file is
 *   not compressed at all, this is because there is not currenly a good way
 *   to compress a stream within PHP
 *
 * Notes:
 *
 * If you do not set a filename, then this library _DOES NOT_ send HTTP
 * headers by default.  This behavior is to allow software to send its
 * own headers (including the filename), and still use this library.
 */

Usage

Stream whole file at a time

A fast and simple streaming archive files for PHP. Here's a simple example:

// Create a new archive stream object (tar or zip depending on user agent)
$zip = \Barracuda\ArchiveStream\Archive::instance_by_useragent('example');

// Create a file named 'hello.txt'
$zip->add_file('hello.txt', 'This is the contents of hello.txt');

// Add a file named 'image.jpg' from a local file 'path/to/image.jpg'
$zip->add_file_from_path('image.jpg', 'path/to/image.jpg');

// Finish the zip stream
$zip->finish();

Stream each file in parts

This method can be used to serve files of any size (GB, TB).

// Create a new archive stream object (tar or zip depending on user agent)
$zip = \Barracuda\ArchiveStream\Archive::instance_by_useragent('example');

// Initiate the stream transfer of some_image.jpg with size 324134
$zip->init_file_stream_transfer('some_image.jpg', 324134);

// Stream part of the contents of some_image.jpg
// This method should be called as many times as needed to send all of its data
$zip->stream_file_part($data);

// Send data descriptor header for file
$zip->complete_file_stream();

// Other files can be added here, simply run the three commands above for each file that is being sent

// Explicitly add a directory to the zip (doesn't recurse - useful for empty
// directories)
$zip->add_directory('foo');
$zip->add_directory('foo/bar');

// Finish the zip stream
$zip->finish();

Installation

Simply run composer require barracudanetworks/archivestream-php inside your project.

Requirements

Limitations

  • Only the Zip64 (version 4.5 of the Zip specification) format is supported.
  • Files cannot be resumed if a download fails before finishing.

Other

You can also add comments, modify file timestamps, and customize (or disable) the HTTP headers. See the class file for details.

Contributors

  • Paul Duncan - Original author
  • Daniel Bergey
  • Andy Blyler
  • Tony Blyler
  • Andrew Borek
  • Rafael Corral
  • John Maguire
  • Zachery Stuart

License

Original work Copyright 2007-2009 Paul Duncan [email protected] Modified work Copyright 2013-2015 Barracuda Networks, Inc.

Licensed under the MIT License

archivestream-php's People

Contributors

ablyler avatar donatj avatar johnmaguire avatar luku avatar meowvalent avatar pablotron avatar pvitaly avatar rairlie avatar rcorral avatar sami-bel avatar samsouder avatar shimeche avatar zacherystuart avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

archivestream-php's Issues

Feature Request: Resume Download

This is a great script. I'm considering using it for our website which has millions of high-res images uploaded by users. This script would allow them to download their images without java-applets or multiple prompts. However, the download sometimes stops, and perhaps it would be possible to use the HTTP RANGE to resume downloads that have stopped midways? (given that all file sizes are given and no compression occurs).

Thanks :)

Offline zip

Hi JohnMaguire,

Is it possible do offline zip. I mean the library can be configured to write to a zip file Instead of streaming.

Thanks,
Trung.

Example function to archive directory tree

archive_stream_zip_directory( '/home/web/public_html', 'web-html.zip' );

function archive_stream_zip_directory( $root, $archive = 'archive.zip' ) {
    if ( ! class_exists( 'RecursiveIteratorIterator' )
        || ! function_exists( 'gzdeflate' ) )
        return 1;

    if ( ! file_exists( $root ) )
        return 2;

    if ( function_exists( 'proc_nice' ) )
        proc_nice( 19 );

    $files = array();
    $added = array();
    $objects = new RecursiveIteratorIterator( new RecursiveDirectoryIterator( $root ), RecursiveIteratorIterator::SELF_FIRST );
    $zip = new ArchiveStream_Zip( $archive );

    foreach ( $objects as $name => $object ) {
        // http://php.net/manual/en/splfileinfo.getbasename.php
        $basename = $object->getBasename();
        if ( '.' === $basename || '..' === $basename || $object->isDir() )
            continue;
        $zip->add_file_from_path( ltrim( $name, '/' ), $name );
        //DBG echo $name.'<br>';
        $added[] = $name;
    }

    $zip->finish();
    if ( function_exists( 'proc_nice' ) )
        proc_nice( 0 );

    return $added;
}

int64_split(): Unable to convert variable to GMP - wrong type

In this file at /vendor/barracudanetworks/archivestream-php/src/Archive.php line 409

I get;

Barracuda\ArchiveStream\Archive::int64_split(): Unable to convert variable to GMP - wrong type

Error. Downloaded zip file is corrupted and error reporting is appended to zip file (when i open with notepad i can see the error trace)

How can I overcome on this?

Zip files larger than 4GB fail on Windows

Hello,
Thanks for this great library. I am able to integrate this with a project that I am working on. The library creates downloadable zips perfectly when the entire zip file size is under 4GB. However, once the zip file size exceeds 4GB, the downloaded zip is not usable anymore. Unzip utilities complain that it is an corrupted zip.
Some more details of the issue:
Format used: Zip
No of files used : 10 (each file averaging about 500MB)
Total zip size: about 4.6 GB
Unzip utilities tried (and failed): Windows default, winzip and ALZip

Can you please let me know if I am missing something or this is a know limitation of the library.

Speed & Time-Limit / stopping after some minutes

Hi there,
i am using ArchiveStream to deliver a large amount of files (test set is ~20 files with 500mb each). However it seems to be quite slow (about 500 - 1000 kb/s transfered from the server while a static download reaches about 5-6 mb/s). I'm not a 100 percent sure but can this be a problem from ArchiveStream?

Second issue is that the download stops after some time (it seems to stop after time not after having downloaded a specific amount of bytes), mostly after about 5 minutes (which is near the php max_execution_time).

This is the code i am using:

$zip = \Barracuda\ArchiveStream\Archive::instance_by_useragent("example", array(
	"large_files_only" => true
));															
foreach($files as $singleFile) {
	$zip->add_file_from_path(basename($singleFile), get_path($singleFile));
}			
$zip->finish();

any help would be highly appreciated, thanks a lot :)

Corrupt Zip archive on macOS

I get a corrupt archive when using this code on macOS Sierra using PHP 7.1:

$zip = new \Barracuda\ArchiveStream\ZipArchive('foobar.zip');

$zip->add_file('hello.txt', 'This is the contents of hello.txt');

$zip->finish();

BetterZip reports the following:

Error: Can not open file as archive

Can't open as archive: 1
Files: 0
Size:       0
Compressed: 0

PHP 8.1 - Deprecated: Implicit conversion from float

Starting in PHP 8.1, I'm getting the following deprecation warning when creating a zip file:

Deprecated: Implicit conversion from float 1.8446744069414584E+19 to int loses precision in /tmp/test/vendor/barracudanetworks/archivestream-php/src/Archive.php on line 439

Thanks for your help.

The archive is corrupt

Dear,

I download the latest version, and run index.php in test folder but I got the error "The archive is corrupt" when open the downloaded archive with WinRar.

Thank you,
Hugo

Feature Request: add_file_from_path remote file

Perhaps there is already a way to do this, but I am trying to zip up files located on a remote server. How can I do that without having to download them first. Example:
$zip->add_file_from_path('videos/video.mp4', "https://dl.dropboxusercontent.com/1/view/jhdiuy2sdfs/somefiles/dd-4/45727_SampleVideo_1280x720_5mb.mp4", $file_opt);

Would be great if it treated that remote path just like a local one.

7-Zip reports Headers Error

Recent versions of 7-Zip report an error for any zip file created with ArchiveStream.

$ 7z t example.zip 

7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=hu_HU.UTF-8,Utf16=on,HugeFiles=on,64 bits,2 CPUs Intel(R) Xeon(R) CPU           E5640  @ 2.67GHz (206C2),ASM)

Scanning the drive for archives:
1 file, 936 bytes (1 KiB)

Testing archive: example.zip

ERRORS:
Headers Error


WARNINGS:
Headers Error

--
Path = example.zip
Type = zip
ERRORS:
Headers Error
WARNINGS:
Headers Error
Physical Size = 936
64-bit = +

    

Archives with Errors: 1

Warnings: 1

Open Errors: 1

example.zip was created using the following code:

<?php

require_once('src/Archive.php');
require_once('src/ZipArchive.php');

use Barracuda\ArchiveStream\ZipArchive;

$zip = new ZipArchive('example.zip');
$zip->add_file_from_path('LICENSE', 'LICENSE');
$zip->finish();

PHP output was simply redirected to ZIP file:

php example.php > example.zip

Older 7-Zip versions (for example 9.20) do not report any problems about the same file:

$ 7z t example.zip 

7-Zip 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
p7zip Version 9.20 (locale=hu_HU.UTF-8,Utf16=on,HugeFiles=on,2 CPUs)

Processing archive: example.zip

Testing     LICENSE

Everything is Ok

Size:       1144
Compressed: 938

Most of out clients use Windows. Some of them reported difficulties to extract our ZIP files. Recommended them to use 7-Zip, downloaded recent version (18.01) tested a file, got Headers Error.

Output stream isn't written to with default param

The output stream isn't written to when using the default param. I originally thought it was an issue with using STDOUT in a non-CLI environment, but after implementing a similar fix to #48, I wasn't seeing any changes.

I added some of our logging into the send() method to see what's going on:

protected function send($data)
{
    if ($this->need_headers)
    {
        $this->send_http_headers();
    }

    \Log::message("Output stream is valid: " . (fstat($this->output_stream) !== false ? 'true' : 'false'));
    \Log::message("Writing " . strlen($data) . " bytes");

    $this->need_headers = false;

    do
    {
        $result = fwrite($this->output_stream, $data);
        \Log::message("Wrote $result bytes");
        $data = substr($data, $result);
        fflush($this->output_stream);
    } while ($data && $result !== false);
}

I received these logs:

Output stream is valid: true
Writing 4096 bytes
Wrote 4096 bytes
Output stream is valid: true
Writing 1084 bytes
Wrote 1084 bytes
Output stream is valid: true
Writing 452 bytes
Wrote 452 bytes

Unfortunately, the tar downloaded was 0 bytes.

Zip archives cannot be opened on OS X (Archive Utility, unzip)

When attempting to unzip a Zip archive on OS X via Finder, the system's archive utliity will only produce a .zip.cpgz file (apparently a temporary file created during decompression). Likewise, using unzip from the command line will fail with the warning "zipfile claims to be last disk of a multi-part archive".

The reason seems to be that unzip on OS X (including El Capitan) has been compiled without Zip64 support (unzip -v output below from OS X 10.9.5):

UnZip 5.52 of 28 February 2005, by Info-ZIP.  Maintained by C. Spieler.  Send
bug reports using http://www.info-zip.org/zip-bug.html; see README for details.

Latest sources and executables are at ftp://ftp.info-zip.org/pub/infozip/ ;
see ftp://ftp.info-zip.org/pub/infozip/UnZip.html for other sites.

Compiled with gcc 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68) for Unix on Aug 24 2013.

UnZip special compilation options:
    COPYRIGHT_CLEAN (PKZIP 0.9x unreducing method not supported)
    SET_DIR_ATTRIB
    TIMESTAMP
    USE_EF_UT_TIME
    USE_UNSHRINK (PKZIP/Zip 1.x unshrinking method supported)
    USE_DEFLATE64 (PKZIP 4.x Deflate64(tm) supported)
    VMS_TEXT_CONV
    [decryption, version 2.9 of 05 May 2000]`

Would it be possible to make the use of Zip64 optional? I can't claim to be even remotely familiar with the Zip specs, and looking at ZipArchive I was unable to figure out the necessary steps.

Zip file is invalid in php version 5.5 and 5.6

HI,
I have some problem, when i create a zip file in server that have php version 5.5 and 5.6, the zip file is invalid. i already test in windows 10 and centos 7.2.
This issue never happen in php version 5.4.

ArchiveStream version 1.0.4

Thank you

Undefined property: $need_headers

I am seeing the following notice:

Notice: Undefined property: Barracuda\ArchiveStream\ZipArchive::$need_headers in /tmp/test/vendor/barracudanetworks/archivestream-php/src/Archive.php on line 323

Looking at the code, it doesn't look like the $need_headers property is being declared.

Thanks for your help.

PHP 5.6 above: Zip will be corrupted because it can't split a 64-bit integer to two 32-bit integers in int64_split function

Hi,

With PHP 5.6 and above consider GMP resources are objects. So need to change int64_split function in Archive.php file to make it work with PHP 5.6+
function int64_split($value)
{
.....
if (is_object($value)){ // php 5.6 above
....
// $high = $this->gmp_convert(substr($hex, 0, 8), 16, 10);
$high = gmp_strval(gmp_init(substr($hex, 0, 8), 16), 10);
// $low = $this->gmp_convert(substr($hex, 8, 8), 16, 10);
$low = gmp_strval(gmp_init(substr($hex, 8, 8), 16), 10);
}
.....
}
I have tested on php 5.6 and php 7.0.4 and it works fine. If php 5.5 below can use the old code

Regards,
Trung.

How to zip file from a url ?

Hello,

I would like to zip file from a remote server and to download from the browser but I have no idea how to acheive that. I saw the issue #34 but I can't make it work, I need to give credentials.
Would be awseome if it works.

Can you download from multiple pages simultaneously?

I have problems with downloads of multiple files from same site.
When I try to open in browser 2 tabs (with pages from same site), that offer streaming download, I can download only from the first tab. When I switch to second tab and try to start new download there as well it dies. If I try to open any other page from same site in the seconds tab it dies as well, telling me the page is unreachable. It works just fine, if I try to visit the same page in another browser.
Repro on: Windows 10, Chrome, Edge

  1. Is it not possible to download 2 files at the same time, using this lib, or is it some Windows only limitation?
  2. Why is the whole site unreachable in the browser that is downloading? Is there something I'm missing?

Undefined offset: 8 in ZipArchive.php

Undefined offset: 8 in ZipArchive.php

When I run the following:

$zip = \Barracuda\ArchiveStream\Archive::instance_by_useragent('example');

$zip->add_file('hello.txt', 'This is the contents of hello.txt');

$zip->finish();

I'm seeing an error:

Notice:  Undefined offset: 8 in /tmp/test/vendor/barracudanetworks/archivestream-php/src/ZipArchive.php on line 292

Looking at the code, it doesn't look like the $file_attribute variable is being added onto the $files array. Should that be an argument to add_to_cdr?

Thanks for your help.

Add a way to not disable output buffering

The Archive constructor calls ob_end_clean():

ob_end_clean();

I get why you do this but it causes problems with phpunit tests (because it uses output buffering to know what the tested code outputted):

Test code or tested code did not (only) close its own output buffers

Would it be possible to add an argument to not disable output buffering?

Support request loop over files in folder

Can someone help me out and tell me what is wrong with following code?
The .tar file being downloaded is only 14kb while the "files" folder contains a video of 800MB.

`<?php
include 'vendor/autoload.php';

// Create a new archive stream object (tar or zip depending on user agent)
$zip = \Barracuda\ArchiveStream\Archive::instance_by_useragent('example');

$directory = 'files';
$files = array_diff(scandir($directory), array('..', '.', '.DS_Store'));

foreach ($files as $file) {
	$fileSize = filesize($directory."/".$file);
	$filePath = $directory."/".$file;
	$zip->init_file_stream_transfer($filePath, $fileSize);
	$zip->stream_file_part($data);
	$zip->complete_file_stream();
}

// Finish the zip stream
$zip->finish();

?>`

Build an archive from a stream

Hello,

I need to build tarballs from streams on my server (coming from HTTP requests or processes).
Do you think it would be possible to have a function that takes a stream as an argument instead of a filename, so I could do something like this:

$zip->add_stream(popen('echo foo'));

file corrupt

hi,
i have encrypted file,
while download multiple file, before add to zip i will decrypt the file. I have 3 file, i sucessfully created the zip, but only one file can be read other are corrupted
apache version 2.4
php 5.5
windows 10

foreach ($fileid as $key => $value) {
$file_info      = $this->CI->pd_files_mdl->getFileById( $value );
            $zip->init_file_stream_transfer($file_info->name, $file_info->unencrypted_size);
$stream = fopen($userStorage.'/'.$file_info->path,'r');
            if ( $file_info->encrypted == 1 ){
                while (!feof($stream)) {
                    $buffer = fread($stream, 1024 * 1024);
                    $data = $aesCipher->decrypt($buffer);
                                $zip->stream_file_part($data);
                  } 
                $zip->complete_file_stream();   
                        }
}

        $zip->finish();

PCLZIP_ERR_BAD_FORMAT (-10)

I have implemented ArchiveStream as part of an update system for my wordpress plugin because the name of the plugin folder varies from user to user. When uploading a .zip file created by the plugin manually to WP it installs fine however if you try to update the plugin via Wordpress' built in update API, it presents this error:

The package could not be installed. PCLZIP_ERR_BAD_FORMAT (-10) : Unable to find End of Central Dir Record signature

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.