ilanschnell / bsdiff4

binary diff and patch using the BSDIFF4-format

bsdiff4: binary diff and patch using the BSDIFF4-format

The code is mostly derived from cx_bsdiff (written by Anthony Tuininga, http://cx-bsdiff.sourceforge.net/). The cx_bsdiff code in turn was derived from bsdiff, the standalone utility produced for BSD which can be found at http://www.daemonology.net/bsdiff. In addition to the two functions (diff and patch) cx_bsdiff provides, this package includes:

• an interface to the BSDIFF4-format
• command line interfaces: bsdiff4 and bspatch4
• tests

The bsdiff4 package defines the following high level functions:

diff(src_bytes, dst_bytes) -> bytes

Return a BSDIFF4-format patch (from src_bytes to dst_bytes) as bytes.

patch(src_bytes, patch_bytes) -> bytes

Apply the BSDIFF4-format patch_bytes to src_bytes and return the bytes.

file_diff(src_path, dst_path, patch_path)

Write a BSDIFF4-format patch (from the file src_path to the file dst_path) to the file patch_path.

file_patch(src_path, dst_path, patch_path)

Apply the BSDIFF4-format file patch_path to the file src_path and write the result to the file dst_path.

file_patch_inplace(path, patch_path)

Apply the BSDIFF4-format file patch_path to the file path in place.

Example:

>>> import bsdiff4
>>> a = 100000 * b'a'
>>> b = bytearray(a)
>>> b[100:106] = b' diff '
>>> p = bsdiff4.diff(a, bytes(b))
>>> len(p)
154
>>> bsdiff4.patch(a, p) == b
True

bsdiff4's Issues

file_diff() can't handle large files?

Running version 1.2.4 installed via AUR package on Archlinux: https://aur.archlinux.org/packages/python-bsdiff4

When trying to run bsdiff4.file_diff on a large file (19GB) it sits for awhile, but crashes with:

  File "/usr/lib/python3.11/site-packages/bsdiff4/format.py", line 85, in file_diff
    dst = read_data(dst_path)
          ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/bsdiff4/format.py", line 64, in read_data
    data = fi.read()
           ^^^^^^^^^
OSError: [Errno 5] Input/output error

Seems to work fine on smaller files. Do I have to manually chunk large files myself?

license?

Please add a license file. ;-)

Is this code licensed under 2-clause BSD license?

Building wheels fails due to distutils incompatibility

Running

$ python setup.py bdist_wheel

produces error:

/usr/lib/python3.8/distutils/dist.py:274: UserWarning: Unknown distribution option: 'entry_points'
  warnings.warn(msg)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: invalid command 'bdist_wheel'

#17 Fixes this and produces viable binary wheels.
Tested on Python 3.8.5 and Python 2.7.18 on Windows and Linux

PY_SSIZE_T_CLEAN warning with python 3.8

I am getting this warning, does it ring a bell ?

  /projects/pix/src/pix/minibsdiff.py:197: DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats
    patchfile_data = bsdiff4.core.patch(oldfile_data, newlen,

generated patch size differs from result of bsdiff

Environment

• OS: Linux Debian 9
• Python module: bsdiff4 version 1.1.5
• bsdiff: Version 4.3-19

I used a image for testing, randomly deleted some bytes from the copy of that image, wanted to make a patch from the broken image to the original one.

Use python and bsdiff4 to write the patch bytes into a file, found its size is bigger than patch file generated from bsdiff command.

But using bspatch command to patch the modified image with python generated patch do works without problem.

Is this normal? thanks.

bsdiff4 cli in verbose mode exits with exception

Installed bsdiff4 from source, for python3.

The offending line:

bsdiff4/bsdiff4/cli.py

Line 35 in 01313ed

size = [getsize(args[i]) for i in xrange(3)]

I do not know much about python2 vs python3 compatibility, but using range instead of xrange seems to work for both versions.

Add LICENSE file to repo

Hi, would it be possible to add a LICENSE file to the repository?

I assume it will be BSD-2-Clause due to the heritage of bsdiff4, but a LICENSE file is how to make that unambiguous.

I am wanting to package this up as a dependency of PyUpdater.
packaging project at https://build.opensuse.org/package/show/home:jayvdb:py-new/python-bsdiff4

Function *diff* run indefinitely (4Mo files)

def read_data(path):
     fi = open(path, 'rb')
     data = fi.read()
     fi.close()
     return data

a = read_data("A")
b = read_data("B")

assert len(a) == 4202496
assert len(b) == 4202496

import bsdiff4
c = bsdiff4.diff(a, b)  # run indefinitely

file_A_and_B.zip

ValueError: Internal error - invalid parameters passed to libbzip2

Some versions of Python, in particular Python 3.3.0, have a known problem in the bz2 module when compressing an empty string, see: http://bugs.python.org/issue16828
When using bzdiff4, the bug create a traceback with the following last line:

ValueError: Internal error - invalid parameters passed to libbzip2

PyArg_ParseTuple `s#` doesn't use ints for size

Commit 35a390c began defining PY_SSIZE_T_CLEAN - the meaning of this macro in the API is as follows:

For all # variants of formats (s#, y#, etc.), the type of the length argument (int or Py_ssize_t) is controlled by defining the macro PY_SSIZE_T_CLEAN before including Python.h. If the macro was defined, length is a Py_ssize_t rather than an int. This behavior will change in a future Python version to only support Py_ssize_t and drop int support. It is best to always define PY_SSIZE_T_CLEAN.

It is sometimes the case that Py_ssize_t is a 64-bit type while int is a 32-bit type, which means that PyArg_ParseTuple clobbers the stack of the patch/diff functions.

Both int origDataLength, newDataLength // ... lines should be updated to Py_ssize_t instead.

PyUpdater installation error with setup.py bsdiff4

Hello I have an issue with downloading PyUpdater and it seems to be that the problem is with the setup.py install for bsdiff4. I used pip install PyUpdater on my VSCode terminal however I keep getting this error:

I am using python 3.9.1 version and I can not find any solutions to this problem. It seems like its having trouble with downloading bsdiff4 lib however I tried pip install bsdiff4 as well and ended up with the same output as in the screenshot. Any help would be appreciated

Segmentation fault when running bsdiff4 on smail files

This test case is simple and happens with versions 1.1.7 and 1.1.8
This test case pass with 1.1.6 and a patch is generated

$ bsdiff4 file.1 file.2 patch
Segmentation fault

the content of those files are :

$ hexdump -C file.1 
00000000  31 32 33 34 35 36 37 38  39 20 39 38 37 36 35 34  |123456789 987654|
00000010  33 32 31                                          |321|
00000013
$ hexdump -C file.2
00000000  31 32 33 34 35 36 37 38  39 30 30 30 39 38 37 36  |1234567890009876|
00000010  35 34 33 32 31                                    |54321|
00000015

Patching files in-place with file_patch does not work

See following:

>>> import bsdiff4
>>> with open("test1", "w") as f:
...     f.write("THIS IS A TEST.")
... 
>>> with open("test2", "w") as f:
...     f.write("THIS IS ANOTHER TEST")
... 
>>> bsdiff4.file_diff("test1", "test2", "patch")
>>> with open("test3", "w") as f:  # make test3 file
...     pass
... 
>>> bsdiff4.file_patch("test1", "test3", "patch")
>>> with open("test3", "r") as f:
...     print f.read()
... 
THIS IS ANOTHER TEST
>>> bsdiff4.file_patch("test1", "test1", "patch")
>>> with open("test1", "r") as f:
...     print f.read()
... 
[NUL][NUL][NUL][NUL][NUL][NUL][NUL][NUL][NUL]NOTHER TEST

Push 1.1.5 to PyPI

The package on PyPI is currently version 1.1.4.

I am running into the bug fixed in #5, and it would be ideal to be able to pull the latest off the package index.

It would be appreciated if this could be updated.

Using 1.1.2 on Windows environment fails to import unix only os.path module samefile

Hi

We recently ran into an issue with trying to update from 1.1.1 to 1.1.2.

We received the following exception when running. It appears like the usage of samefile has been included which is noted in os.path as a unix only module.

Traceback (most recent call last):
File "C:\Python27\lib\site-packages\mock.py", line 1607, in _inner
return f(_args, *_kw)
File "C:\Program Files\Jenkins\jobs\payroll (GERRIT)\workspace\src\wwlib\model\test_updater.py", line 413, in test_apply_patch
os.path.join(updater.frozen_updates_dir, 'TestProgram-2012.0.1.linux.from-2012.0.0.patch'))
File "C:\Program Files\Jenkins\jobs\payroll (GERRIT)\workspace\src\wwlib\model\updater.py", line 472, in apply_patch
bsdiff4.file_patch(source_filename, temporary_filename, patch_filename)
File "C:\Python27\lib\site-packages\bsdiff4\format.py", line 120, in file_patch
from os.path import isfile, samefile
ImportError: cannot import name samefile

Requires Admin to install on Windows

Installing bsdiff4 via pip install bsdiff4 on windows errors out, as it requires Microsoft Visual C++ 14.0 or greater, which can be installed through VS Build Tools, or any subsets of it. Installing VS Build Tools (or any superset of it) requires admin, which I do not have on my main development machine, and the script I am writing requires bsdiff4 and Python 3.10 (for match-case). While there are precompiled .whl files for versions up to 3.9 available here, they have no versions for Python 3.10. May I suggest that you supply precompiled .whl files in the releases section, so this can be installed without admin?