octopusdeploy / octodiff Goto Github PK

SignatureReader for instance, it makes use of Length and Position of stream. If the stream is a NetworkStream it would throw an exception. What i did to support it, is to send an empty end chunk in the client and create a copy of the SignatureReader class but removing that lines using position and length, and using the last chunk as the end of stream. BinaryDeltaReader does something like this also.

Attempting to open the project in VS 2017 community edition gives errors.

Octodiff\Octodiff.xproj: Failed to migrate XProj project Octodiff. 'dotnet migrate --skip-backup -s -x "...\Octodiff\source\Octodiff\Octodiff.xproj" "...\Octodiff\source\Octodiff\project.json" -r "...\AppData\Local\Temp\gup2fn3o.mk3" --format-report-file-json' exited with error code 1.

Octodiff\Octodiff.xproj: Migration failed. System.ArgumentNullException: Value cannot be null. Parameter name: version at NuGet.Versioning.VersionRangeBase.Satisfies(NuGetVersion version, IVersionComparer comparer) at NuGet.Versioning.VersionRangeBase.Satisfies(NuGetVersion version) at Microsoft.DotNet.ProjectJsonMigration.Rules.MigratePackageDependenciesAndToolsRule.<>c__DisplayClass11_0.b__2(VersionRange p) at System.Linq.Enumerable.TryGetFirst[TSource](IEnumerable1 source, Func2 predicate, Boolean& found) at System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable1 source, Func2 predicate) at Microsoft.DotNet.ProjectJsonMigration.Rules.MigratePackageDependenciesAndToolsRule.ToPackageDependencyInfo(ProjectLibraryDependency dependency, IDictionary2 dependencyToVersionMap) at Microsoft.DotNet.ProjectJsonMigration.Rules.MigratePackageDependenciesAndToolsRule.MigrateDependencies(Project project, MigrationRuleInputs migrationRuleInputs, NuGetFramework framework, IEnumerable1 dependencies, SlnFile solutionFile, ProjectItemGroupElement itemGroup) at Microsoft.DotNet.ProjectJsonMigration.Rules.MigratePackageDependenciesAndToolsRule.Apply(MigrationSettings migrationSettings, MigrationRuleInputs migrationRuleInputs) at Microsoft.DotNet.ProjectJsonMigration.DefaultMigrationRuleSet.Apply(MigrationSettings migrationSettings, MigrationRuleInputs migrationRuleInputs) at Microsoft.DotNet.ProjectJsonMigration.ProjectMigrator.MigrateProject(MigrationSettings migrationSettings) at Microsoft.DotNet.ProjectJsonMigration.ProjectMigrator.Migrate(MigrationSettings rootSettings, Boolean skipProjectReferences) at Microsoft.DotNet.Tools.MigrateCommand.MigrateCommand.Execute() at Microsoft.DotNet.Tools.Migrate.MigrateCommandCompose.Run(String[] args) at Microsoft.DotNet.Cli.Program.ProcessArgs(String[] args, ITelemetry telemetryClient) at Microsoft.DotNet.Cli.Program.Main(String[] args)

Hi there

Can we use in Unity3D for Standalone (Win, macOS) builds?

Hello!

I hope you are doing well!

We are a security research team. Our tool automatically detected a vulnerability in this repository. We want to disclose it responsibly. GitHub has a feature called Private vulnerability reporting, which enables security research to privately disclose a vulnerability. Unfortunately, it is not enabled for this repository.

Can you enable it, so that we can report it?

Thanks in advance!

PS: you can read about how to enable private vulnerability reporting here: https://docs.github.com/en/code-security/security-advisories/repository-security-advisories/configuring-private-vulnerability-reporting-for-a-repository

Hello,

I am looking into calling Octodiff from within our C# application.

The README explains how to use the command line utility and I have been able to successful execute the library by calling the Main function of the program, like so:

Octodiff.Program.Main(new string[] { "signature", oldFilePath, outputPath });

I can see the types like Octodiff.Core.SignatureBuilder and Octodiff.Core.DeltaApplier, but would love some documentation on how to use it.

Line 42 casts length as an int. When length is greater that 2GB the cast converts this to a negative value and causes an exception when the byte array is instantiated.

Removing the cast, resolves the issue.

The README says:

inspired by rdiff, and like rdiff the algorithm is based on rsync

I was curious if I'd be able to sync files between rdiff (on *nix systems) and Octodiff on Windows. I tried it out and noticed there is definitely a superficial incompatibility - they use different magic numbers - but I wasn't sure if there was a deeper incompatibility.

Other than the magic numbers are there any major differences between octodiff and rdiff files? Do you think rdiff compatibility is possible in the future?

Hi Everyone,
In one of the projects, backend team use octodiff binary patch for csv files.

Now as frontend developer I want to send patch data in payload. Where can I find npm library that can generate this patch.

I have one more question, I have noticed one thing that is if I using a complex( a png file of 1 mb or a jpg file) file the patching is not working properly. I mean to say that its not creating the new file.

Can You please tell me why is this happening.
If you want I will share you the delta, signature and the image files so that we will able to overcome the issue and make the thing perfect

Hope to hear back from you soon

Line 52 in SignatureBuilder.cs in WriteMetadata(Stream stream, ISignatureWriter signatureWriter)
...
var hash = HashAlgorithm.ComputeHash(stream);
...

This line does not appear to have any effect. hash as passed to WriteMetaData, is not used. Removal of this line does not effect the resulting signature file/delta/patch files or the resulting patched files in my testing. This seems to calculate a value (that takes a very long time on large files) and then throw that value away. What is the purpose of this line?

When trying to reference the dll in my code I am unable to access the SignatureWriter class. Can we please get this class set as public in file ISignatureWriter.cs

I appreciate the support.

Hi i tried the example from Using OctoDiff classes within your own application section in a dotnet 6 console app using the Octopus.Octodiff Version 2.0.261 package. The new file is not updated with the delta. Any ideas?

Could we undo a file with octodiff that has been previously created if we have the diff files?

Hi,

In Adler32 algorithm https://en.wikipedia.org/wiki/Adler-32 the prime used in modulo operation equals 65521 (largest prime smaller than 65536).

A = 1 + D1 + D2 + ... + Dn (mod 65521)
B = (1 + D1) + (1 + D1 + D2) + ... + (1 + D1 + D2 + ... + Dn) (mod 65521)
   = n×D1 + (n−1)×D2 + (n−2)×D3 + ... + Dn + n (mod 65521)

In the sources Adler32RollingChecksum.cs you cast to ushort which effectively does a mod 65536. Is there any particular reason for that?

The zlib RFC https://tools.ietf.org/html/rfc1950 that also defines Adler32 states that

That 65521 is prime is important to avoid a possible large class of two-byte errors that leave the check unchanged.

Regards,
Grzegorz Blok

Description

When using the V2 rolling checksum algorithm, files that are identical or very slightly different result in huge deltas: the whole new file gets added as the delta.

Environment

• repo freshly cloned from the current master branch (commit d87ee31)
• VS2022 17.2.6 on Windows 10 x64

I had to make a small code change so the command line app would use the V2 algorithm by default:

diff --git a/source/Octodiff/Core/SupportedAlgorithms.cs b/source/Octodiff/Core/SupportedAlgorithms.cs
index 2cc2aa5..5552f13 100644
--- a/source/Octodiff/Core/SupportedAlgorithms.cs
+++ b/source/Octodiff/Core/SupportedAlgorithms.cs
@@ -52,7 +52,7 @@ namespace Octodiff.Core
 
         public virtual IRollingChecksum Default()
         {
-            return Adler32Rolling();
+            return Adler32Rolling(true);
         }
 
         public virtual IRollingChecksum Create(string algorithm)

Steps to reproduce

• grab a random binary file; my test was kernel32.dll from windows\system32
• create 2 copies of it: copy1.dll and copy2.dll
• modify copy2.dll very slightly; I simply changed the first byte from 'M' to 'A'
• run octodiff to create the deltas:
  • Octodiff.exe signature kernel32.dll signature.bin
  • Octodiff.exe delta signature.bin copy1.dll delta1.bin
  • Octodiff.exe delta signature.bin copy2.dll delta2.bin
• observe how the delta files are very "not delta-y"

Other notes

The V1 version of the algorithm does produce expectedly small delta files.

Also, it would be interesting to use the xxHash algorithm instead of SHA1 (SHA1 has an average speed of 0.28 GB/s, while xxHash has 5.4 GB/s).

https://cyan4973.github.io/xxHash/

Originally posted by @arabasso in #21 (comment)

For Octopus we would need to figure out how we get the two ends to agree on the algorithm to be used.

Hi While I am trying to load the project by sln file in Visual Studio 2017 but I am unable to do so as the projects are not loading .

Please help me with a version which will get loaded in visual studio

Octopus Deploy has recently reserved the Octopus prefix on NuGet. To avoid dependency confusion when referencing multiple NuGet package sources, this prefix is being appended to owned packages. Once Octopus.Octodiff has been published to NuGet, the existing non-prefixed Octodiff package will be marked as deprecated.

Migration will involve simply changing project package references from Octodiff to Octopus.Octodiff.

I'm curious to know if you think the library can generate a Signature file from a stream that does not support Seeking.

If possible this would be a great benefit to the library, as you can generate a signature in real time from a download, without storing the thie file itself.