coryrwest / b2.net Goto Github PK

Occurrence:
IFiles.DownloadByName
Files.ParseDownloadResponse
Utilities.CheckForErrors

      try
      {
        b2Error = JsonConvert.DeserializeObject<Utilities.B2Error>(result);
      }

When downloading a file with a slash in name, e.g. my/image/2.jpeg, B2.NET will urlencode it producing my%2Fimage%2F2.jpeg.

Backblaze will not accept this and return the following HTML page:

B2.NET sees the status code as failure and attempts to parse the page as JSON, resulting in a JSON parsing exception.

to display the file in the browser after get download authorization the only way is to pass via get? Authorization? = ... or we were able to send via header authorization

since now! Thanks

Congratulations fantastic library.
When i upload a file "test_1.zip" with 6335KB in size, with B2Client.Files.Upload() the file is in Backblze B2 with the same size, but if i do it with LargeFileUploadTest() the file is in B2 with approximately double the size and corrupted.
Any suggestion?

Hello!

Thank you for this great library! It is epic, I am writing a bit of software for the B2 team, and will use this library for it.

I see on my B2 Dashboard, I can view the total count of all files within a bucket:

Reviewing the B2 CLI API, I see we can view the total size of a Bucket:
See here

Reviewing the B2 Http API, I do not see an option to get the count of all files within a bucket:
See here

Is it possible to get the count of all files within a bucket?

Basically I want to add a progress bar, as retrieving files can take a long time, even using batches of 1000.

Thanks again!

Nuget lists the package as compatible with Newtonsoft.Json >= v9.0, but on calling B2Client.Authorize the following exception is thrown:

System.IO.FileNotFoundException: Could not load file or assembly 'Newtonsoft.Json, Version=9.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed' or one of its dependencies. The system cannot find the file specified.
File name: 'Newtonsoft.Json, Version=9.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed'
   at B2Net.B2Client.Authorize(B2Options options)
   at B2Net.B2Client..ctor(String keyId, String applicationkey, Int32 requestTimeout)
...

Am I missing something, or would it be possible to make it compatible with a newer version of the dependency?

I've been working with B2.NET C# Client for some time with some success, but lately my application stopped working for no apparent reason.

I verified that whenever I created the B2 client
var client = new B2Client(_b2AccountId, _b2ApplicationKey);
It raise the error "The request was aborted: Could not create SSL/TLS secure channel."

The solution I found to get the application working was to put the following code before creating the client:

ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls
| SecurityProtocolType.Tls11
| SecurityProtocolType.Tls12
| SecurityProtocolType.Ssl3;

I hope it helps others with the same problem, however I would like to know if it would not be possible to embed this code snippet in the B2.NET?

Thanks

There are certain error messages we get from the B2 api that point to a resolvable issue (such as needing a new uploadUrl).

Errors that indicate that we need a new uploadUrl

• Request Timeout (408)
• Any 5xx error
• Too Many Requests (429)
• Unauthorized (401) and an error code of expired_auth_token
• Broken pipe

When one of the above errors happens do the following internally:

• Get new UploadUrl or get new Auth Token
• Try upload again

Currently, file upload is handled with, at minimum

var client = new B2Client("ACCOUNTID", "APPLICATIONKEY");
var uploadUrl = await client.Files.GetUploadUrl("BUCKETID");
var file = await client.Files.Upload("FILEDATABYTES", "FILENAME", uploadUrl)

but the B2Client constructor can take "BUCKETID" that is required for constructing uploadUrl.

Could it not be handled internally, so that the upload can be done with

var options = new B2Options() {
	AccountId = "YOUR ACCOUNT ID",
	KeyId = "YOUR APPLICATION KEYID",
	ApplicationKey = "YOUR APPLICATION KEY",
	BucketId = "OPTIONAL BUCKET ID",
	PersistBucket = true/false
};
var client = new B2Client(Options);
var file = await client.Files.Upload("FILEDATABYTES", "FILENAME")

?

That would also make it friendlier for dependency injection in ASP.NET, since only client would need to be passed, no need to pass any options or config.

Alternatively, maybe something like

var options = new B2Options() {
	AccountId = "YOUR ACCOUNT ID",
	KeyId = "YOUR APPLICATION KEYID",
	ApplicationKey = "YOUR APPLICATION KEY",
	BucketId = "OPTIONAL BUCKET ID",
	PersistBucket = true/false
};
var client = new B2Client(Options);
var uploadUrl = await client.Files.GetUploadUrl(client.options.BucketId);
var file = await client.Files.Upload("FILEDATABYTES", "FILENAME", uploadUrl)

might also do, but I see no point since it can be handled internally.

Hello,

I just wanted to know if there was any plan to add support for the key APIs - listKeys, writeKeys and deleteKeys?

Time permitting, I think we at @LogicSoftInd would be happy to send in a patch for the same

We can build and run the app with B2.net package in both Debug and Release mode. However, when publishing the app, VS shows a bunch of errors such as:

C:\Projects\Test.API\Test.API.csproj(0,0): Error NU1605: Warning As Error: Detected package downgrade: System.IO.FileSystem.Primitives from 4.3.0 to 4.0.1. Reference the package directly from the project to select a different version.
Test.API -> Test.Infrastructure -> B2Net 0.7.5 -> NETStandard.Library 1.6.0 -> System.Console 4.0.0 -> runtime.win.System.Console 4.3.0 -> System.IO.FileSystem.Primitives (>= 4.3.0)
Test.API -> Test.Infrastructure -> B2Net 0.7.5 -> NETStandard.Library 1.6.0 -> System.IO.FileSystem.Primitives (>= 4.0.1)
C:\Projects\Test.API\Test.API.csproj(0,0): Error NU1605: Warning As Error: Detected package downgrade: System.IO.FileSystem.Primitives from 4.3.0 to 4.0.1. Reference the package directly from the project to select a different version.
Test.API -> Test.Infrastructure -> B2Net 0.7.5 -> NETStandard.Library 1.6.0 -> System.IO.FileSystem 4.0.1 -> runtime.win.System.IO.FileSystem 4.3.0 -> System.IO.FileSystem.Primitives (>= 4.3.0)
Test.API -> Test.Infrastructure -> B2Net 0.7.5 -> NETStandard.Library 1.6.0 -> System.IO.FileSystem.Primitives (>= 4.0.1)

These packages are not referenced directly in the project so we can manually change the version. Any suggestions on how to resolve this problem? Thanks.

I'm having some difficulty uploading large files, with issues such as energie breaks, computer shutdown before upload completes, etc...

I'm thinking making some changes in my upload method by doing the following. Instead of putting the parts in a List<byte []>, save all the parts in a directory into text files, after that then began to upload each of the parts, deleting .txt part files after successful upload.

So in case the pc was disconnected i could restart the process sent only the missing parts until all parts are uploaded and then Finish the file in Backblase B2 server.

I would like your opinion on whether this process could work or will be impossible.

Other opinions to solve this problem are welcome

Hi!

Firstly Thanks for you work.

Could you support .NET 4.5 and higher (I have an ASP.NET MVC project working with 4.5.2 for example).

For the moment, I can't install your nuget package into a project targeting .NET 4.5.2.

Thanks in advance !

Best regards.

Antoine.

Is it possible to add a parameter for Content-Type when uploading files?

The problem is a simple typo here: that line should reference b2ContentDisposition, not maxFileCount.

hello i try to follow the code upload large file size and it possible to upload file not over more than 1 GB but my point is i want to upload file over than 15 GB per file This is my code

List<byte[]> parts = new List<byte[]>();

using (var stream = System.IO.File.Open(Path.Combine(@"D:\research\2019\Pandora\Pandora_Api\Pandora_Api\video\", items.itemsbucketIds,Path.GetFileNameWithoutExtension(items.fileName), filebuffers.FileName), FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
	stream.Seek(0, SeekOrigin.Begin);

	// Read and verify the data.
	byte[] buffer = new byte[100 * (1000 * 1000)];
	for (int i = 0; i < stream.Length; i++)
	{
		Console.WriteLine(i +"/"+ stream.Length);

		// Resize the byte[] array with the size of the remainder of the stream
		if (buffer.Length > stream.Length - i )
		{
			Console.WriteLine(stream.Length);
			buffer = new byte[stream.Length - i];
		}

		stream.Read(buffer, 0, buffer.Length);
		i += buffer.Length;

		parts.Add(buffer);
		Thread.Sleep(1000);
	}
	Array.Clear(buffer, 0, buffer.Length);
}
var shas = new List<string>();
foreach (var part in parts)
{
	string hash = Utilities.GetSHA1Hash(part);
	shas.Add(hash);
}

B2File start = null;
B2File finish = null;
try
{
	start = await client.LargeFiles.StartLargeFile(Path.GetFileName(filebuffers.FileName), "", items.itemsbucketIds);
	for (int i = 0; i < parts.Count; i++)
	{
		var uploadUrl = await client.LargeFiles.GetUploadPartUrl(start.FileId);
		var partResult = await client.LargeFiles.UploadPart(parts[i], i + 1, uploadUrl);
		Thread.Sleep(2000);
	}
	finish = await client.LargeFiles.FinishLargeFile(start.FileId, shas.ToArray());
}
catch (Exception e)
{
	return BadRequest(e);
}
return Ok(finish); 

Hi @coryrwest,

I've just updated my B2.NET package from 0.0.4.6 to the current 0.5.3 version and I encounter an "internal error" when I'm trying to upload a file.

My code:

this.options = new B2Options()
{
	AccountId = accountId,
	ApplicationKey = applicationKey,
	BucketId = bucketId,
	PersistBucket = true
};

this.client = new B2Client(B2Client.Authorize(this.options));

var uploadUrl = await client.Files.GetUploadUrl(this.options.BucketId);

//Method parameters: string fileName, byte[] data
//THE EXCEPTION OCCURED HERE
var file = await client.Files.Upload(data, fileName, uploadUrl, false, this.options.BucketId);

The error:

B2Exception {"internal error"} Status 500

B2Net.Utilities.CheckForErrors(HttpResponseMessage response)\r\n à B2Net.Http.ResponseParser.<ParseResponse>d__01.MoveNext()\r\n--- Fin de la trace de la pile à partir de l'emplacement précédent au niveau duquel l'exception a été levée ---\r\n à System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n à System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n à System.Runtime.CompilerServices.TaskAwaiter1.GetResult()\r\n à B2Net.Files.<Upload>d__11.MoveNext()\r\n--- Fin de la trace de la pile à partir de l'emplacement précédent au niveau duquel l'exception a été levée ---\r\n à System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n à System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n à System.Runtime.CompilerServices.TaskAwaiter1.GetResult()\r\n`

Any idea?

Thank you for your help.

Best regards,

Antoine.

Sending the same HTTP message instance is exactly what the retry logic attempts to do:

Stack trace:

System.InvalidOperationException: The request message was already sent. Cannot send the same request message multiple times.
   at System.Net.Http.HttpClient.CheckRequestMessage(HttpRequestMessage request)
   at System.Net.Http.HttpClient.SendAsync(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at B2Net.Files.Upload(Byte[] fileData, String fileName, B2UploadUrl uploadUrl, String contentType, Boolean autoRetry, String bucketId, Dictionary`2 fileInfo, CancellationToken cancelToken)
   at B2Net.Files.Upload(Byte[] fileData, String fileName, B2UploadUrl uploadUrl, Boolean autoRetry, String bucketId, Dictionary`2 fileInfo, CancellationToken cancelToken)

Good work on the api, thanks.

Would you make httpclient's timeout a configurable parameter? It times out after the default 100 seconds.

The function download by id dont start, Could you give me an example of how do I use the function?
Please!

System.AggregateException
HResult=0x80131500
Message=Er zijn één of meer fouten opgetreden.
Source=mscorlib
StackTrace:
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at System.Threading.Tasks.Task.Wait()
at LaundryStatistiek.archiver.Archiveer.Go() in D:\SOFT\Insal\HANDSFREE\BetaalAutomaat\BEZETMELDER_soft\LaundryStatistiek\archiver\Archiveer.cs:line 46
at LaundryStatistiek.Program.Main(String[] args) in D:\SOFT\Insal\HANDSFREE\BetaalAutomaat\BEZETMELDER_soft\LaundryStatistiek\Program.cs:line 33
With a 1 bucket - WriteOnly application key, i have these exceptions
Changing the key with an all-access key (write/read,all buckets) everything works.

Inner Exception 1:
AuthorizationException: There was an error during authorization. See inner exception for details.

Inner Exception 2:
Exception: If you are using an Application key and not a Master key, make sure that you are supplying the Key ID and Key Value for that Application Key. Do not mix your Account ID with your Application Key.

I'm guessing it is not because of things like this:

It would be nice if it were since the underlying HttpClient is, but if that requires too much effort, perhaps a line in the documentation clarifying the library's thread safety would be helpful.

hi cory ,
first i would like to say thank you so much - great job man !!!
there is a request about the delimiter .
if i want to make a request from a bucket - only folders or some different hierarchy and not all the hierarchy i have to use a delimiter "/".
i saw that you didn't implement this issue.

backblaze related that issue
https://www.backblaze.com/b2/docs/b2_list_file_names.html

is there any chance that you will add it soon.. very soon :)

I'm trying out the library and the first thing I did was upload a file to the bucket manually (called bg.jpg) and then tried to use the library in code to get a download authorization token. I have the following code:

var result = await _b2.Files.GetDownloadAuthorization(
    "bg.jpg",
    30,
    Environment.GetEnvironmentVariable("B2_BUCKET_ID"),
    ""
);

The environment variable has my numeric bucket ID that was randomly assigned when I created the bucket.

When I run this code, it throws an ArgumentException which looks like it comes from the HTTP libraries internal to .NET Core. The message inside is "Only 'http' and 'https' schemes are allowed.\nParameter name: value". I was wondering if anyone could help me with this? I don't see any spot in the API of the library to indicate protocol, so I'm confused as to why it's complaining about this.

Platform:
OS: Antergos
.NET Core SDK: 2.1.301

Contents of csproj file:

<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>netcoreapp2.1</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="B2Net" Version="0.5.3.1" />
    <PackageReference Include="Microsoft.AspNetCore.App" />
  </ItemGroup>

</Project>

Per B2 Lifecycel Rules (https://www.backblaze.com/b2/docs/lifecycle_rules.html) DaysFromUploadingToHiding should be nullable.

According to the b2_authorize_account API documentation, the returned authorization token is valid for "at most 24 hours". However, the B2Client instance lifetime could exceed this (e.g. if it is created as a singleton in a long-running application). While this could be handled at the application level, it would be nice if B2Client could automatically detect an expired token and generate a new one.

Allow uploading of streams to B2 with SHA1 check at the end.

This is a similar issue (#17) to the one I created not long ago. I'm getting the same error message but the situation is different. For a bit of context here, Backblaze now officially supports application keys and have launched the application key API endpoints. The term "master application key" is used to refer to what used to be called the "master key".

I have no problem using the B2Client methods if I Authorize using my master application key, but if I use any other application key, ones that I create specifically for my applications to use, I get this error message from an inner exception. I tried creating application keys with and without expiry, each with full read and write access to the one bucket I was working with, and the problem still occurred. I tried specifying the bucket ID (the same one the key has full access to) both as I Authorized the B2Client and as I performed the operations, like GetUploadUrl. Problem still occurred.

I was wondering if this could be related to the library not having support for application keys (in fact I remember reading that the author has no intention of supporting those features). But this problem does confuse me a bit since these application keys should be the same as the keys used before, as far as I know, just with limited power.

Error:
I got this error when uploading the file.

An unhandled exception occurred while processing the request.

FormatException: The format of value "4_001a40cddf2c4b70000000000_0184e5f8_da6677_acct_--ObgNwZn8YYuv-rPa7cqujc1QU=" is invalid.

It seems that value is the authorization token.

Step to produce:
_options = options;
_client = new B2Client(B2Client.Authorize(_options));
B2File result = await _client.Files.Upload(data, Guid.NewGuid().ToString() + "." + extension, _options.BucketId);

Version:
ASP.NET Core 2.0
B2NET 0.5.2

I have a service that uploads a 10mb file to object storage. It does this every 3 minutes. Typically, an upload takes anywhere between 4-15 seconds. The uploads work well for a good part of the day but every so often it would fail with this exception. I have seen this happen for the past 6 days.

I haven't gone into the code to check. Maybe you know something about this?

problem with bigfile that all bytes in memory
httpclient can write to stream directly

When uploading a file the request must include a content-length header which consist of the length of the file + the length of the 40 bytes SHA1. But it seem to me the code only sent the length of the file and doesn't include the length of the SHA1.

FileUploadRequestGenerators.cs, link 53

B2 API documentation

Or am I just not capable to reading the code :-/

If would be great to be able to update corsRules via the Buckets.Update method, since this is not possible through b2's Web GUI.

hi cory,
Happy new year man.
i was wondring i am trying to list the buckets with an application key and not the master application key but i am getting an error.
i am adding the response i am getting

waiting for you response

thanks and happy new year

B2Net error when no master application key.txt

There may be scenarios where the B2Client will be created during a time period where there shouldn't be any HTTP requests being created.

Ideally, the B2Client.Authorize wouldn't be called until either the first operation, the first access of B2Client.Capabilities, or manually called.

In the event that the Authorize method is called via the first operation it should be done fully async, rather than relying on the .Result property of the task.

As an aside, using task.Result is a blocking call, and you use this in Files (via Utilities), Bucket, Authorization, and LargeFiles... This means that the library is not "fully async" as per

In anycase, thank you for the library!

Simply adding this line:

<PackageReference Include="B2Net" Version="0.7.5"/>

to my .csproj file causes all of my frontend api requests to fail:

It seems to be messing with the cors policy.