nodecraft / b2-cloud-storage

Backblaze B2 Cloud Storage API Client. Implements all of the B2 Cloud Storage APIs, with helper methods for uploading files.

b2-cloud-storage

Backblaze B2 Cloud Storage API Client

b2-cloud-storage is an API wrapper for all current Backblaze B2 API operations. It provides helper methods for uploading files of all sizes, and takes care of the necessary chunking, parting, and API retries that need to happen to ensure files are uploaded successfully.

This module adheres the integration guidelines as published by Backblaze at https://www.backblaze.com/b2/docs/integration_checklist.html.

Basic Example

'use strict';
const b2CloudStorage = require('b2-cloud-storage');

const b2 = new b2CloudStorage({
	auth: {
		accountId: '<accountId>', // NOTE: This is the accountId unique to the key
		applicationKey: '<applicationKey>'
	}
});

b2.authorize(function(err){
	if(err){ throw err; }

	// this function wraps both a normal upload AND a large file upload
	b2.uploadFile('/path/to/file.zip', {
		bucketId: '<bucketId>',
		fileName: 'file.zip', // this is the object storage "key". Can include a full path
		contentType: 'application/zip',
		onUploadProgress: function(update){
			console.log(`Progress: ${update.percent}% (${update.bytesDispatched}/${update.bytesTotal}`);
			// output: Progress: 9% 9012/100024
		}
	}, function(err, results){
		// handle callback
	});
});

Documentation

You can read the full documentation here.

Roadmap:

• Improve unit tests and code coverage
• Add helper methods to delete all file versions for a single file
• Add helper methods to better query paginated search

License

b2-cloud-storage's Issues

Add helper methods for better query paginated search

As title.

Issues with both download file functions

There is a couple problems with the download file functions, currently writing unit tests and didn't want to combine the PR's. I will be making a PR to fix these

• They require the downloadUrl from the authorize function but that isn't stored. This means apiUrl needs to be changed
• They need to have appendPath set to false
• Headers for downloadFileById aren't being set
• data.url for request gets overridden

TypeError: url.URL is not a constructor

Hello, i'm using b2-cloud-storage npm package in my react-website，and run code like your Basic Example，but it's not work. how can i resolve this ? Thanks !

An in-range update of eslint-config-nodecraft is breaking the build 🚨

The devDependency eslint-config-nodecraft was updated from 2.0.1 to 2.1.0.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

eslint-config-nodecraft is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details

• ❌ Node 12 Test: There are 1 failures, 0 warnings, and 0 notices.
• ❌ Node 10 Test: There are 2 failures, 0 warnings, and 0 notices.

---

Commits

The new version differs by commits.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.

---

Your Greenkeeper Bot 🌴

An in-range update of eslint-config-nodecraft is breaking the build 🚨

The devDependency eslint-config-nodecraft was updated from 1.7.4 to 1.7.5.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

eslint-config-nodecraft is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details

• ❌ Node 10 Test: There are 1 failures, 0 warnings, and 0 notices.
• ❌ Node 8 Test: There are 1 failures, 0 warnings, and 0 notices.

---

Commits

The new version differs by 1 commits.

• 7b89370 1.7.5 release

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.

---

Your Greenkeeper Bot 🌴

b2_copy_file not implemented

Backblaze recently implemented the b2_copy_file method. This is not yet implemented.

Documentation: https://www.backblaze.com/b2/docs/b2_copy_file.html

See announcement here: https://www.backblaze.com/blog/backblaze-b2-copy-file-beta-is-now-public/

Handle 503/service_unavailable errors

Occasionally when uploading files, even after an upload URL has been retrieved, B2 can respond with the following when attempting to upload:

{
  "code": "service_unavailable",
  "message": "c001_v0001113_t0035 is too busy",
  "status": 503
}

This is by design, and should trigger this library to attempt to get a new upload URL and retry.

We handle this more gracefully (-ish) in uploadFileSmall, but do not handle it at all in uploadFileLarge. For uploadFileSmall, we should be requesting a new upload URL on a retry, rather than just uploading again to the "too busy" URL. For uploadFileLarge, this check would would probably make sense around line 1402 of index.js, replacing the upload URL we just used and got an error from, with a new one.

Testing should be done as per https://www.backblaze.com/b2/docs/integration_checklist.html. A header can be set to simulate artificial errors.

No warning when non-authorized

If you perform an upload file request without authorization it returns a Javascript error rather than user error.

const config = {...};
const b2CloudStorage = require('b2-cloud-storage');
const b2 = new b2CloudStorage(config);
b2.uploadFile('filename', {...}, (err) => {...});

Error: TypeError: Cannot read property 'recommendedPartSize' of undefined.

Add support for Promises, with async/await

This library is pretty mature now and works really well, but still uses callbacks. There's nothing wrong with callbacks, but a lot of modern code nowadays uses promises via async/await, and having this library conform to that would be ideal.

Things to think about:

• Should we bump a major version and just remove callbacks entirely?
• Should we try to support both callbacks and promises?

b2_copy_file limited to 5GB

As per discussion with backblaze, b2_copy_file is limited to 5GB. We need add a helper method or similar to detect the file size, and then use b2_copy_part if necessary.

Add helper methods to delete all file versions for a single file

As title.

Drop dependency on `request`

request is now unmaintained. It still works really well and is actively used, but when we resolve #30, we should transition to something like node-fetch that uses promises.

Improve code coverage

Test coverage is currently pretty low at around 20%: https://coveralls.io/github/nodecraft/b2-cloud-storage

More tests and extended tests are required.

b2_upload_file/b2_copy_file should cleanup already uploaded parts

As title. If an upload/copy is cancelled, the already uploaded/copied parts aren't cleaned up, leaving dangling file parts in b2.