Chunked, pausable, recoverable uploading to Google Cloud Storage directly from the browser.
- User selects a file
- File + a Google Cloud Storage resumable upload URL are given to
gcs-browser-upload
- File is read in chunks
- A checksum of each chunk is stored in
localStorage
once succesfully uploaded - If the page is closed and re-opened for some reason, the upload can be resumed by passing the same file and URL back to
gcs-browser-upload
. The file will be validated against the stored chunk checksums to work out if the file is the same and where to resume from. - Once the resume index has been found,
gcs-browser-upload
will continue uploading from where it left off. - At any time, the
pause
method can be called to delay uploading the remaining chunks. The current chunk will be finished.unpause
can then be used to continue uploading the remaining chunks.
import Upload from 'gcs-browser-upload'
let input = document.getElementById('fileInput')
let pause = document.getElementbyId('pause')
let unpause = document.getElementbyId('unpause')
let upload = null
input.addEventListener('change', async () => {
upload = new Upload({
id: 'foo',
url: 'https://www.googleapis.com/..../....',
file: input.files[0],
onChunkUpload: (info) => {
console.log('Chunk uploaded', info)
}
})
try {
await upload.start()
console.log('Upload complete!')
} catch (e) {
console.log('Upload failed!', e)
} finally () {
upload = null
}
})
pause.addEventListener('click', () => {
if (upload) {
upload.pause()
}
})
unpause.addEventListener('click', () => {
if (upload) {
upload.unpause()
}
})
{
id: null, // required - a unique ID for the upload
url: null, // required - GCS resumable URL
file: null, // required - instance of File
chunkSize: 262144, // optional - chunk size must be a multiple of 262144
contentType: 'text/plain', // optional - content type of the file being uploaded
debug: false, // optional - if true, console will be used as a logger
onChunkUpload: () => {} // optional - a function that will be called with progress information
resumable: true, // optional - if true, calculates the checksum of every chunk which allows to resume upload after refreshing tab. May lead to browser crash with bigger files.
storage: window.localStorage, // optional - storage mechanism used to persist chunk meta data
}
Various errors are thrown if something goes wrong during uploading. See src/errors.js for the different types. These are exported as a property on the Upload
class.
make bootstrap // install dependencies
make test // run tests
make test-watch // continuously run tests
This project was created by the Engineering team at Qubit. As we use open source libraries, we make our projects public where possible.
We’re currently looking to grow our team, so if you’re a JavaScript engineer and keen on ES2016 React+Redux applications and Node micro services, why not get in touch? Work with like minded engineers in an environment that has fantastic perks, including an annual ski trip, yoga, a competitive foosball league, and copious amounts of yogurt.
Find more details on our Engineering site. Don’t have an up to date CV? Just link us your Github profile! Better yet, send us a pull request that improves this project.