Uploading file in chunks using range header about faraday-multipart

[iMacTia]

Hi @triplef, I'm afraid this might not be in the multipart middleware capabilities at the moment.
Just to understand, is the server expecting multiple individual requests (each on a separate connection), or is this supposed to be a streaming request (single connection reused for many requests)?

Also, I'll move this ticket under the new faraday-multipart repository since that's where the middleware lives now, you might get a few notifications from GitHub 😄

[triplef]

Thanks for your feedback! I’m not sure there’s a requirement regarding reusing the connection, I’m guessing it should work either way but I’d probably try to reuse first.

Do you happen to know of any other project that might be able to do this?

[iMacTia]

I mostly work with AWS and use their sdk when I need something plug-and-play. So I made a quick search and found that Azure offers something similar in Ruby: https://github.com/azure/azure-storage-ruby

It would make sense to use that instead of re-implementing it using Faraday. What do you think?

[triplef]

Yeah that makes a lot of sense and I looked into this, unfortunately that SDK doesn’t seem to have the necessary initializers to work with an externally provided URL for our use case (see Azure/azure-storage-ruby#211).

I’ll see if I can get more info there or find some other way. Thanks again!

[iMacTia]

I may be wrong, but I suspect they're simply lacking documentation around it. Looking at the BlobService code, I found a reference to a storage_sas_token. It seems like the initializer accepts many more options than what is shown in the README.md. Is that still not enough to achieve what you need? (Sorry, I'm just trying to help but I know close to zero about Azure 😅)

[iMacTia]

I forgot the link to the reference obviously 🤦 : https://github.com/Azure/azure-storage-ruby/blob/master/blob/lib/azure/storage/blob/blob_service.rb#L56

[triplef]

I still don’t see a way to give it a full URI (the URI seems to be always built from individual components, see blob_uri), but I found another solution: Setting a header like x-ms-version: 2021-02-12 increases the maximum blob size to 5 GiB, which is more than enough for our purposes.

So I ended up with the following:

filePart = Faraday::FilePart.new(filePath, 'application/octet-stream')
Faraday.new(headers: {
  'x-ms-blob-type': 'BlockBlob',
  'x-ms-version': '2021-02-12'
}) {|c|
  c.response :raise_error
  c.adapter Faraday.default_adapter
}.put(fileUploadUrl, filePart, {'Content-Length': filePart.size.to_s})

Thanks again for your help!

[iMacTia]

Thanks for coming back and sharing your solution! I'm sure that will help others in future ❤️