lancemccarthy / action-azureblobupload

A GitHub action to upload files to Azure Blob storage, compatible with all Actions runner operating systems.

Hi there 👋, my name is Lance, I'm a professional nerd.

Blog Twiter LinkedIn

• ⚡ Fun fact: I am also a Microsoft MVP, see https://bit.ly/LanceMVP
• 🔭 I’m currently working on cool .NET MAUI stuff
• 🌱 I’m currently learning Spanish
• 💬 Ask me about anything .NET, Hololens/Mixed Reality, IoT, Azure, MAUI, CI-CD
• 📫 How to reach me: Twitter is best. If you don't have twitter, shoot me en email... lance at dvlup dot com
• 😄 Pronouns: Him/He

Day Job

I am the Manager of Technical Support at Progress Software. My team and I provide technical assistance for developers using the Telerik and Kendo UI devtools. I absolutely love writing code and watching it come alive in the user's hands. My passion is mostly UI, but also the things that powers that UI... from backend APIs to AI-powered fun.

As an example, check out Flusher, the complete system that flushes the toilet for my cat using Windows IoT, SignalR, Azure Custom Vision and Xamarin Forms to Flush a Toilet.

GitHub Stats

action-azureblobupload's Issues

Possibility to use SAS token instead of connection string

Hi,
Would it be maybe possible to introduce possibility to use SAS Token instead of connection string?

Azure cli supports this

az storage blob upload --account-name $ACCOUNT --container-name $CONTAINER --type block --file delete-me --sas-token $SAS

Action is renaming uploaded files, not preserving original name

I have two files in a folder to upload to the root of an existing container (no sync, no cleaning). I can't figure out looking at the code why this action is renaming my files. Here's the debug output (masking some names with $ because these are customer product downloads).

As you can see from the following logs, it's replacing my filenames with timestamps (YYYYMMDD) and appending als to one of them. I can't see in your code where you're using a datestamp, so I'm assuming that's coming from my filename... it's almost like it's having issues with the underscore in my filename... but why?

##[debug]..=> '/home/runner/work/course-$$$$/course-$$$$'
##[debug]=> '/home/runner/work/course-$$$$/course-$$$$/../azblobupload'
##[debug]Result: '/home/runner/work/course-$$$$/course-$$$$/../azblobupload'
##[debug]Loading env
Run LanceMcCarthy/[email protected]
  with:
    connection_string: ***
    container_name: downloads
    source_folder: /home/runner/work/course-$$$$/course-$$$$/../azblobupload
    clean_destination_folder: false
    fail_if_source_empty: false
  env:
    UTILITY_RELATIVE_PATH: utils/build-materials-download
    CURRENT_DATESTAMP: 20200913
##[debug]sourceFolder: /home/runner/work/course-$$$$/course-$$$$/../azblobupload
##[debug]--- cleaned: /home/runner/work/course-$$$$/course-$$$$/../azblobupload
##[debug]localFilePath: /home/runner/work/course-$$$$/azblobupload/$$$$-$$$1_20200913.zip
##[debug]--- cleaned: /home/runner/work/course-$$$$/azblobupload/$$$$-$$$1_20200913.zip
##[debug]finalPath: als_20200913.zip...
##[debug]sourceFolder: /home/runner/work/course-$$$$/course-$$$$/../azblobupload
##[debug]--- cleaned: /home/runner/work/course-$$$$/course-$$$$/../azblobupload
##[debug]localFilePath: /home/runner/work/course-$$$$/azblobupload/$$$$-$$$2_20200913.zip
##[debug]--- cleaned: /home/runner/work/course-$$$$/azblobupload/$$$$-$$$2_20200913.zip
##[debug]finalPath: 20200913.zip...
Uploaded /home/runner/work/course-$$$$/azblobupload/$$$$-$$$1_20200913.zip to downloads/als_20200913.zip...
Uploaded /home/runner/work/course-$$$$/azblobupload/$$$$-$$$2_20200913.zip to downloads/20200913.zip...
##[debug]Node Action run completed with exit code 0
##[debug]Finishing: Upload ZIPs to Azure Storage Blob

Using 1.6, now I am losing the last character of the filename

Thanks for looking at the previous issue with it losing the first character of the filename... but now it's losing the last character.

My action:

env:
    AZURE_WEBAPP_PACKAGE_PATH: '.' 

- name: Copy test data to blob storage container        
  uses: LanceMcCarthy/[email protected]
  with:
    connection_string: XXX
    container_name: dataloader
    source_folder: '${{ env.AZURE_WEBAPP_PACKAGE_PATH }}/TestData'
    destination_folder: sample
    clean_destination_folder: true

I have a file under the TestData folder named "Xx-Yyy-Z Data.xlsx"

It uploads to the container, but the name is "Xx-Yyy-Z Data.xls" - missing the final "x".

Node.js 16 actions are deprecated

Recently GitHub started to warn about usage of Node.js 16 actions in workflows:

Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: thollander/actions-comment-pull-request@v2. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.

No_Name Folder Created in Blob container

when instructing the AzureblobSDK to upload a file to the blob container without a root folder, it creates an empty folder:

Usage example should use v1.8.0 instead

Hello, thank you for creating this action.

I noticed that I had to use v1.8.0 instead of v1.8 as the release and example indicates, otherwise the Github Action runner will fail:

Double Forward Slashes Persist

In special circumstances where the double forward slash was at the end of a string segment, this is not removed by normalize or custom regex checks.

Skip blobs already present in destination?

I would like to specify that the action should not overwrite blobs that are already present in the destination container. Would that be achievable?

retry handling?

Hi 👋
I've been experimenting with this extension.
It seems to struggle when lots of files need to be uploaded eventually throwing lots of

UnhandledPromiseRejectionWarning: FetchError: request to <redacted> failed, reason: connect ETIMEDOUT 52.239.221.226:443
    at ClientRequest.<anonymous> (/home/runner/work/_actions/LanceMcCarthy/Action-AzureBlobUpload/v2/webpack:/action-azure-upload-blob/node_modules/node-fetch/lib/index.js:1491:1)
    at ClientRequest.emit (events.js:314:20)
    at TLSSocket.socketErrorListener (_http_client.js:427:9)
    at TLSSocket.emit (events.js:314:20)
    at emitErrorNT (internal/streams/destroy.js:92:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:60:3)
    at processTicksAndRejections (internal/process/task_queues.js:84:21)
(node:1942) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 11196)

I believe this extension would need some kind of generic retry handling with an exponential backoff to cool down when running hot on the service side.
Additionally, and that part might need to be fixed in the storage client, the client might be establishing one new TCP connection per file upload, which is a recipe to get port exhaustion on the client side.

If no destination_folder specified, it creates one as "no name" and truncates the first character of the filename

I have a storage container named "testdata" and in my project I have a folder called "TestData" with an Excel File (test.xlsx). I want the test.xlsx file to go in the "root" of the "testdata" container. So I omitted the "destination_folder".

I used this:

    - name: Copy test data to blob storage container        
      uses: LanceMcCarthy/[email protected]
      with:
        connection_string: XXX
        container_name: testdata
        source_folder: '${{ env.AZURE_WEBAPP_PACKAGE_PATH }}/TestAPI/TestData'
        clean_destination_folder: true
        fail_if_source_empty: true  

It ended up creating a folder that shows as

<no name>

and creates the "test.xlsx" file as "est.xlsx", with the initial "t" missing.

I also tried using:

destination_folder: /

but that does the same thing.

Intermittent files missing in Azure Storage Blob after upload

After excuting this step in my github action.

  - uses: LanceMcCarthy/Action-AzureBlobUpload@v2
    name: Upload application files to Azure Blob
    id: upload
    with:
      connection_string: ${{ secrets.AZURE_CONNECTION_STRING }}
      container_name: '${{ env.Software_Publish_Container }}'  
      destination_folder: '${{ env.Software_Publish_Blob }}\Application Files'
      source_folder: '${{ env.Publish_Path }}\Application Files'
      clean_destination_folder: false
      is_recursive: true
      delete_if_exists: false

when I have then come to check the destination_folder in Azure Storage Explorer sometimes files are missing.

I thought maybe there would be a delay in them appearing but after waiting 10mins and refreshing the files did not appear.

These files are dll and needed for the deployment of the application I am uploading

The source_folder structure is sort of like the following

 - Application Files
       - Application Name_1_0_0_0
           - Applicatoin Folder
           - Applicatoin Folder
           - Applicatoin Folder
           - Applicatoin Folder
           - Applicatoin Folder
           - a.dll
           - b.dll
           - c.dll
           - d.dll
           - e.dll
           - application name.manifest

Sometimes it misses a couple dll file or it misses the .manifest file

The print out of the github action says all of them uploaded successfully.

Similar to this issue:

#113

Please advise how best to fix this.

Question: Is it possible to rename the file on upload

so I have an image i want to upload but the source image will not conform to the naming requirement used by the application that in the end will be looking for.

Is it possible to rename the file on upload

Truncated blob name

Hey Lance!

Was using this to upload an executable to Azure with the following name: ./build/bin/data-composer-0.9.7.exe. It worked, but the blob name was ta-composer-0.9.7.exe.

Any thoughts on what might have happened?

Here's the configuration I used:

      - name: Upload to Azure Blob Storage
        uses: LanceMcCarthy/[email protected]
        with:
          connection_string: ${{ secrets.AZURE_BLOB_CONNECTION_STRING }}
          container_name: releases
          source_folder: ./build/bin/

Uploaded file is missing

Hello, thank you for writing this github action - it was incredibly useful.

We've been using this package to upload the results (3 files) of an application build to a specific Azure container for distribution.

We've noticed that at times only 2 of the 3 files are uploaded - this occurs randomly. The action will print that all 3 have been uploaded, but when I check the Azure Container, only 2 files remain.

This is a screenshot of the github action output and our current configuration:

I am wondering if this is related to the clean_destination_folder setting.
Specifically, do we need to await on line 38 when we call delete?

Here is what I think is happening:

1. Program calls delete but does not wait for the result.
2. After deletecalls, proceeds to the uploading part
3. A race condition occurs because uploads will overwrite (at least it looks it will, and at times, the first file is uploaded before the blob delete is completed via garbage collection.
   
   

My suggestion is we should try awaiting the delete call, and see if this solves the issue.

Please let me know if you would like me to submit a PR. Would be glad to help.

Default recursive doesn't work

Hi,

Sometimes it doesn't upload subdirectories. See log snippet.
The list of the files created by a find dist command.

2021-01-29T12:21:41.6825240Z [12:21:41] Using gulpfile ~/work/kundkhu/kundkhu/gulpfile.js
2021-01-29T12:21:41.6836695Z [12:21:41] Starting 'build'...
2021-01-29T12:21:41.6850326Z [12:21:41] Starting 'nunjucks'...
2021-01-29T12:21:41.6852331Z Rendering nunjucks files..
2021-01-29T12:21:41.7522048Z [12:21:41] Finished 'nunjucks' after 67 ms
2021-01-29T12:21:41.7527514Z [12:21:41] Finished 'build' after 69 ms
2021-01-29T12:21:41.7649242Z dist
2021-01-29T12:21:41.7650816Z dist/index.html
2021-01-29T12:21:41.7653647Z dist/szakacskonyv
2021-01-29T12:21:41.7655308Z dist/szakacskonyv/original
2021-01-29T12:21:41.7656803Z dist/szakacskonyv/original/122.jpg
2021-01-29T12:21:41.7658262Z dist/szakacskonyv/original/55.jpg
2021-01-29T12:21:41.7659585Z dist/szakacskonyv/original/59.jpg
2021-01-29T12:21:41.7732347Z dist/szakacskonyv/eszt_rost.html
2021-01-29T12:21:41.7732990Z dist/szakacskonyv/bufsztek.html
2021-01-29T12:21:41.7733587Z dist/szakacskonyv/images
2021-01-29T12:21:41.7734165Z dist/szakacskonyv/images/122.jpg
2021-01-29T12:21:41.7734764Z dist/szakacskonyv/images/55.jpg
2021-01-29T12:21:41.7735900Z dist/szakacskonyv/images/96-2.jpg
2021-01-29T12:21:41.7736508Z dist/szakacskonyv/images/59.jpg
2021-01-29T12:21:41.7850478Z dist/szakacskonyv/tolt_rost.html
2021-01-29T12:21:41.7851003Z dist/szakacskonyv/tejf.html
2021-01-29T12:21:41.7851528Z dist/szakacskonyv/vadnyul.html
2021-01-29T12:21:41.7852085Z dist/szakacskonyv/sonkas_tek.html
2021-01-29T12:21:41.7852933Z dist/szakacskonyv/csaszarszelet.html
2021-01-29T12:21:41.7853681Z dist/szakacskonyv/szakacskonyv copy.html
2021-01-29T12:21:41.7854668Z dist/szakacskonyv/nyulpastetom.html
2021-01-29T12:21:41.7855389Z dist/szakacskonyv/szakacskonyv.html
2021-01-29T12:21:41.7923169Z ##[group]Run LanceMcCarthy/[email protected]
2021-01-29T12:21:41.7923732Z with:
2021-01-29T12:21:41.7926342Z   connection_string: ***
2021-01-29T12:21:41.7926879Z   container_name: $web
2021-01-29T12:21:41.7927250Z   source_folder: dist/
2021-01-29T12:21:41.7927852Z   fail_if_source_empty: true
2021-01-29T12:21:41.7928497Z   clean_destination_folder: false
2021-01-29T12:21:41.7928968Z ##[endgroup]
2021-01-29T12:21:42.5629647Z Uploaded dist/index.html to $web/index.html...

Flatens file structure - subfolders contents go to root

This setup is flattening the file structure in the Blob destination

code:

      - name: Azure Blob Upload with Destination folder
      uses: LanceMcCarthy/[email protected]
      with:
        connection_string: ${{ secrets.AZBLOBCONNECTIONSTRING }}
        container_name: '$web'
        source_folder: dist/i6
        destination_folder: i6
        clean_destination_folder: true

Log says:

Uploaded dist/i6/assets/svg-country-flags/svg/bi.svg to $web/i6/bi.svg

This:

    source_folder: dist/i6

also flattens.

Temp work around, each folder has a build step, e.g I have now added:

      - name: Blob 1
      uses: LanceMcCarthy/[email protected]
      with:
        connection_string: ${{ secrets.AZBLOBCONNECTIONSTRING }}
        container_name: '$web'
        source_folder: dist/i6
        destination_folder: i6
        clean_destination_folder: true
        is_recursive: **false**

      - name: Blob 2
      uses: LanceMcCarthy/[email protected]
      with:
        connection_string: ${{ secrets.AZBLOBCONNECTIONSTRING }}
        container_name: '$web'
        source_folder: dist/i6/assets
        destination_folder: i6/assets
        clean_destination_folder: **false**
        is_recursive: **false**

Upgrade Action from node12 to node16

GitHub has deprecated node v12 -based Actions https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/

This SHOULD be a simple switch in the action.yml file => https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runs-for-javascript-actions

This enhancement is to perform the update and validate everything still works.