azure-samples / storage-blobs-go-quickstart

Code for QuickStart article for accessing Blobs with Go

page_type	languages	products	description	urlFragment
sample	go	azure	This repository contains a simple sample project to help you getting started with Azure storage using Go as the development language.	storage-blobs-go-quickstart

Transfer objects to and from Azure Blob storage using Go

This repository contains a simple sample project to help you getting started with Azure storage using Go as the development language.

Prerequisites

To complete this tutorial:

• Install Go 1.8 or later

If you don't have an Azure subscription, create a free account before you begin.

Create a storage account using the Azure portal

First, create a new general-purpose storage account to use for this quickstart.

1. Go to the Azure portal create a storage account menu.
2. Enter a unique name for your storage account. Keep these rules in mind for naming your storage account:
   • The name must be between 3 and 24 characters in length.
   • The name may contain numbers and lowercase letters only.
3. Select your subscription.
4. For Resource group, create a new one or use an existing resource group.
5. Select the Location to use for your storage account.
6. Click Create to create your storage account.

Sign in with Azure CLI

To support local development, the DefaultAzureCredential can authenticate as the user signed into the Azure CLI.

Run the following command to sign into the Azure CLI.

az login

Assign RBAC permissions to the storage account

Azure storage accounts require explicit permissions to perform read and write operations. In order to use the storage account, you must assign permissions to the account. To do that you'll need to assing an appropriate RBAC role to your account. To get the objectID of the currently signed in user, run az ad signed-in-user show --query objectId.

Run the following AzureCli command to assign the storage account permissions:

az role assignment create --assignee "<ObjectID>" --role "Storage Blob Data Contributor" --scope "<StorageAccountResourceID>"

Learn more about Azure's built-in RBAC roles, click here.

Note: Azure Cli has built in helper fucntions that retrieve the storage access keys when permissions are not detected. That functionally does not transfer to the DefaultAzureCredential, which is the reason for assiging RBAC roles to your account.

Download and Install the Azure Storage Blob SDK for Go

From your GOPATH, execute the following command:

go get github.com/Azure/azure-sdk-for-go/sdk/azidentity

go get github.com/Azure/azure-sdk-for-go/sdk/storage/azblob

At this point, you can run this application. It creates an Azure storage container and blob object then cleans up after itself by deleting everything at the end.

Run the application

Open the storage-quickstart.go file.

Replace <StorageAccountName> with the name of your Azure storage account.

Run the application with the go run command:

go run storage-quickstart.go

More information

The Azure storage documentation includes a rich set of tutorials and conceptual articles, which serve as a good complement to the samples. For more samples on the Azure Storage SDK for GO, check out the examples here.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

storage-blobs-go-quickstart's Issues

Update the golang Azure samples to use go modules

Please provide us with the following information:

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [x] feature request
- [x] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Install with latest go language version (that supports go module)

Any log messages given by the failure

Expected/desired behavior

Expected to import/use the sample code with go module support for the consuming any of the Azure features

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)
Any of the OS with their versions

Versions

Mention any other details that might be useful

Not really sure how to repro this issue. I will need to create a sample which tries to use Azure libraries as go modules (which don't exist right now). The rest of the libraries would be possible to import.

---

Thanks! We'll be in touch soon.

Sample does not highlight the need for application - level pagination handling

Please provide us with the following information:

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [ ] feature request
- [X ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

This being a first touchpoint for many consumers of Azure, I wanted to highlight a naiive assumption I made consuming this API that would have led to data loss in prod.
There is no indication in this getting started that the application is expected to handle HTTP pagination by observing response codes and setting up subsequent read ops. This breaks from Go idioms readers- one reader per blob is a reasonable assumption.
I would suggest calling out the inspection of these HTTP variables in the example - or even mentioning the need in comments- as a warning to API consumers as to the limitations of what the library covers for the application programmer.

I got bit by this. That's on me. I hope we can save someone else from the same issue.

Several compile time errors found

Please provide us with the following information:

This issue is for a: (mark with an x)

- [ x] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

go run storage-quickstart.go

Any log messages given by the failure

# command-line-arguments
./storage-quickstart.go:54:21: assignment mismatch: 1 variable but serviceClient.NewContainerClient returns 2 values
./storage-quickstart.go:71:22: blobClient.UploadBufferToBlockBlob undefined (type *azblob.BlockBlobClient has no field or method UploadBufferToBlockBlob)
./storage-quickstart.go:71:64: undefined: azblob.HighLevelUploadToBlockBlobOption
./storage-quickstart.go:101:21: cannot use azblob.RetryReaderOptions{} (value of type azblob.RetryReaderOptions) as type *azblob.RetryReaderOptions in argument to get.Body

Expected/desired behavior

Output as detailed on the readme instructions

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

Linux 5.15.0-41-lowlatency #44-Ubuntu SMP PREEMPT Wed Jun 22 15:40:35 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Versions

go version go1.18.1 linux/amd64

Mention any other details that might be useful

---

Thanks! We'll be in touch soon.

Example not working with `azidentity.NewDefaultAzureCredential(nil)`

This issue is for a: (mark with an x)

- [x] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Run the example as documented in the README:

• create the storange account
• edit storage-quickstart.go by replacing the <StorageAccountName> with the actual name to be used
• az login
• go run storage-quickstart.go

Any log messages given by the failure

# go run storage-quickstart.go
Azure Blob storage quick start sample
Creating a container named quickstart-4817313775875140775
Creating a dummy file to test the upload and download
2022/05/06 13:24:35 Failure to upload to blob: ===== RESPONSE ERROR (ErrorCode=AuthorizationPermissionMismatch) =====
Description=This request is not authorized to perform this operation using this permission.
RequestId:8d3546a9-e01e-0063-453b-61bd38000000
Time:2022-05-06T11:24:15.5872142Z, Details: (none)
exit status 1

Expected/desired behavior

# go run storage-quickstart.go
Azure Blob storage quick start sample
Creating a container named quickstart-7859879233567791186
Creating a dummy file to test the upload and download
Listing the blobs in the container:
https://aoeuaoeuaoeuaoeu.blob.core.windows.net/quickstart-7859879233567791186/quickstartblob-7024034036063403663

hello world this is a blob

Press enter key to delete the blob fils, example container, and exit the application.

Cleaning up.
Deleting the blob quickstartblob-7024034036063403663
Deleting the blob quickstart-7859879233567791186

OS and Version?

Arch Linux, current

Versions

master branch

Mention any other details that might be useful

I have tried various things to get Blob access working with azidentity.NewDefaultAzureCredential(nil) but failed miserably (I am logged in as owner on the root management group, so the permissions should be all set). There is also a section in the README.md that suggest to take note of the storage account name as well as the storage account key for later use. However they are not used anymore in the quickstart.

The thing I got working is to replace the credentials, err := azidentity.NewDefaultAzureCredential(nil) with the following:

       accountName, ok := os.LookupEnv("AZURE_STORAGE_ACCOUNT_NAME")
       if !ok {
               panic(errors.New("AZURE_STORAGE_ACCOUNT_NAME could not be found"))
       }
       accountKey, ok := os.LookupEnv("AZURE_STORAGE_ACCOUNT_KEY")
       if !ok {
               panic(errors.New("AZURE_STORAGE_ACCOUNT_KEY could not be found"))
       }
       credential, err := azblob.NewSharedKeyCredential(accountName, accountKey)

... and then...

       serviceClient, err := azblob.NewServiceClientWithSharedKey(url, credential, nil)
       // ...
       blobClient := containerClient.NewBlockBlobClient(url + containerName + "/" + blobName)

... instead of...

       serviceClient, err := azblob.NewServiceClient(url, credential, nil)
       // ...
       blobClient, err := azblob.NewBlockBlobClient(url+containerName+"/"+blobName, credential, nil)
       if err != nil {
               log.Fatal(err)
       }