isabella232 / durablefunctions-apiscraping-nodejs

Build an Azure Durable Functions that will scrape GitHub for opened issues and store them on Azure Storage.

page_type	products	languages	name	urlFragment	description
sample	azure azure-functions azure-storage	javascript nodejs	Retrieve opened issue count on GitHub with Azure Durable Functions (Node)	retrieve-opened-issue-count-on-github-with-azure-durable-functions	Build an Azure Durable Functions that will scrape GitHub for opened issues and store them on Azure Storage.

Retrieve opened issue count on GitHub with Azure Durable Functions

Build

The project requires the latest version of the Azure Functions CLI.

It can be installed by running the following code:

npm i -g azure-functions-core-tools@core --unsafe-perm true

More installation options are also available.

Running the Sample

Pre-requisite

• GitHub Personal Access Token
  • How to create a Personal Access Token
• Azure Functions Tooling
• Visual Studio Code (optional)
  • Azure Functions Extension for debugging
• Azure Subscription (if running on Azure)
  • Get a free Azure Trial Subscription

Locally

Change the GitHubToken value in local.settings.json to match your GitHub Personal Access Token created previously in the pre-requisite.

To ensure that everything is configured properly run the following commands within the FanOutFanInCrawler folder.

func extensions install
npm install

Once the dependencies has been installed, you can either run it from VS Code by running code . and hitting F5 on your keyboard. If you want to run it from the command line, you can run it with the following command within the FanOutFanInCrawler folder.

func host start

On Azure

To deploy components on Azure you will require to install and login with the Azure CLI.

First, you will need to provision the service. Look into the provision.ps1 file provided to familiarize yourself with the resources we are going to create.

Then you can execute the file with the previously generated GitHub.

.\provision.ps1 -githubToken <TOKEN> -resourceGroup <ResourceGroupName> -storageName <StorageAccountName> -functionName <FunctionName>

If you do not have PowerShell installed, you can simply take the commands within the file and run it manually.

Contribute

You can contribute to this sample by following the guidelines.

Resources

• Azure Functions Documentation
• Introduction to Azure Functions
• Durable Functions overview
• Durable Functions pattern used in this sample
  • Chaining
  • Fan-out/fan-in