dougbw / azure-devops-data-collector

Azure DevOps Data Collector

azure-devops-data-collector

This module pulls data from the Azure DevOps REST API and publishes to Azure table storage. The data can then be modeled in PowerBI into pretty graphs and stuff.

This can provide a holistic view across an organization of:

• Resource usage (how many repos, projects, pipelines, etc)
• Development activity (commits, pull requests)
• Pipeline success rate/duration
• Deployments to environments

Build status

Installation

This module can be installed from the PowerShell Gallery

Install-Module AzureDevopsDataCollector

Pre-requsuites

• Azure PowerShell Module
• Azure Storage Account
• PowerBI Desktop

Usage (inside an AzDo pipeline)

The quickest way to use this is to run it inside a scheduled Azure DevOps pipeline, as the job can consume predefined variables containing the organization name and api token. This pipeline can then be triggered on a scheduled using the built-in pipeline scheduling feature.

• Create a repo in Azure DevOps and copy the following files into it from the Example directoy:
  • azure-pipelines.yml
  • Start-Pipeline.ps1
• Create a library variable set named storage-account-vars containing the following variables
  • StorageAccountName
  • StorageAccountKey
• Create a pipeline pointing to "azure-pipelines.yml" in your new repo

Usage (outside an AzDo pipeline)

Import-Module AzureDevopsDataCollector
Import-Module Az
$Params = @{
    Organization = $Organization
    PersonalAccessToken = $PersonalAccessToken
    StorageAccountName = $StorageAccountName
    StorageAccountKey = $StorageAccountKey
}
Invoke-AzDoDataCollector @Params

Using the dashboard

Open the powerbi template file from this repo. Upon opening it should prompt for a storage account name and key. Save it as a powerbi report file (.pbix) once you have completed the initial data load.

This template contains the data model and relationships between the data (E.g commits to repos, deployments to environments) so use this as a starting point and delete any pages / visualizations you don't want.

What data is collected

The data which is currently pulled from the Azure DevOps REST API. The module can be extended to collect data from other api endpoints through configuration files although I have not documented the schema for this yet.

* Projects
    * Pipelines
        * Pipeline runs
    * Environments
        * Deployments
    * Repos
        * Commits
        * Pull requests
* Work Items