Story

As a product owner I want to explore what is needed to run the module from Azure Automation runbooks with managed identity or other available means of authentication so we can offer users a choice of Azure services for running the module in a cost efficient manner so users can save on cloud spend.

Acceptance Criteria

• Managed Identity tested and enhanced for Azure Automation
• If code enhancement is required to make MSI auth work: A proposal in this issue for enhancing the CI test workflow and related Azure resource to implement testing for the MSI auth before finalizing the PR
• End-user documentation updated with instructions and examples

As an end-user I want to be sure the retention settings for build and artifacts are sensible. Therefore I want the module to export and check relevant settings from Azure DevOps so I can be sure these settings are not overlooked in an audit.

Acceptance Criteria

• Implemented export of build- and artifact retention
• Rules with sensible default values
• Rules should be configurable to end-user's compliance requirements

Story

As a product owner I want to have a GitHub actions based environment for testing VM based MSI auth so end-user can be sure this works. Solution should be cost efficient, have auto-shutdown and be easy to turn on.

Acceptance Criteria

• VM based test must output coverage, coverage to be merged by codecov.io
• Fix any bugs in current MSI VM auth code that arise in testing
• If MSI works different between Linux and Windows based pwsh core, both cases should be testable.
• ACA is the main platform for testing. All tests run on that platform should continue to pass. The VM platform should only run tests that are specific to running the module on a vm.

Limitations

Due to the direct impact on project budget, this task is only assignable to the project maintainer @webtonize

As a PO I want transparency in the objects selected for inspection so users can have a clear understanding of the scope of the audit. Therefore I want the selectors used for inspecting only production resources deprecated and rules adapted accordingly. To help users get started quickly a decent suppressiongroups yaml should be offered as a replacement.

Acceptance Criteria

• PSRule Production selectors removed from the codebase
• Rules changed to inspect all objects formerly scoped by the selectors
• Example YAML and instructions for a best-practice set of PSRule suppression groups

Story

As a repository maintainer I want to be aware of branches that have not been updated for a while (2 to 3 months) so I can remove stale branches and keep my repository tree clean.

Acceptance Criteria

• Branch export includes last commit date for each branch
• Rule to check for branches not pushed in last 90 days
• The 90 days default value should be configurable with any valid int as nr of days
• The rule should have the lowest severity level

Story

As an end-user I want 2 baselines I can use for either public or private projects so I can check the best practices for each case as applicable so I always have the best appropriate security measures.

Acceptance Criteria

• 2 Baselines: Open Source and Private, private should be the default
• Check settings:
  • Project visibility
  • Repository visibility
  • Badge access
  • Other as applicable, please research

As an end-user I want current and future objects to have a clear name that depicts their position in the larger Azure DevOps architecture. That way I can use the module on multiple projects and still relate the output and also allows future parts of objects to be named and inspected seperatly while mainting the ability to locate the inspected object

Acceptance Criteria

• All exported Azure DevOps objects have an ObjectName field with a name convention of org.project.objectname.childname.grandchildname
• Rules have been tested and output checked/improved for maximum readability

Story

As an end-user I want more serviceconnection capabilities than just the current ARM type, for example I want to check if I am not using non-recommended classic-ARM-connectors, this to ensure I am using the most secure connections available.

Acceptance Criteria

• Export all service connections regardless of type
• Ensure current ruleset is either migrated to new way of working or can coexist with it
• Ensure module version number is updated according to any breaking changes
• Ensure documentation is clear and up to date
• Add a rule ensuring classic ARM connections are not used in the Azure DevOps project

Story

As a Product Owner I want best practice recommendations for all service connection types. As limited Azure sponsoring is available, I want to eplore all service connection types for Azure related resources so we can build and test these usecases before funding ends.

Acceptance Criteria

• Inventory of best practices with references documented in this issue.
• Follow up issues created for required rules, 1 issue per service connection type.

Story

As an end-user I want to be sure all my Azure DevOps objects can be exported, even if I have special characters in the object names so I can be sure my complete Azure DevOps environment will get audited and I will not be surprised in human audit.

Acceptance Criteria

• Non-filesystem characters in input Object Names should be replaced by safe characters
• String replacement function should be a single function called from Export- functions
• Export- functions should be changed to use the new function
• When exporting object names, any folder hierarchy in Azure DevOps (like pipeline and release folder structure) should be preserved in the exported filename
• Azure DevOps test project and test cases should reflect handling with and without a folder structure for those objects

Story

As a module publisher I want end users using only the PowerShell Gallery to be able to read the latest release notes for the module as published on GitHub so users can stay up to date via the gallery alone.

Acceptance Criteria

• Nothing is published to PSGallery official module for this project as development work in this feature, any testing needs to be done with some garbage module the developers owns other than PSRule.Rules.AzureDevOps.
• The module manifest in the repository does not need to be updated on publishing
• The current PSGallery publishing workflow should be used/enhanced.

As a developer I want the unit tests for rules split over multiple files so I have better overview, co-pilot works better and have less merge conflicts and therefore I can develop more efficiently.

Acceptance Criteria

• Full exports from the Azure DevOps REST API should only happen once per TokenType and AuthType. In other words: The number of calls to the REST API should not increase when splitting the files.
• File should be split to the ObjectType grouping level at minimum, split per rule is more preferred.
• Unit tests for rules should only start after all the module's functions have been tested.

Story

As an end-user I want to be able to use a fine-grained personal access token for data collection, therefore the Export functions should use documented REST API endpoints instead of the endpoints used by the API so I can specify the access required by the token and avoid unnecesarry large permission sets.

Acceptance Criteria

• Build/GeneralSettings REST API endpoint documented here is used over Contributions/Hierarchy UI endpoint in the export functions for pipeline settings.
• Rules covering pipeline settings are migrated to the data schema used by the REST API
• Pester tests implemented for data collection and ruleset

Story

As an end-user I want maximum probability in detection possible sensitive information so I can feel assured secrets are stored in a secure manner

Acceptance Criteria

• Detect both the individual keys as well as full connection strings for the most popular Azure Services
• Detect SSH Private Keys
• Detect SSL/TLS Private Keys
• Popular AWS Services
• Detection is updated for variable-groups, GUI Build Pipelines, GUI Releases

Story

As a developer I prefer multiple smaller files over 1 big one to prevent merge conflicts when adding tests for new functions.

Acceptance Criteria

• Split function test files to cover the same structure as the function files for the module.
• Split rules test files to cover the same structure as the rules files.
• No significant increase in amount of calls the Azure DevOps REST API`
• No significant increase in overall test duration
• No decrease in code coverage based on the module's Functions directory

For the Azure DevOps pipeline example, "Grave Accent `" symbol should be removed.
Link to code:
https://github.com/cloudyspells/PSRule.Rules.AzureDevOps/blob/e609897620d10de08b88a8be416994066142ce23/pipelines/azure-pipelines.yml#L53C38-L53C38

Error message from Azure DevOps Pipeline:

Export-AzDevOpsRuleData: /home/vsts/work/_temp/364686e8-433e-44db-9604-197f4e66cf3b.ps1:11
Line |
  11 |    -OutputPath './results.sarif' `
     |    ~~~~~~~~~~~
     | Cannot bind parameter because parameter 'OutputPath' is specified more
     | than once. To provide multiple values to parameters that can accept
     | multiple values, use the array syntax. For example, "-parameter
     | value1,value2,value3".

Story

As a security specialist I want to module to be able to run in a read-only role so I can be confident the module does not make modifications to Azure DevOps.

Acceptance Criteria:

• Service Principal Authentication with secret
• Managed Identity Authentication
• Must store and update token for re-use in the session, so not re-authenticate for each request when doing a full export
• Document how to set up minimal permissions role + assignment for the module

🐞 Bug Report

Describe the bug

Running Export-AzDevOpsRuleData on a new repo, with a new pipeline not on the main/default branch fails with the following error:

Invoke-RestMethod: /Users/[email protected]/PSRule.Rules.AzureDevOps/src/PSRule.Rules.AzureDevOps/Functions/DevOps.Pipelines.Core.ps1:223:21
Line |
223 | … $response = Invoke-RestMethod -Uri $uri -Method Get -Headers $header
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| {"$id":"1","innerException":null,"message":"TF401174: The item 'test-pipeline.yaml' could not be found in the
| repository 'PmcTest' at the version specified by '' (resolved to commit
| '97cc80fe33624aeff34d81adb6a4a409aabf69bf')","typeName":"Microsoft.TeamFoundation.Git.Server.GitItemNotFoundException,
| Microsoft.TeamFoundation.Git.Server","typeKey":"GitItemNotFoundException","errorCode":0,"eventId":3000}

Is this a regression?

I don't know.

To Reproduce

1. Create a new repo in azure devops with just a readme file
2. Create a new branch of master/main
3. Add a test yaml pipeline and commit (note - DO NOT merge to master/main)
4. Create a pipeline in azure devops referencing the yaml above on the new branch
5. Run Export-AzDevOpsRuleData

Expected behaviour

Data Collection should run without throwing an error.

Media prove

Your environment

OS: Darwin 22.6.0 Darwin Kernel Version 22.6.0
PowerShell version: 7.3.3
PSRule version: 2.9.0
PSRule.Rules.AzureDevOps version: v0.2.0

Additional context

Export command was run from my dev mac using a ReadOnly PAT.

Story

As an end user I want to make sure I am compliant with the recommendations regarding group membership so I can be sure I have a secure best practice environment.

Acceptance Criteria

• Enumerates Azure DevOps groups to users; expanding child groups including AAD groups only if possible through Azure DevOps REST API alone
• Includes a working export of all default ADO groups
• Includes a rule that ensures the Project Administrators group includes no less than 2 and no more than 4 members.

Story

As an end-user I want to be able to run the module with different sets of REST API permissions on the PAT in line with my compliance/security requirements so I can avoid the tool becoming a security risk.

Acceptance Criteria

• Main Export function must have a option parameter for FullAccess, FineGrained and ReadOnly settings.
  • FullAccess may use any endpoint we can get useful information for analyzing security posture.
  • FineGrained is limited to the items we can specify in setting a fine-grained personal access token.
  • ReadOnly is limited to Read permissions we can set in a fine-grained personal access token
• Test cases covering the 3 options implemented for both export functions and running the ruleset after export. Rules should not run when data for their purpose has not been exported.

📚 Documentation Issue Report

Describe the bug

When clicking on the link https://github.com/cloudyspells/PSRule.Rules.AzureDevOps/blob/main/CONTRIBUTING.md#code-of-conduct the user receives a Not Found page.

When clicking on any of the following links nothing happens (as the links reference missing markdown sections)

When clicking on the rule hyperlinks in https://github.com/cloudyspells/PSRule.Rules.AzureDevOps/blob/main/docs/token-permissions.md user gets 404 page not found

To Reproduce Contributing (Not Found)

1. From code tab click on CONTRIBUTING.md
2. Click on Code of Conduct link in table of contents
3. Click on PSRule.Rules.AzureDevOps Code of Conduct

To Reproduce Inactivity

1. From code tab click on CONTRIBUTING.md
2. Click on Code of Conduct link in table of contents
3. Click on Styleguides, Commit Messages or Join the Project Team

To Reproduce Rules Link (Not Found)

1. From code tab click on docs/token-permissions.md
2. Click on a rule link in Token Scope for Rules section

Media prove

Describe the solution you'd like

Hyperlink corrected so the user does not see a Not Found page.
Sections for Styleguides, commit messages and join the project team added to documentation.

Additional context

The link in the CONTRIBUTING.md markdown file needs a small amendment.

The current link is:
https://github.com/cloudyspells/PSRule.Rules.AzureDevOpsblob/master/CODE_OF_CONDUCT.md

but this needs changing to:
https://github.com/cloudyspells/PSRule.Rules.AzureDevOps/blob/main/CODE_OF_CONDUCT.md

i.e. change branch from master to main

In Token Scope for Rules example link:

https://github.com/cloudyspells/PSRule.Rules.AzureDevOps/blob/main/docs/src/PSRule.Rules.AzureDevOps/en/Azure.DevOps.Tasks.VariableGroup.NoPlainTextSecrets.md

but this needs changing to:

https://github.com/cloudyspells/PSRule.Rules.AzureDevOps/blob/main/src/PSRule.Rules.AzureDevOps/en/Azure.DevOps.Tasks.VariableGroup.NoPlainTextSecrets.md

i.e. remove /docs from the link path

🐞 Bug Report

Describe the bug

• Repo export fails when repo has a single branch

Is this a regression?

No

To Reproduce

1. Export project with single repo and single branch

Expected behaviour

• Repo should export without issues

Your environment

• OS:
• PowerShell version: 7.3
• PSRule version: 2.9
• PSRule.Rules.AzureDevOps version: 0.4.0

Need to add fix in code in case if Azure DevOps service connection contains backslash /:

      "name": "CI/CD template Repo Connection",
      "description": ""
    }
  ],
  "ObjectType": "Azure.DevOps.ServiceConnection",

Link to line in code:

Error message from Azure DevOps pipeline:

Out-File: /home/vsts/.local/share/powershell/Modules/PSRule.Rules.AzureDevOps/0.4.3/Functions/DevOps.ServiceConnections.ps1:156
Line |
 156 |  … Depth 100 | Out-File "$OutputPath/$($serviceConnection.name).ado.sc.j …
     |                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | Could not find a part of the path '/home/vsts/work/_temp/CI/CD template
     | Repo Connection.ado.sc.json'.

As a product owner I want to ensure all cmdlets are covered in unit testing for the various TokenType settings and PATs. Not just for the cmdlets that actually expect different behaviour, but all cmdlets should be covered by the various connections

Acceptance Criteria

• All module cmdlets that depend on the $script:connection object created by Connect-AzDevOps should have a test case for the workflow ADO_PAT, ADO_PAT_READONLY and ADO_PAT_FINEGRAINED personal access tokens.

🐞 Bug Report

Describe the bug

• Azure.DevOps.Repos.Branch.BranchPolicyIsEnabled gives a false positive when Require minimum reviewers is not selected as branch policy and other policies are enabled.

Is this a regression?

No

To Reproduce

1. Enable only a Merge strategy policy on a branch
2. Run the PSRule scan

Expected behaviour

• Any enabled branch policy should make this rule pass

Your environment

• OS: Ubuntu 22.04 LTS
• PowerShell version: 7.3
• PSRule version: 2.9
• PSRule.Rules.AzureDevOps version: v0.4.2

As an end-user I want to be able to inspect my Azure DevOps environment without exporting the configuration as files stored on a disk. Therefore I want the Export- cmdlets to have an option to pass the AzDo objects to the pipeline instead of exporting as json files so I can rest assured no sensitive information is left lingering on disks available to others.

Acceptance crtiteria

• All Export- cmdlets (including those internal to the module) have a -PassThru parameter to output PSObjects to the pipeline.
• -PassThru parameter is tested in CI with a Pester test passing the output to Invoke-PSRule.
• -PassThru parameter is mutually exclusive with the -OutputPath parameter.
• Parameter is built in way that allows it to easily be made the default export option in future releases of the module.

As an end-user I want insights in to the ACLs on more objects than currently supported. I want to check ACLs on service connections, environments and variable groups so I can be sure security best practices are adopted in regards to ACLs on those objects

Acceptance Criteria

• ACL export functionality for mentioned objects
• At minimum a rule to check the objects do not inherit permissions from the project

Story

As an end user I want a better structured Wiki giving answers to relevant questions like installation, running and analyzing the modules results so I have a better understanding of the value of the module

Acceptance Criteria:

• Wiki Home page looks nice and structured
• Wiki has sections for:
  • Module functions / commands
  • Individual rules
  • Basic usage instructions
  • External references like medium blog articles, PSRule, Azure DevOps security best practices etc
• Wiki publishing workflow ensures deprecated docs are also removed from Wiki repo

As a project maintainer I want other to be able to contribute and have transperancy in the testing environment. At this time, all pester tests are conducted in hand-built private Azure DevOps projects. To enable potential contributors I want to have a piece of code that can be run to deploy a test environment suiteable for running the implemented pester tests.

Acceptance Criteria

• When using REST API this code should be in PowerShell, but Terraform or similar IaC tool may also be used
• Must deploy 1 project where all rules fail and 1 project where all rules within current project VSE licensing pass
• Adapt existing Pester tests as needed to facilitate naming conventions and the split from 1 mixed project to 2 clear defined pass and fail environments.
• Implemented as both a GitHub Actions workflow and an Azure DevOps Pipelines YAML.
• Should include cleanup script and/or manual cleanup instructions

As an end-user I want to have a broader view of my environment than just the default branch of my repo. I want the module to collect data for all branches and I want the module to allow me to use the resource exclusion method provided by PSRule in the configuration file.

Acceptance Criteria

• Relevant cmdlets for repository data collection are updated to collect all branches
• Existing branch related rules are updated to work and tested in CI
• PSRule resource exclusion by config file is tested in CI

As a user I want to easily track back objects to their Organization and Project. Although the name convention allows to do this to some degree, use of the . character in project or resource names will mess up any automation. Therefore I want to use a more solid method of attaching this information so I can build nice reports in Azure Monitor.

Acceptance Criteria:

• id field in each object should be a JSON string with the required information

Story

As an end-user I want to ensure my Project Valid Users group does not have any custom role assignments as this should be a default minimum privilege group only.

Acceptance Criteria

• Rules should not fail on the system default assignments
• Rules should fail on any custom assigned roles on any resources
• Rules should not fail simply by disabling inheritance on an item in Azure DevOps