Having issue suppressing credential scanner results with .gdnsuppress file about security-devops-azdevops

[chrisnielsen-MS]

Hi @DevOpsAzurance, thank you for reaching out! I'm happy to help you get this resolved. First off, I can see that you are manually constructing the suppressions file which can be error-prone and confusing. Have you tried generating a suppression file using the GDN_RUN_OUTPUTSUPPRESSIONFILE environment variable and comparing the contents? If not, please give that a try and see if it helps. You can read more about it in the table of supported environment variables here: https://github.com/microsoft/security-devops-azdevops/wiki

[DevOpsAzurance]

Hi @DevOpsAzurance, thank you for reaching out! I'm happy to help you get this resolved. First off, I can see that you are manually constructing the suppressions file which can be error-prone and confusing. Have you tried generating a suppression file using the GDN_RUN_OUTPUTSUPPRESSIONFILE environment variable and comparing the contents? If not, please give that a try and see if it helps. You can read more about it in the table of supported environment variables here: https://github.com/microsoft/security-devops-azdevops/wiki

Part of our Pipeline-As-Code allows DevOps team to add hashes, files paths and placeholders to be suppressed. So it get's built in an automated manual process.

this is the template json we use:

      $credSuppressionTemplate = @'
      {{
          "tool": "Credential Scanner",
          "suppressions": [
              {{
                  "file": [
      {0}
                  ],
                  "_justification": "False positive files"
              }},
              {{
                  "placeholder": [
      {1}
                  ],
                  "_justification": "False positive lines"
              }},
              {{
                  "hash": [
      {2}
                  ],
                  "_justification": "False positive hashes"
              }}
          ]
      }}
      '@

And we allow automation to fill in the fields using:

$credSuppressionTemplate -f "$strSuppressionFiles","$strSuppressionPlaceholders","$strSuppressionHashes" | Out-File -FilePath .config/CredScanSuppressions.json -Encoding utf8 -Force

the output suppression file I got is returning the hashes, which is problematic, because we are running the tool at the root of the repo, but the pipeline is only concerned about one project out of two. I want to exclude the other paths, but suppressions does not accept wildcards, so trying to add the file paths so any future changes, will be suppressed, instead of using hashes which will cause a new false positive.

{
    "hydrated": false,
    "properties": {
        "helpUri": "https://eng.ms/docs/microsoft-security/security/azure-security/cloudai-security-fundamentals-engineering/security-integration/guardian-wiki/microsoft-guardian/general/suppressions",
        "hydrationStatus": "This file does not contain identifying data. It is safe to check into your repo. To hydrate this file with identifying data, run `guardian hydrate --help` and follow the guidance."
    },
    "version": "1.0.0",
    "suppressionSets": {
        "default": {
            "name": "default",
            "createdDate": "2024-04-23 20:13:14Z",
            "lastUpdatedDate": "2024-04-23 20:13:14Z"
        }
    },
    "results": {
        "7897d9d52460c52d4bdadac5f541a2f4f897440a845a77bae4597b6640d64cb4": {
            "signature": "7897d9d52460c52d4bdadac5f541a2f4f897440a845a77bae4597b6640d64cb4",
            "alternativeSignatures": [],
            "memberOf": [
                "default"
            ],
            "createdDate": "2024-04-23 20:13:14Z"
        },
        "092b6ef5ac77143789f747f7b2139bd1260990b3aec72ff018a74cc7b01cabac": {
            "signature": "092b6ef5ac77143789f747f7b2139bd1260990b3aec72ff018a74cc7b01cabac",
            "alternativeSignatures": [
                "f20f43937ce582e53641938340a9c8a58616e1a0a1c1468d3fb75b2ceb5e901e"
            ],
            "memberOf": [
                "default"
            ],
            "createdDate": "2024-04-23 20:13:14Z"
        },
        "5e66fbf5b767ffc89212e43d17a76394d2c3128f11d6e9a5c6e3da1a6249a9c3": {
            "signature": "5e66fbf5b767ffc89212e43d17a76394d2c3128f11d6e9a5c6e3da1a6249a9c3",
            "alternativeSignatures": [
                "5a2a092ffe417b9094f90884e068080afaae75f0ee3b58c98935517d59332f40"
            ],
            "memberOf": [
                "default"
            ],
            "createdDate": "2024-04-23 20:13:14Z"
        },
        "e03032d4a9ad066785c4b20e559c671b2a0829b78069ed70ae0c24c0bab7e82b": {
            "signature": "e03032d4a9ad066785c4b20e559c671b2a0829b78069ed70ae0c24c0bab7e82b",
            "alternativeSignatures": [
                "861712aaf8a6de62452e31a9a8fe53644c491972e8b6a86e91a2a9362be5807c"
            ],
            "memberOf": [
                "default"
            ],
            "createdDate": "2024-04-23 20:13:14Z"
        },
        "515f09c25304103cb819e7014e3b8b7ad4b5e429423e816e97c23da1b0865144": {
            "signature": "515f09c25304103cb819e7014e3b8b7ad4b5e429423e816e97c23da1b0865144",
            "alternativeSignatures": [
                "aa665d8a162a70ee3ac145eee84747e0f10cfb0a4adbd0315e55d31808eb714c"
            ],
            "memberOf": [
                "default"
            ],
            "createdDate": "2024-04-23 20:13:14Z"
        },
        "7889bd0856caa9ae9bb6afe87de9290e2e612cd75f7818f865a514c488cf154a": {
            "signature": "7889bd0856caa9ae9bb6afe87de9290e2e612cd75f7818f865a514c488cf154a",
            "alternativeSignatures": [
                "40e8a53fc8672cf51853c4483ff1a6caa321e2671037fe6d367bb878821546c0"
            ],
            "memberOf": [
                "default"
            ],
            "createdDate": "2024-04-23 20:13:14Z"
        },
        "8059d408810d9c5c5c10a96c8c9ae20aedcf26a7b9979179ed0ffbc8ad2fff06": {
            "signature": "8059d408810d9c5c5c10a96c8c9ae20aedcf26a7b9979179ed0ffbc8ad2fff06",
            "alternativeSignatures": [
                "5f872581a4ef33068acb7beeea26bdd7cdd3d868d769814a5d422d9332933a21"
            ],
            "memberOf": [
                "default"
            ],
            "createdDate": "2024-04-23 20:13:14Z"
        },
        "2efb759f5f84ea9d5d330d08ff02bced716da76799adbd42ba8f2b55b417dd27": {
            "signature": "2efb759f5f84ea9d5d330d08ff02bced716da76799adbd42ba8f2b55b417dd27",
            "alternativeSignatures": [
                "7c1e930e322ce4eb1d69e4d1272ad9ab68fea0d9e5546cd70a4432aece9bbe66"
            ],
            "memberOf": [
                "default"
            ],
            "createdDate": "2024-04-23 20:13:14Z"
        }
    }
}

How can I use paths instead...

[chrisnielsen-MS]

Thank you for explaining your scenario @DevOpsAzurance

Based on your description, I think a simpler path may be to pass through the argument to CredScan that makes it scan a particular folder. You can override the default with the env variable GDN_CREDSCAN_TARGETDIRECTORY. Here is an example.

• task: MicrosoftSecurityDevOps@1
  displayName: 'Run MSDO'
  env:
  GDN_CREDSCAN_TARGETDIRECTORY: 'Relative/Path/To/Folder'

(Please disregard incorrect indentation above, it keeps reformatting on me)

[DevOpsAzurance]

Thank you for explaining your scenario @DevOpsAzurance

Based on your description, I think a simpler path may be to pass through the argument to CredScan that makes it scan a particular folder. You can override the default with the env variable GDN_CREDSCAN_TARGETDIRECTORY. Here is an example.

• task: MicrosoftSecurityDevOps@1
  displayName: 'Run MSDO'
  env:
  GDN_CREDSCAN_TARGETDIRECTORY: 'Relative/Path/To/Folder'

(Please disregard incorrect indentation above, it keeps reformatting on me)

That particular environment variable is not in the list at:

https://github.com/microsoft/security-devops-azdevops/wiki

Is there a more complete list that is not currently shared.

[chrisnielsen-MS]

Unfortunately the exclusion of CredScan from those docs is intentional as the tool was officially deprecated from MSDO last September but it has not been removed entirely yet. For a better experience with secret scanning in Azure DevOps, you may want to look at GitHub Advanced Security for Azure DevOps.

[DevOpsAzurance]

Unfortunately the exclusion of CredScan from those docs is intentional as the tool was officially deprecated from MSDO last September but it has not been removed entirely yet. For a better experience with secret scanning in Azure DevOps, you may want to look at GitHub Advanced Security for Azure DevOps.

How would I create a the suppression file, but using paths? I ran pipeline using the command to output the suppression file, but it is using the hashes. I don't want to run into constant issues, if they do edit the file, so would rather have the suppression done at the file level? I cannot find something similar to it. Since the GDN_RUN_CONFIG can take multiple configs, I tried the following:

 $terrascanSuppressionTemplate = @'
      {{
        "tools": [
          {{
              "tool": {{
                "name": "Terrascan",
                "version": "Latest"
              }},
              "suppressions": [
                  {{
                      "file": [
          {0}
                      ],
                      "_justification": "False positive files"
                  }}
              ]

          }}
        ]
      }}
      '@

I have seen similar for the Credential Scanner tool, and instead of arguments, has suppressions. It fails though as it seems to be making the call to terrascan in a mannger, that instead I get the "how to properly use" terrascan output:

 Analyze:
    Running Terrascan 1.18.0.1
    ------------------------------------------------------------------------------
    /usr/EveDORunnersLinux1/_work/_msdo/packages/nuget/Microsoft.Guardian.TerrascanRedist_linux_amd64.1.18.0.1/tools/terrascan 
    Terrascan
    
    Detect compliance and security violations across Infrastructure as Code to mitigate risk before provisioning cloud native infrastructure.
    For more information, please visit https://runterrascan.io/
    
    Usage:
rascan [command]
    
    Available Commands:
      completion  Generate the autocompletion script for the specified shell
      help        Help about any command
      init        Initializes Terrascan and clones policies from the Terrascan GitHub repository.
      scan        Detect compliance and security violations across Infrastructure as Code.
      server      Run Terrascan as an API server
      version     Terrascan version
    
    Flags:
      -c, --config-path string      config file path
      -h, --help                    help for terrascan
      -l, --log-level string        log level (debug, info, warn, error, panic, fatal) (default "info")
          --log-output-dir string   directory path to write the log and output files
      -x, --log-type string         log output type (console, json) (default "console")
      -o, --output string           output type (human, json, yaml, xml, junit-xml, sarif, github-sarif) (default "human")
          --temp-dir string         temporary directory path to download remote repository,module and templates
    
terrascan [command] --help" for more information about a command.
    Tool run time: 0.025032 seconds
    ------------------------------------------------------------------------------
    Terrascan completed with exit code 0
    ------------------------------------------------------------------------------
    
  Process:
    Convert:
      Converting any raw tool logs to Sarif format ...
      Completed converting raw tool logs to Sarif format.
    Import:
##[error]JsonReaderException: Unexpected character encountered while parsing value: T. Path '', line 0, position 0.
##[error]MSDO CLI exited with an error exit code: 1

I am assuming the failure is that it expected output in json, and instead got a readme.