GithubHelp home page GithubHelp logo

veracode-manual-for-github's Introduction

❗ This repository is an outdated version. For the most up-to-date version please go the official manual supported by VERACODE

Official Veracode manual for GitHub

Veracode manual for GitHub

Introduction

This page (or manual) incorporate few option which Veracode customers can use to integrate Veracode Scanning solutions within their own workflow processes.

The following discussed concepts are valid for most CI solutions, however, the examples given here focus on GitHub workflow and unavoidably GitHub Actions implementations.

Few general links to additional Veracode resource collections you may find useful:

The basic - adding Veracode Security Scanning to workflow

As most integration options, here are few ways to introduce our different AST scanning options to a workflow.

Veracode Static Scanning - Upload and Scan

Script

A basic script using a wrapper is documented at Veracode help center and looks as follow:

java -jar vosp-api-wrapper-java<version>.jar -action uploadandscan -vid <Veracode API ID> -vkey <Veracode API key> -appname myapp -createprofile true -teams myteam -criticality VeryHigh -sandboxname sandboxA -createsandbox true -version <unique version> -scantimeout 30 -selectedpreviously true -filepath /workspace/myapp.jar

To use the above script within a GitHub workflow we will need to download the latest version of the API Wrapper and run the above script.

See example 

name: Veracode Static Scan

# Controls when the action will run. 
on:
  # Triggers the workflow on push or pull request events but only for the master branch
  push:
    branches: [ master, release/* ]
  pull_request:
    branches: [ master, develop, main, release/* ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest
    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      - name: create dir prior
        run: |
          mkdir dls
      - name: Download Veracode Wrapper
        working-directory: ./dls/
        run: |
          javawrapperversion=$(curl https://repo1.maven.org/maven2/com/veracode/vosp/api/wrappers/vosp-api-wrappers-java/maven-metadata.xml | grep latest |  cut -d '>' -f 2 | cut -d '<' -f 1)
          echo "javawrapperversion: $javawrapperversion"
          curl -sS -o VeracodeJavaAPI.jar "https://repo1.maven.org/maven2/com/veracode/vosp/api/wrappers/vosp-api-wrappers-java/$javawrapperversion/vosp-api-wrappers-java-$javawrapperversion.jar"
      - name: Run Upload and Scan
        continue-on-error: true
        run: |
          java -jar VeracodeJavaAPI.jar -action uploadandscan -vid ${{secrets.VERACODE_ID}} -vkey ${{secrets.VERACODE_KEY}} -appname ${{ github.repository }} -createprofile true -teams teamA -criticality VeryHigh -sandboxname sandboxA -createsandbox true -version ${{ github.run_id }} -scantimeout 30 -selectedpreviously true -filepath /workspace/app.zip


❕ The above is good to know, however, it is better to use one of the other options below

Pre-built Docker Image

Veracode released and maintain a set of public Docker Images which are available at DockerHub. One of these has the API Wrapper pre-packaged and does not requires the convoluted download script.

An example using the Docker image use can be found here:

See example 

name: Veracode Static Scan

# Controls when the action will run. 
on:
  # Triggers the workflow on push or pull request events but only for the master branch
  push:
    branches: [ master, release/* ]
  pull_request:
    branches: [ master, develop, main, release/* ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

jobs:
  # This workflow contains a single job called "SAST-Scan"
  SAST-Scan:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest
    # The Build and Sandbox name generation omitted from the inline sample
    needs: [generate-sandbox-name, Build]
    # Steps represent a sequence of tasks that will be executed as part of the job
    container: 
      image: veracode/api-wrapper-java:latest
      options: --user root
    steps:
      - name: Get-scan-target
        uses: actions/download-artifact@v2 # See https://github.com/actions/download-artifact
        with:
          name: built-file
      - name: scan
        run: |
          java -jar /opt/veracode/api-wrapper.jar -vid ${{secrets.VERACODE_ID}} -vkey ${{secrets.VERACODE_KEY}} -action UploadAndScan -createprofile true -appname ${{ github.repository }} -version "${{ github.run_id }}" -sandboxname ${{needs.generate-sandbox-name.outputs.sandbox-name}} -createsandbox true  -scantimeout 30 -filepath ./verademo.war


💡 The above is also example using multiple jobs with shared artifacts between them

GitHub Action

An easier way to incorporate the upload and scan into a workflow is using Veracode supported upload-and-scan GitHub action

To write a workflow with the official action we can use the documentation in the action page

See example 

name: Veracode Static Scan

# Controls when the action will run. 
on:
  # Triggers the workflow on push or pull request events but only for the master branch
  push:
    branches: [ master, release/* ]
  pull_request:
    branches: [ master, develop, main, release/* ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: [ self-hosted, generic ]
    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      - name: Veracode Upload And Scan
        uses: veracode/[email protected]
        with:
          # appname
          appname: ${{ github.repository }}
          # createprofile
          createprofile: true
          # filepath
          filepath: result.zip #Path to dlls/src/jars/wars....
          # version
          version: ${{ github.run_id }}
          # vid
          vid: ${{ secrets.VERACODE_ID }}
          # vkey
          vkey: ${{ secrets.VERACODE_KEY }}
          # true or false
          createsandbox: true
          # name of the sandbox
          sandboxname: sandboxA
          # wait X minutes for the scan to complete
          scantimeout: 0
          # business criticality - policy selection
          criticality: "High"

Align sandbox name with branch name (Optional)

If you look into the above example, you'll noticed the sandbox name is a fixed name. A fixed sandbox name will not do well in the long run as we probably want to scan different changes/versions in a different sandboxes.

An alternative to that is to align the sandbox name with the repository branch name. In order to achieve that we can modify the workflow definition to include a step prior to the scan to save the branch name as an attribute and use it when we submit for scan.

Example workflow:

Inline example 

name: Veracode Static Scan

# Controls when the action will run. 
on:
  # Triggers the workflow on push or pull request events but only for the master branch
  push:
    branches: [ master, release/* ]
  pull_request:
    branches: [ master, develop, main, release/* ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  generate-sandbox-name:
    runs-on: ubuntu-latest
    outputs:
      sandbox-name: ${{ steps.set-sandbox-name.outputs.sandbox-name }}
    steps:
      # Creates the sandbox(logical release descriptive status of current branch)
      - id: set-sandbox-name
        name: set-sandbox-name
        run: |
          echo ${{ github.head_ref }}
          branchName="${{ github.head_ref }}"
          if [[ -z "$branchName" ]]; then
            branchName="${{ github.ref }}"
          fi
          
          if [[ $branchName == *"master"* ]]; then
          echo "::set-output name=sandbox-name::Master"
          elif [[ $branchName == *"main"* ]]; then
          echo "::set-output name=sandbox-name::Main"
          elif [[ $branchName == *"elease/"* ]]; then
          echo "::set-output name=sandbox-name::$branchName"
          else
          echo "::set-output name=sandbox-name::Development"
          fi        
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest
    needs: [ generate-sandbox-name ]
    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      - name: Veracode Upload And Scan
        uses: veracode/[email protected]
        with:
          # appname
          appname: ${{ github.repository }}
          # createprofile
          createprofile: true
          # filepath
          filepath: result.zip #Path to dlls/src/jars/wars....
          # version
          version: ${{ github.run_id }}
          # vid
          vid: ${{ secrets.VERACODE_ID }}
          # vkey
          vkey: ${{ secrets.VERACODE_KEY }}
          # true or false
          createsandbox: true
          # name of the sandbox
          sandboxname: "${{needs.generate-sandbox-name.outputs.sandbox-name}}"
          # wait X minutes for the scan to complete
          scantimeout: 0
          # business criticality - policy selection
          criticality: "High"


💡 - The above example uses multiple jobs definition which we will explain further at the Flow Control section section

Pipeline Scan

Script

The basic script for Pipeline documented at the Veracode help center and in its most basic form it looks as follow.

java -jar --veracode_api_id "${{secrets.VERACODE_ID}}" --veracode_api_key "${{secrets.VERACODE_KEY}} pipeline-scan.jar --file <file.zip>

A more advanced and context aware options are documented here.

Within a Github workflow we can add the above scan script right after a step to download the latest Pipeline Scan code.

See example 

name: Veracode Pipeline Scan

# Controls when the action will run. 
on:
  # Triggers the workflow on push or pull request events but only for the master branch
  push:
    branches: [ master, release/* ]
  pull_request:
    branches: [ master, develop, main, release/* ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest
    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      - name: create dir prior
        run: |
          mkdir dls
      - name: Download pipeline code
        working-directory: ./dls/
        run: |
          curl https://downloads.veracode.com/securityscan/pipeline-scan-LATEST.zip -o veracode.zip
          unzip veracode.zip
      - name: Run Pipeline Scanner
        continue-on-error: true
        run: java -jar ./dls/pipeline-scan.jar --veracode_api_id "${{secrets.VERACODE_ID}}" --veracode_api_key "${{secrets.VERACODE_KEY}}" --file "result.zip" -jo true -so true --project_url https://www.github.com/$GITHUB_REPOSITORY -p $GITHUB_REPOSITORY -r $GITHUB_REF


👉 - it is recommended to include the application name and if possible the sandbox or branch name in the pipeline scan attribute to help with the platform reports. [-p <project name/repository name>] [-r <project ref/branch name>]

Pre-built Docker Image

Similar to what we have for Upload-and-Scan docker image - we also have a Pipeline Scan image.

An example of scan using that image can be found at:

See inline example 

name: Veracode Pipeline Scan

# Controls when the action will run. 
on:
  # Triggers the workflow on push events but only for non-main branches
  push:
    branches: [
      - test/*
      - dev*
      - release/** ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

jobs: 
  Pipeline-scan:
    needs: build
    # The Build job omitted from the inline sample
    runs-on: ubuntu-latest
    container: 
      image: veracode/pipeline-scan:latest
      options: --user root
    steps:
      - name: get archive
        uses: actions/download-artifact@v2
        with:
          name: verademo.war
          path: /tmp
      - name: pipeline-scan
        run: | 
          java -jar /opt/veracode/pipeline-scan.jar \
            -vid "${{secrets.VERACODE_API_ID}}" \
            -vkey "${{secrets.VERACODE_API_KEY}}" \
            --fail_on_severity="Very High, High" \
            --file /tmp/verademo.war \
            -jf results.json \
            -fjf filtered_results.json 
        continue-on-error: true
      - name: save standard results
        uses: actions/upload-artifact@v1
        with:
          name: PipelineScanResults
          path: results.json
      - name: save filtered results
        uses: actions/upload-artifact@v1
        with:
          name: filtered-results
          path: filtered_results.json


Agent-Based SCA

Script

The basic script for Agent-Based SCA CLI (Command line interface) is documented at the Veracode help center

The basic form on a PC using cURL is also the simplest way to be use in a CI workflow.

curl -sSL https://download.sourceclear.com/install | sh

As all of our integrations, more advanced scan options are documented in our help center

For Agent-Based SCA (security scanning of 3rd party components) solution we have a very simple script which can easily put in a workflow.

See example 

name: SCA on change in dependencies definition

# Controls when the workflow will run
on:
  # Triggers the workflow on push where package-lock.json modifies or pull request events
  push:
    paths:
      - 'package-lock.json'
  pull_request:
  
  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # The workflow consist of a single job to quickly scan dependencies
  SCA_Scan:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest

    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - name: Check out repo 
        uses: actions/checkout@v2

      # run quick scan on the project
      - name: SCA Scan
        env: 
          SRCCLR_API_TOKEN: ${{ secrets.SRCCLR_API_TOKEN }}
        run: curl -sSL https://download.sourceclear.com/ci.sh | sh -s scan --quick


💡 The above example script is using a very specific trigger for changes in TypeScript/JavaScript packaging definition file package-lock.json. Different programming language will require different on scan trigger settings.

GitHub Action

The is an unofficial community authored GitHub Action to run the Agent-Based scan which allows scanning and break build base on finding with specific CVSS

See more details at the GitHub marketplace page of the Veracode SCA GitHub Action

See example 

name: Veracode SCA Scan

on: 
  push:
    paths-ignore:
      - "README.md"
  schedule:
    - cron: 15 14 * * 6

jobs:
  veracode-sca-task:
    runs-on: ubuntu-latest
    name: Scan repository for Issues

    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Run Veracode SCA
        env:
          SRCCLR_API_TOKEN: ${{ secrets.SRCCLR_API_TOKEN }}
        uses: lerer/veracode-sca@v1 
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          quick: true
          create-issues: false 
          fail-on-cvss: 3

Import Findings

With our different Static scanning, we now have the ability to import the findings using different techniques

Visualize Pipeline Findings as Pull Request comment

The basic form of 'import' is rather a surfacing of the result. When a Pipeline Scan run within a workflow, all messages are kept inside the workflow. However, when we are using the scan as part of a Pull Request, we may want to copy the Scan output as a comment in the Pull Request main Conversation tab.

To do that we can utilize build-in GitHub scripting functionality. (This is not a Veracode function)

See basic example

Another inline example 

name: Veracode SAST Scan

# Controls when the action will run. 
on:
  # Triggers the workflow on push or pull request events but only for the master branch
  pull_request:
    branches: [ master, develop, main, release/* ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  static_pipeline_scan:
    # The type of runner that the job will run on
    runs-on: [ self-hosted, generic ]
    
    steps:
      - name: create dir prior
        run: |
          mkdir dls
      - name: Download pipeline code
        working-directory: ./dls/
        run: |
          curl https://downloads.veracode.com/securityscan/pipeline-scan-LATEST.zip -o veracode.zip
          unzip veracode.zip
      - name: Run Pipeline Scanner
        continue-on-error: true
        run: java -Dpipeline.debug=true -jar ./dls/pipeline-scan.jar --veracode_api_id "${{secrets.VERACODE_ID}}" --veracode_api_key "${{secrets.VERACODE_KEY}}" --file "result.zip" -jo true -so true
      - id: get-comment-body
        if: ${{ github.head_ref != '' }}
        run: |
          body=$(cat results.txt)
          body="${body//$'===\n---'/'===\n<details><summary>details</summary><p>\n---'}"
          body="${body//$'---\n\n==='/'---\n</p></details>\n==='}"
          body="${body//$'\n'/'<br>'}"
          echo ::set-output name=body::$body
      - name: Add comment to PR
        if: ${{ github.head_ref != '' }}
        uses: peter-evans/create-or-update-comment@v1
        with:
          issue-number: ${{ github.event.pull_request.number }}
          body: ${{ steps.get-comment-body.outputs.body }}
          reactions: rocket

Pipeline scan as comment in PR conversation


The inline example is from a an actual pull request

💡 A different output can be done by further/differently manipulating the text output

Pipeline Scan as GitHub Security issues

For Enterprise GitHug Accounts with a license for 'GitHub Advanced Security', we can import our Pipeline Scan result to the dedicated Security issues.

That option is available in GitHub Actions marketplace - Veracode Static Analysis Pipeline Scan and SARIF import A (relatively older) use can be found at: Lerer/verademo-sarif

See example 

name: Pipeline Scan with creation of GitHub Security issues

# Controls when the workflow will run
on:
  # Triggers the workflow on push where package-lock.json modifies or pull request events
  push:
  pull_request:
  
  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # The workflow consist of a single job to quickly scan dependencies
  Pipeline_to_Security_Issues:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest

    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - name: create dir prior
        run: |
          mkdir dls
      - name: Download pipeline code
        working-directory: ./dls/
        run: |
          curl https://downloads.veracode.com/securityscan/pipeline-scan-LATEST.zip -o veracode.zip
          unzip veracode.zip
      - name: Run Pipeline Scanner
        continue-on-error: true
        run: java -jar ./dls/pipeline-scan.jar --veracode_api_id "${{secrets.VERACODE_ID}}" --veracode_api_key "${{secrets.VERACODE_KEY}}" --file "result.zip" -jo true -so true 
      - name: Convert pipeline scan output to SARIF format
        id: convert Pipeline results to SARIF format 
        uses: Veracode/[email protected]
        with:
          pipeline-results-json: results.json
          output-results-sarif: veracode-results.sarif
          source-base-path-1: "^com/veracode:src/main/java/com/veracode"
          source-base-path-2: "^WEB-INF:src/main/webapp/WEB-INF"
          finding-rule-level: "3:1:0"

      - name: upload SARIF file to repository
        uses: github/codeql-action/upload-sarif@v1
        with: # Path to SARIF file relative to the root of the repository
          sarif_file: veracode-results.sarif


Pipeline Scan results as Security issues

Policy findings as Security Issues

Similar to the above, we can also import issues from our Policy/Sandbox scans in the platform. That is again using an action.

Importing Platform static results as Security 

name: Import Static Flaws as GitHub Security issues

# Controls when the workflow will run
on:
  # Triggers the workflow based on predefined criteria
  push:
  # pull_request:
  
  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  veracode-sandbox-scan:
    # The specific build and Scan on your code
    ....

# Import flaws into code scanning alerts
  import-flaws-code-scanning-alerts:
    runs-on: ubuntu-latest
    needs: veracode-sandbox-scan
    steps:
      - name: Veracode Flaw Importer
        # Use the 'if' condition below if you only want to import on scan failure
        if: ${{ failure() }}
        uses: julz0815/veracode_flaw_importer@main
        env:
          VERACODE_API_KEY_ID: '${{ secrets.VERACODE_API_ID }}'
          VERACODE_API_KEY_SECRET: '${{ secrets.VERACODE_API_KEY }}'
        id: import_flaws
        with:
          # Application name in the platform
          app_name: 'Verademo'
          # Sandbox name in the platform
          sandbox_name: 'Github - ${{ github.ref }}'
      - uses: actions/upload-artifact@master
        with:
          name: flaws
          path: /home/runner/work/Verademo/Verademo/fullResults.json
      - uses: github/codeql-action/upload-sarif@v1
        with:
          sarif_file: /home/runner/work/Verademo/Verademo/fullResults.json

Upload and Scan / Pipeline Scan as Issues

Lastly, for organization who don't have the license for 'GitHub Advanced Security' or simply don't want to use the Security issues, they can import the finding as any other issue in the Issues section.

That is achieve by the Veracode scan results to GitHub issues action.

Example for such issues can be seen here:

Screenshots

Issues created from Pipeline Scan results


Issues created from importing Static Scan results

SCA import findings

Agent-Based - result parsing script

As simple as it get with the Agent-Based, here is an example on how to surface SCA finding from Agent-Based SCA scan within a workflow job.

Part of Workflow example 

name: Secure with SCA Agent Based

# Controls when the workflow will run
on:
  # Triggers the workflow on push where package-lock.json modifies or pull request events
  push:
    branches: [test*]
  pull_request:
  
  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:

  # The workflow consist of a single job to quickly scan dependencies
  SCA_Scan:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: SCA Scan
        env: 
          SRCCLR_API_TOKEN: ${{ secrets.SRCCLR_API_TOKEN }}
        run: |
          git version
          curl -sSL https://download.sourceclear.com/ci.sh | sh -s -- scan --recursive > veracode-sca-results.txt
      - name: Check for existing Vulnerabilities
        id: check-vulnerabilities
        run: |
          cat veracode-sca-results.txt
          TEMP_VULN_EXISTS=$(grep 'Risk Vulnerabilities' veracode-sca-results.txt | awk '{sums += $4} END { test = (sums>0); print test}')
          TEMP_VULN_SUM=$(grep 'Risk Vulnerabilities' veracode-sca-results.txt | awk '{sums += $4} END { print sums}')
          echo ::set-output name=fail::$TEMP_VULN_EXISTS
          echo ::set-output name=sums::$TEMP_VULN_SUM
      - name: SCA Pass
        if: ${{ steps.check-vulnerabilities.outputs.fail == 1 }}
        uses: actions/github-script@v3
        env:
          VULN_NUM: ${{ steps.check-vulnerabilities.outputs.sums }}
        with:
          script: |
            console.log(process.env.VULN_NUM);
            core.setFailed(`Found ${process.env.VULN_NUM} Risk Vulnerabilities in your open source libraries`);

Github workflow summary with multiple jobs


Agent-Based - via GitHub Action to Issues

With the same Veracode Software Composition Analysis GitHub Action mentioned above, you can use the Veracode Agent-based scan and convert the output to GitHub Issues.

See example 

name: Veracode SCA Scan

on: 
  push:
    paths-ignore:
      - "README.md"
  schedule:
    - cron: 15 14 * * 6

jobs:
  veracode-sca-task:
    runs-on: ubuntu-latest
    name: Scan repository for Issues

    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Run Veracode SCA
        env:
          SRCCLR_API_TOKEN: ${{ secrets.SRCCLR_API_TOKEN }}
        uses: lerer/veracode-sca@v1 
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          quick: true
          create-issues: true 
          fail-on-cvss: 1
          min-cvss-for-issue: 1


Unfortunately, as of now, we don't have an official (or unofficial) simple action to import finding for SCA result - Upload and Scan.

❗ This is a place-holder for further future implementation.

From Melvin Conway - "Organizations, who design systems, are constrained to produce designs which are copies of the communication structures of these organizations." (Copyright 1967)

I added the above quote as it always pop in my mind when I try to show a workflow and the other side is responding with a variation of: "But, this is not how we do things here...".

All the different "utilities" mentioned in the previous sections are simply building blocks which can be used in different combinations. Every organization eventually have to decide what best serve their existing or intended processes.

We can however (and in some situations should) suggest or recommend few best practices.

The next sections are few suggestions!

Flow Control

This section is mostly around utilizing GitHub build-in functionality to implement a desire process.

Branch protection (good practice for Pull Request)

Protected Branches feature is the answer to a (repeated) question - How can we block a Pull Request for failed scan.

The answer is in a 'Require Status Checks Before Merging' section in the above manual.

For those with the right GitHub permissions, here are the Step-by-Step instructions for setting up the Branch Protection Rules

If we enable branch protection, any failed job within a workflow can prevent pull request from being approved ➡️ We then need to make sure a failed Scan is also failing the job in the workflow❗

Screenshot from Github

Github branch protection rule definition example


PR blocked due to failed checks

Splitting into Jobs - Adding Checks

Another scenario is for example if we want to have parallel different scan processes to achieve:

  • Parallel processing - faster overall flaw time
  • Or, separate statuses - for example SCA and Static.

💡 Every job in a workflow will report back its own status as a check.

We can either create separate workflows or we can create a single flow with separate to Jobs.

The following example is taken from this implementation

Example for flaw output with multiple jobs

Github workflow summary with multiple jobs

Workflow example with parallel Agent-Based and Pipeline Scan 

name: Secure with multiple separate jobs

# Controls when the workflow will run
on:
  # Triggers the workflow on push where package-lock.json modifies or pull request events
  push:
    branches: [test*]
  pull_request:
  
  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  Build:
    runs-on: [ubuntu-latest]
    steps:
      - name: Check out repo 
        uses: actions/checkout@v2
      - name: Set up JDK 1.8
        uses: actions/setup-java@v1
        with:
          java-version: 1.8
      - name: Build with Maven
        working-directory: ./app/
        run: mvn -B package --file pom.xml
      - name: Zip Files
        uses: vimtor/action-zip@v1
        with:
          files: ./app/target/verademo.war
          dest: result.zip
      - uses: actions/upload-artifact@v2 # Copy files from repository to docker container so the next uploadandscan action can access them.
        with:
          path: result.zip # See: https://github.com/actions/upload-artifact
          name: built-zip
        
  generate-sandbox-name:
    runs-on: [ ubuntu-latest ]
    outputs:
      sandbox-name: ${{ steps.set-sandbox-name.outputs.sandbox-name }}
    steps:
      # Creates the sandbox(logical release descriptive status of current branch)
      - id: set-sandbox-name
        name: set-sandbox-name
        run: |
          echo ${{ github.head_ref }}
          branchName="${{ github.head_ref }}"
          if [[ -z "$branchName" ]]; then
            branchName="${{ github.ref }}"
          fi
          echo "::set-output name=sandbox-name::$branchName"
  
  Pipeline_Scan:
    runs-on: ubuntu-latest
    needs: [ Build, generate-sandbox-name ]
    steps:
      - uses: actions/download-artifact@v2
        with:
          name: built-zip
      - name: create dir prior
        run: |
          mkdir dls
      - name: Download pipeline code
        working-directory: ./dls/
        run: |
          curl https://downloads.veracode.com/securityscan/pipeline-scan-LATEST.zip -o veracode.zip
          unzip veracode.zip
      - name: Run Pipeline Scanner
        run: java -jar ./dls/pipeline-scan.jar --veracode_api_id "${{secrets.VERACODE_ID}}" --veracode_api_key "${{secrets.VERACODE_KEY}}" --file "result.zip" -jo true -so true
       
  # The workflow consist of a single job to quickly scan dependencies
  SCA_Scan:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest
    needs: [ Build, generate-sandbox-name ]
    steps:
      - uses: actions/checkout@v2
      - name: SCA Scan
        env: 
          SRCCLR_API_TOKEN: ${{ secrets.SRCCLR_API_TOKEN }}
        run: |
          git version
          curl -sSL https://download.sourceclear.com/ci.sh | sh -s -- scan --recursive > veracode-sca-results.txt
      - name: Check for existing Vulnerabilities
        id: check-vulnerabilities
        run: |
          cat veracode-sca-results.txt
          TEMP_VULN_EXISTS=$(grep 'Risk Vulnerabilities' veracode-sca-results.txt | awk '{sums += $4} END { test = (sums>0); print test}')
          TEMP_VULN_SUM=$(grep 'Risk Vulnerabilities' veracode-sca-results.txt | awk '{sums += $4} END { print sums}')
          echo ::set-output name=fail::$TEMP_VULN_EXISTS
          echo ::set-output name=sums::$TEMP_VULN_SUM
      - name: SCA Pass
        if: ${{ steps.check-vulnerabilities.outputs.fail == 1 }}
        uses: actions/github-script@v3
        env:
          VULN_NUM: ${{ steps.check-vulnerabilities.outputs.sums }}
        with:
          script: |
            console.log(process.env.VULN_NUM);
            core.setFailed(`Found ${process.env.VULN_NUM} Risk Vulnerabilities in your open source libraries`);

Example for how check will show as part of Pull Request

Github pull request checks with multiple jobs

The above example is showing a Pull request without Branch Protection enabled

Auto Pull request for vulnerable dependencies after merge into Main/Master

Right after a pull request is approved, the target branch will issue a push event (as new code is merged). This is an opportunity to run a scan to automatically produce a Pull request for known 3rd party components vulnerabilities (based on Vulnerable Methods and/or Severity of Vulnerabilities found in a scan).

Veracode SCA Auto pull request only apply to the following supported languages: Java, Python, Ruby, JavaScript, Objective-C, and PHP.

❗ Make sure to read the instructions as it involves creation of Github token.

See pull Request example:

See inline workflow 

name: Auto Pull Request on SCA findings

# Controls when the workflow will run
on:
  # Triggers the workflow on push where package-lock.json modifies or pull request events
  push:
    branches: [ master,main ]
  
  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # The workflow consist of a single job to quickly scan dependencies
  SCA_Scan for PR:
    # The type of runner that the job will run on
    runs-on: [ self-hosted, generic ]

    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - name: Check out repo 
        uses: actions/checkout@v2

      # run quick scan on the project
      - name: SCA Scan
        env: 
          SRCCLR_API_TOKEN: ${{ secrets.SRCCLR_API_TOKEN }}
          SRCCLR_SCM_TOKEN: ${{ secrets.SRCCLR_GITHUB_TOKEN }}
          SRCCLR_SCM_TYPE: GITHUB
          # PR on: methods
          SRCCLR_PR_ON: low
          SRCCLR_NO_BREAKING_UPDATES: true
          SRCCLR_IGNORE_CLOSED_PRS: true
          EXTRA_ARGS: '--update-advisor --pull-request --unmatched'
        run: |
          git config --global user.email "${{ secrets.USER_EMAIL }}"
          git config --global user.name "${{ secrets.USER_NAME }}"
          curl -sSL https://download.sourceclear.com/ci.sh | sh -s -- scan $EXTRA_ARGS

Policy scan output as GitHub Issue

Scaling in an Organization

For larger organization who seek more help on how to utilize the above suggestions and ease the onboarding process of their different teams - to encouraging faster and easier adoption of DevSecOps.

Shared workflow templates

Workflow define and save in each repository, however, instead of copy-paste for every repository we can use GitHug shared workflows by creating .github repository in the account.

In addition to shared workflows, you should also consider Organization shared Secrets

❗ Keep in mind - Pipeline Scan throughput limited to 6 scans/min

See following shared workflows example:

How does it look like in workflow creation screen

New workflow screen

For the above example, the Organization name is: lerer-veracode

Use Pipeline scan baseline

Pipeline scan provides the ability to use baseline acting as the "approved mitigations" or accepted risk level which instruct the Pipeline Scan to return finding other than the ones in the provided baseline.

It can be useful to re-base the Baseline file after approving a Pull Request

Only if we OK to look at Pull Request approval also as an Approval for accepting security risk going into certain (main/master) branches.

An option to handle such a rebase is to commit the baseline as a file in the repository. The advantage in this option is to have the baseline file available to developers working with the code and synchronize the baseline with their local cloned repository.

Example - commit baseline file after Approved Pull request to main/master 

name: Push baseline file

# Controls when the workflow will run
on:
  push:
    branches: [master , main]
  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:  
  Build:
    runs-on: [ubuntu-latest]
    steps:
      - name: Check out repo 
        uses: actions/checkout@v2
      - name: Set up JDK 1.8
        uses: actions/setup-java@v1
        with:
          java-version: 1.8
      - name: Build with Maven
        working-directory: ./app/
        run: mvn -B package --file pom.xml
      - name: create dir prior
        run: |
          mkdir dls
      - name: Download pipeline code
        working-directory: ./dls/
        run: |
          curl https://downloads.veracode.com/securityscan/pipeline-scan-LATEST.zip -o veracode.zip
          unzip veracode.zip
      - name: Run Pipeline Scanner
        # Baseline should only be created when using filtered results without baseline file as input
        run: |
          java -jar ./dls/pipeline-scan.jar --veracode_api_id "${{secrets.VERACODE_ID}}" --veracode_api_key "${{secrets.VERACODE_KEY}}" --file "app/target/verademo.war" -fs "Very High, High" -jo true -so true --project_url https://www.github.com/$GITHUB_REPOSITORY -p $GITHUB_REPOSITORY -r $GITHUB_REF 
        continue-on-error: true
      - name: list files
        run: |
          yes | cp filtered_results.json baseline.json
          git config user.email "${{secrets.USER_EMAIL}}"
          git config user.name "${{secrets.USER_NAME}}"
          git add "baseline.json"
          git commit -m "Updates baseline"
          # git push origin ${{ github.head_ref }}
          git push origin ${{ github.ref }}


💡 Do not use a baseline file in the Pipeline scan command in order to generate a new baseline file, as the filtered_results.json file will not output what is needed as a new baseline file

Shared Downloaded Policies for Pipeline Scan

Pipeline scan today can download a policy file based on policies defined in the Veracode Platform. We can subsequently use a downloaded policy as an input to a Pipeline scan as a fail/pass condition - similar to the Upload-and-Scan logic.

However, we don't recommended downloading the policy file every time we run a pipeline scan:

  • Policies are rarely changed (compared to the scanned code)
  • Policies are shared across multiple Application profile - hence, shared between multiple repositories

...maybe we can store it in a shared location...and alias their name to protect from policy name change...

We can control and only refresh policy files once a day and store in a shared location to be used by any other repository workflow.

Here is an option:

  1. Let's download and store policies using a nightly scheduled workflow to a shared repository
  2. We will download the policy from the shared repository prior to running pipeline scan in a workflow
  3. We will template the above for the easy import from any other repository
Storing policies in a shared Repository 

name: Update Latest Policies for Veracode Pipeline Scanner 
on:
  workflow_dispatch:
  
  schedule:
    - cron: 30 14 * * *

jobs:
  download_policies_global:
    runs-on: ubuntu-latest
    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - uses: actions/checkout@v2
      - name: create dir prior
        run: |
          git pull
          mkdir dls
      - name: Download pipeline code
        working-directory: ./dls/
        run: |
          curl https://downloads.veracode.com/securityscan/pipeline-scan-LATEST.zip -o veracode.zip
          unzip veracode.zip
      - name: Download Policy High Risk
        continue-on-error: true
        run: java -jar ./dls/pipeline-scan.jar --veracode_api_id "${{secrets.VERACODE_ID}}" --veracode_api_key "${{secrets.VERACODE_KEY}}" --request_policy "Fix High Severity - non scored"
      - name: Download Policy Highly Sensitive
        continue-on-error: true
        run: java -jar ./dls/pipeline-scan.jar --veracode_api_id "${{secrets.VERACODE_ID}}" --veracode_api_key "${{secrets.VERACODE_KEY}}" --request_policy "Acceptable quality"
      - name: Clean-ups and alias
        run: |
          echo "Remove veracode files"
          rm -rf dls/*
          mkdir -p sec
          mv -f Fix_High_Severity_-_non_scored.json sec/High.json
          mv -f Acceptable_quality.json sec/HighlySensitive.json
      - name: Push the policies
        continue-on-error: true
        run: |
          git config user.email "${{secrets.USER_EMAIL}}"
          git config user.name "${{secrets.USER_NAME}}"
          git add "sec/High.json"
          git add "sec/HighlySensitive.json"
          git commit -m "Updates Policies"
          git push origin ${{ github.ref }}

Downloading the policy from another workflow 

name: Pipeline Scan with remote policy

# Controls when the workflow will run
on:
  push:
  pull_request:
  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:  
  Build:
    runs-on: [ubuntu-latest]
    steps:
      - name: Check out repo 
        uses: actions/checkout@v2
      - name: Set up JDK 1.8
        uses: actions/setup-java@v1
        with:
          java-version: 1.8
      - name: Build with Maven
        working-directory: ./app/
        run: mvn -B package --file pom.xml
      - name: create dir prior
        run: |
          mkdir dls
      - name: Download pipeline code
        working-directory: ./dls/
        run: |
          curl https://downloads.veracode.com/securityscan/pipeline-scan-LATEST.zip -o veracode.zip
          unzip veracode.zip
      - name: Get policy file
        run: | 
          curl https://raw.githubusercontent.com/lerer-veracode/policies/main/sec/High.json -o policy.json
      - name: Run Pipeline Scanner
        # Baseline should only be created when using filtered results without baseline file as input
        run: |
          java -jar ./dls/pipeline-scan.jar --veracode_api_id "${{secrets.VERACODE_ID}}" --veracode_api_key "${{secrets.VERACODE_KEY}}" --file "app/target/verademo.war" --policy_file "./policy.json" -jo true -so true --project_url https://www.github.com/$GITHUB_REPOSITORY -p $GITHUB_REPOSITORY -r $GITHUB_REF

Downloading the policy from another workflow - as template (Java) 

name: Pipeline Scan with remote policy

# Controls when the workflow will run
on:
  push:
  pull_request:
  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:  
  Build:
    runs-on: [ubuntu-latest]
    steps:
      - name: Check out repo 
        uses: actions/checkout@v2
        # Replace compilation for different languages
      - name: Set up JDK 1.8
        uses: actions/setup-java@v1
        with:
          java-version: 1.8
      - name: Build with Maven
        working-directory: ./app/
        run: mvn -B package --file pom.xml
      - name: create dir prior
        run: |
          mkdir dls
      - name: Download pipeline code
        working-directory: ./dls/
        run: |
          curl https://downloads.veracode.com/securityscan/pipeline-scan-LATEST.zip -o veracode.zip
          unzip veracode.zip
      - name: Get policy file
        # Remove all import policies which are not relevant to your code and
        # keep only the line with the policy relevant to your code in the repository
        run: | 
          curl https://raw.githubusercontent.com/lerer-veracode/policies/main/sec/HighlySensitive.json -o policy.json
          curl https://raw.githubusercontent.com/lerer-veracode/policies/main/sec/High.json -o policy.json
      - name: Run Pipeline Scanner
        # Modify target scan file (here "app/target/verademo.war") for different code base or languages 
        run: |
          java -jar ./dls/pipeline-scan.jar --veracode_api_id "${{secrets.VERACODE_ID}}" --veracode_api_key "${{secrets.VERACODE_KEY}}" --file "app/target/verademo.war" --policy_file "./policy.json" -jo true -so true --project_url https://www.github.com/$GITHUB_REPOSITORY -p $GITHUB_REPOSITORY -r $GITHUB_REF


💡 The last example is what you would put in the workflows shared via your organization .github repository.

Should we run a Policy Scan?

Technically the above different scan options enable you to execute direct scans as Policy Scan. However, after working with multiple customers, some prefers to only run scans in a Sandbox - including the Main branch.

Those who are using purely Sandbox scanning prefer to use the manual "Promote Scan" in the platform as a Security control.

Example for branch name setup 

name: Veracode Static Scan

# Controls when the action will run. 
on:
  # Triggers the workflow on push or pull request events but only for the master branch
  push:
    branches: [ master, release/* ]
  pull_request:
    branches: [ master, develop, main, release/* ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  generate-sandbox-name:
    runs-on: ubuntu-latest
    outputs:
      sandbox-name: ${{ steps.set-sandbox-name.outputs.sandbox-name }}
    steps:
      # Creates the sandbox(logical release descriptive status of current branch)
      - id: set-sandbox-name
        name: set-sandbox-name
        run: |
          echo ${{ github.head_ref }}
          branchName="${{ github.head_ref }}"
          if [[ -z "$branchName" ]]; then
            branchName="${{ github.ref }}"
          fi
          
          if [[ $branchName == *"master"* ]]; then
          echo "::set-output name=sandbox-name::Master"
          elif [[ $branchName == *"main"* ]]; then
          echo "::set-output name=sandbox-name::Main"
          elif [[ $branchName == *"elease/"* ]]; then
          echo "::set-output name=sandbox-name::$branchName"
          else
          echo "::set-output name=sandbox-name::Development"
          fi        
  
  Job-A:
    .....

What if we do want to automatically promote - on Pull request approval?

In a scenario which you would like to automatically promote a scan to a policy, you can use the following Veracode Application Sandboxes Helper Action which allows you to promote a scan from a sandbox to a policy. As well, you can use the same GitHub Action to also delete the Sandbox right after the scan was promoted to a Policy (maybe as a clean-up activity)

Example for Scan promotion on approved pull request 

name: Veracode Static Scan

# Controls when the action will run. 
on: 
  pull_request_review: # a trigger when a Pull Request Review submitted
    types: [submitted]
        
  workflow_dispatch:

jobs:
  veracode-promote-to-policy:
    runs-on: ubuntu-latest
    name: Pull Request Review Submitted

    steps:

      ... # your other job steps

      - name: Promote Scan on Approval
        # run only if the pull request got approved
        if: ${{ github.event.review.state == 'approved' }}
        env:
          VERACODE_API_ID: '${{ secrets.VERACODE_API_ID }}'
          VERACODE_API_SECRET: '${{ secrets.VERACODE_API_SECRET }}'
        uses: lerer/veracode-sandboxes-helper@v1 
        with:
          activity: "promote-latest-scan"
          app-name: "<YOUR VERACODE APPLICATION NAME>"
          sandbox-name: "<SANDBOX_NAME>" # "${{ github.event.pull_request.head.ref }}"
          delete-on-promote: true # Optional: also Deleting the Sandbox


❕ It is up to each Organization (or business unit) to define their preferred controls and processes.

❕ Organizations with integration into their CD process may decide to run a Policy Scan on what is just or about to be deploy or deliver

How can you help - open for needed contribution

The following list of items are feature which currently don't exist (or, I am not aware of) which would be nice if they will be available as a nicely packaged Action:

  • Enhance the SCA Agent Based Action to support issues from ➡️ Licenses, and Outdated libraries
  • Generation of Pipeline Scan Baseline file from Static Profile mitigations
    • Some work was done by Tim Jarrett here using Python script which can be leveraged
  • Policy/Sandbox scan ➡️ Summary report action

veracode-manual-for-github's People

Contributors

lerer avatar

Stargazers

avatar avatar avatar

Watchers

avatar avatar avatar

Forkers

veracodevince datalogics-aarroyo tjarrettveracode julz0815 christyson

veracode-manual-for-github's Issues

Suggestion: supporting PR comments instead of GH issues

Hi !
We're currently users of Veracode, mostly with jenkins, and scanning PRs is really the best of both worlds in my opinion.

I know other scanners are able to only review the diff, not with a deep SAST analysis of course, but they do gain a simple advantage... they prevent introducing the issues in the production branches efficiently.

Now I know that pipeline scan is intended for that, and it may be fairly fast, but the main and only gripe I have is that most of the support is on Github issues, which don't really translate to issues that "don't really exist yet", since they are not merged yet.

In my opinion the comments in PRs should be used, and I maintain https://github.com/security-alert/security-alert for that purpose (using SARIF input, to comment issues in PRs)

If the veracode wrapper could do that out of the box it would be great for compatibility (and not just GH actions)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.