microsoft / sarif-tutorials

User-friendly documentation for the SARIF file format.

SARIF Tutorials

Introduction

SARIF, the Static Analysis Results Interchange Format, defines a standard format for the output of static analysis tools. It is a powerful and sophisticated format suited to the needs of a wide variety of tools. For this reason — and because the format is defined in a 220-plus page specification written in formal language! — it can be hard to learn SARIF and to figure out what parts of it you need to use.

These tutorials aim to present SARIF in a more approachable way. We'll start with some background: Why do we need SARIF? Where did it come from? What can it do? Then we'll dive into the format, exploring the most basic concepts first, then moving on to more advanced concepts.

The advanced concepts usually apply to only a subset of SARIF producers and consumers, so you don't to read everything. Just read the introductory material, then pick and choose the additional topics that interest you.

Sample files

You can find the sample files displayed in the tutorials under the samples directory. They are all valid SARIF files unless I say otherwise.

Links to the specification

At times, the tutorials link to a section of the SARIF specification for more detailed information or descriptions of advanced scenarios. These links look like this: §3.13: sarifLog object and they point into the HTML version of the spec. There are also PDF and .docx versions in the SARIF 2.1.0 CS01 (Committee Specification 1) folder on the OASIS web site.

The specification is definitive (and if we want to get really technical, the .docx version of the specification is definitive, but let's assume there are no bugs in the PDF or HTML converters). If it seems like something in these tutorials disagrees with the spec, let me know and I'll either fix the tutorials or make sure that a bug is filed against the spec.

Disclaimer

The SARIF specification is a Committee Specification from OASIS. But despite the fact that I'm the co-Editor (with Michael Fanning) and primary wordsmith of the specification, these tutorials are not an OASIS work product or endorsed by OASIS in any way. They represent my personal interpretation and explanation of the standard.

Status

These tutorials now contain enough information to give you a solid background in SARIF. As you will see from the "TODO" entries in the table of contents, there's much more I'd like to write about. But these are advanced topics that I will address when I have time.

If you'd like an explanation of a SARIF feature that I haven't covered, please let me know by filing an issue in this repo

Table of contents

• Introduction
  • What is SARIF?
  • About static analysis tools
  • Why SARIF?
  • A simple example
  • The plan for these tutorials
• The basics
  • The log file and the object model
  • Logs and runs
  • Tools: driver and extensions
  • Property bags
  • Results
    • Message
    • Rule identifier
    • Level
    • Locations
      • The locations array
      • Physical and logical locations
  • Artifacts
    • Defining artifacts
    • Linking results to artifacts
  • Rule metadata
• Beyond the basics
  • Related locations
  • More about messages
    • Markdown messages
    • Messages from metadata
    • Messages with arguments
    • Messages with embedded links
      • Links in text and Markdown
      • Links to locations
  • Invocations
  • Notifications
    • Tool execution notification and tool configuration notifications
    • Notifications vs. results
  • Taxonomies
  • Code flows
  • Automation
    • The id and guid properties
    • The correlationGuid property
• Advanced topics
  • Tool extensions (TODO)
  • Translations (TODO)
  • Handling large files (TODO)
  • Result matching (TODO)
  • The sarif: URI scheme (TODO)
• Appendices
  • Fitness for purpose: An overview
  • Fitness for purpose: Automatic bug filing
  • Authoring rule metadata and result messages
  • Displaying results in a viewer
  • The SARIF Multitool
  • The history of SARIF (TODO)
  • Glossary
  • Resources

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Legal Notices

Microsoft and any contributors grant you a license to the Microsoft documentation and other content in this repository under the Creative Commons Attribution 4.0 International Public License, see the LICENSE file, and grant you a license to any code in the repository under the MIT License, see the LICENSE-CODE file.

Microsoft, Windows, Microsoft Azure and/or other Microsoft products and services referenced in the documentation may be either trademarks or registered trademarks of Microsoft in the United States and/or other countries. The licenses for this project do not grant you rights to use any Microsoft names, logos, or trademarks. Microsoft's general trademark guidelines can be found at http://go.microsoft.com/fwlink/?LinkID=254653.

Privacy information can be found at https://privacy.microsoft.com/en-us/

Microsoft and any contributors reserve all other rights, whether under their respective copyrights, patents, or trademarks, whether by implication, estoppel or otherwise.

sarif-tutorials's Issues

Explain how to compute effective failure level based on configuration

One contributor to a result's default display state in a viewer (see #23) is its level. The spec explains how the effect level is computed in a complicated way based on result.level, rule.defaultConfiguration.level, and applied policy.

Add an Appendix that describes this algorithm (which viewers must implement, and for which there really should be an SDK API) in terms a human can understand.

@michaelcfanning FYI.

Update "$Schema" from rtm.5 in samples/*.sarif

The samples in this repo all use "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json"
which does not appear to be recognized by the viewer at https://sarifweb.azurewebsites.net/

Provide a comprehensive set of samples for use as documentation and as test assets

This is a tracking item that will contain a comprehensive list of the samples we want to provide, with check boxes to mark those that are finished.

• Code flows: Demonstrate codeFlows with features such as location messages and importance.
• Code flows as "event sequences" (call stack per location).
• Stacks
• Rule metadata: Demonstrate rule metadata and its linkage with results.
• Embedded text content
• Embedded binary content
• Snippets, with text, binary, and rendered properties.
• Region and context region: Including snippets in both properties.
• Region variants: line/column, charOffset/charLength, byteOffset/byteLength, combinations.
• originalUriBaseIds: including chaining, descriptions, and top-level element with no uri.
• Complex Markdown in messages.
• external property files: including dictionary-valued external properties, array-valued external properties, and array-valued properties split across multiple files.
• internalExternalProperties
• taxonomies
• translations
• Tool plug-ins: Including rule metadata lookup in plug-ins via toolComponentReference.
• Policies: Showing override of defaultConfiguration.
• suppressed results, including use of suppression status to show review progress.
• baselines: Showing all of unchanged, updated, absent, and new results.
• logicalLocations: Including run.logicalLocations, parenting, fully qualified names, and references through index in a result.
• Integer index links: Including references to related locations, codeFlow locations, and stack locations, and showing links in both Markdown and plain-text messages.
• Fixes: Using the HTML attribute quoting example.
• Redaction of sensitive properties: Exhaustive set.
• "Arguments-only" messges.
• Multiple runs in a single log file.
• Run with no results, but with toolConfigurationNotifications and toolExecutionNotifications (including exceptions), with all different failure levels.
• Non-failure results, e.g., "pass" and "informational" (exhaustive).
• Addresses
• Attachments: both file- and run-level.
• Web requests and responses.
• Decorated name.
• Version control details
• Run automation details
• Graphs: Result-level and run-level.
• Comprehensive result (including codeFlow) driven entirely by logical locations.

Uniquely qualifying details should be as leftmost as possible in plaintext messages.

Probably could add this subtlety close to where we discuss the summary of issue because it's due to the related underlying concern, truncated screen real estate.

When viewing large #'s of issues in the VS error list, if you put the uniquely qualifying data, such as the scan target file name, towards the left, it is easier for users to distinguish results.

If you preface each result with 128 characters of static text, it becomes much harder.

The first sentence should provide a concise summary of the issue. This is important because it enables the reader to quickly understand the problem and because some viewers might truncate the message to one sentence in contexts where screen real estate is limited.

Explain uriBaseIds and originalUriBaseIds

Document usage of samples

Per @michaelcfanning:

We should produce documentation that describes the sample files and how they should be used by consumers. This will allow viewer developers to try to exercise the test assets in a productive way without understanding all the many nuances expressed in the SARIF spec (which is only obvious to people with a deep understanding of the spec). For example:

• The "suppressions" sample should explicitly call out that this data is designed to ensure that viewers correctly exclude or include suppressed results (and don't surprise users by showing those results in a way that suggests they are active and should be addressed).
• The originalUriBaseIds sample should be helpful for viewers that are attempting to remap relative links in a SARIF log file. If a log is viewed in an environment that maps directly to the paths expressed in an original uri base id (i.e., the local enlistment matches the analysis, or the local machine was itself used for the scan), consumers should be able to successfully stitch together working links, using other information expressed in the physical location data.

Can the location in the result object be clickable in web view?

I know when we double click the path in VS Code, the editor will bring us to the exact location of the error. Do we have similar feature in web view?

In my repo, I have a file named urls.md under the docs/docs2 folder and I define the location in the following way:

         "locations": [
            {
              "physicalLocation": {
                "artifactLocation": {
                  "uri": "docs/docs2/urls.md",
                },
                "region": {
                  "startLine": 11,
                  "startColumn": 7
                }
              }
            }
          ]

When I run this in an Azure Pipeline, the path is just not clickable. However, if I replaced the uri property with some an absolute url, such as https://bing.com, it is clickable.

Provide guidance on "perceived severity".

Four SARIF result properties interact to determine

1. Whether a viewer should display the result by default, and
2. With what "severity" a viewer should present the result.

Those properties are level, kind, baselineState, and suppressions. The default visibility also depends on the scenario. For example, in a CI scenario, only results with baselineState: "new" should be displayed by default, while in other scenarios, the "unchaged" results should also be displayed by default.

Add an Appendix providing rules for a uniform viewer experience based on these factors.

@michaelcfanning FYI

docs/Multitool.md shows dependency on unsupported . net core 2.1

https://dotnet.microsoft.com/en-us/download/dotnet/2.1 shows that net core 2.1 is EOL and unsupported, but docs/Multitool.md says is as a prerequisite.

The sarif.multitool I have uses netcoreapp3.1 or something as a runtime..

Does this tool work with netcore 6.x or 8.x?

Provide a "fitness for purpose" Appendix for GitHub Developer Security Portal ingestion

... similar to the one for bug filing.

@michaelcfanning FYI

Change all ruleIds to TUT1nnn, e.g., TUT1001