microsoft / sarif-sdk

.NET code and supporting files for working with the 'Static Analysis Results Interchange Format' (SARIF, see https://github.com/oasis-tcs/sarif-spec)

sarif-sdk

The SARIF SDK contains .NET code and supporting files for working with the Static Analysis Results Interchange Format (SARIF). For more information about SARIF, see the SARIF Home Page. You can read the SARIF specification, or file issues in the SARIF GitHub repo.

Getting started

To add the SARIF SDK to your project, install the Sarif.Sdk NuGet package. Sarif.Sdk depends on Newtonsoft.Json, which is installed automatically when you install Sarif.Sdk.

The types in the SARIF SDK are in the Microsoft.CodeAnalysis.Sarif namespace.

The SARIF SDK provides a set of classes which represent the elements of the SARIF format. We refer to this as the "SARIF object model". The root type that represents a SARIF log file is SarifLog. Other types in the SARIF object model are Result, PhysicalLocation, etc..

Note: The SARIF SDK's build process automatically generates the SARIF object model classes from the SARIF JSON schema, which you can find at src/Sarif/Schemata/sarif-schema.json. Although these files do exist in the repo (under src/Sarif/Autogenerated), you should never edit them by hand.

In addition to the object model, the SARIF SDK provides a set of helper classes to facilitate using Newtonsoft.Json to read and write SARIF log files.

Building the SDK

If you want to build the SDK from source, rather than consuming the NuGet package, please refer to Devbox Setup for help setting up a compilation environment.

Then you should be able to build in Visual Studio or on the Command Line using BuildAndTest.cmd.

Accomplishing common tasks

To learn how to accomplish common tasks with the SARIF SDK, such as reading and writing files from disk, see the How To page.

Code of conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information, see the Code of Conduct FAQ, or contact [email protected] with any additional questions or comments.

sarif-sdk's Issues

Some of the test files in the functional test project are invalid

Introducing a functional test to validate the files in SarifConverterTestData directory showed that 22 of them are invalid. Skip the test until the files are fixed.

This issue is to fix the files, then reenable the test.

Create exhaustive sample SARIF file

An exhaustive fictional SARIF file that populates every possible thing. Each thing should have multiple instances where permissible (two may be sufficient). This file will be a useful example, help drive testing for consumers, and help drive design.

Write functional test to detect breaking schema changes

• Acquire a set of sample SARIF files produced by various tools.
• Create a test that validates them against the current schema

The idea is that if the test fails, we report the breaking change to our partners so they can update their tools to produce the new format version.

@michaelcfanning, @srivatsn: FYI. I entered this item to track the work I mentioned in today's standup.

Test 'related locations' functionality

We should test this using XSS detection or perhaps a JS 'you overwrote a global' analysis.

Generate rule documentation and helper classes from RuleDescriptors

Create a tool which generates rule documentation and helper classes from RuleDescriptors.

It might ease adoption of the sarif SDK if we had tooling which took the metadata from the RuleDescriptors and created rule documentation, classes for each rule and any associated helper methods (e.g. methods to create result objects for rules.) The adoption workflow would look like this.

1. Developer authors the RuleDescriptors for the static analysis tool.
2. Developer runs an SDK tool which consumes the RuleDescriptors and creates help documentation for each rule.
3. Developer runs an SDK tool (this could be the same tool as in step 2) which consumes the RuleDescriptors and creates C# classes/files for each rule. The rule strings are put into resource files. The classes have helper methods which the developer can use to access the strings or perhaps create Result objects for the rule.

SARIF validator console output should be in standard VS format

PDB exceptions don't necessarily require stack reporting

For a missing PDB, we don't need to report the full stack. We should alter the core logging utility to accept a string message (which can be a concise 'could not locate PDB' message or a complete stack, a la ex.ToString()', up to the discretion of the caller).

We need complete validation for JSON files

we should have helpers/visitors in the SDK that perform comprehensive, detailed validation.
our JSON schema needs to exercise more validation capabilities, such as oneOf. We don't do this today because we generate the schema from a G4. when we author the JSON schema directly, we should build it out

Viewer should show PREfast code paths through function

We should ask Kathir for some SARIF results that contain path-through-function metadata. Viewer should highlight these paths on navigating to a specific associated result.

Schema: property name "tag" s.b. "tags" throughout

SARIF representations should be driven by JSON scheme

The current G4-driven mechanism is not sufficiently expressive for deriving the JSON schema. We should investigate hand-maintaining a JSON scheme and constructing managed serialization helpers from that form.

resultLog.runLogs.runInfo should contain the list the configuration values for the run

It would be nice of the configuration values for the run were stored in resultLog.runLogs.runInfo. While most of the configuration can be gleaned from the command line, if the user specifies the "default" configuration for the run, the person reading the log file might not know what the "default" settings are for the tool. It would be better if run configuration were specifically detailed in the runInfo section.

Schema: replacement.insertedBytes s.b. string, not integer

I know I filed this yesterday but I don't see it. Feel free to close the dup when/if you find it.

Write VSIX for SARIF language service

The idea is, you open a SARIF file in VS, and it shows sqiggles on (for example) invalid property names or invalid object content.

Scrub for localization before things get out of hand

We need an actual icon in our VS menu item rather than the default.

I have a triskele graphic that we would like to use for SARIF. We need to convert this graphic into an appropriate bundle of images and get them integrated in the VSIX.

We need a tagger integrated in the viewer that adds squigglies for all files.

On navigating to an issue in a previously unopened file, the file should open in the editor and squigglies should be overlaid everywhere we see an issue. The squiggly should provide a pop-up tool-tip with the full message text. Ultimately, a right click option on the squiggly that integrates with our up and down vote feature would be good.

Note that when squigglies are enabled, we might not need the navigation code that currently selects the relevant code range associated with a static analysis result on double-click. We should see how this behavior looks when we get to building this feature.

Add an optional property to ruleDescriptor which specifies the location of the online definition of the rule

It would be nice if ruleDescriptor had a URI property which specifies an endpoint where the rule definition can be retrieved from.

In our meeting, we had a deeper conversation about how this property could be used. In general, we thought the URI (and it's contents) can be thought of as a contract for a rule. For example, if the URL was http://sarif.microsoft.com/codedefects/unusedusingstatements, this indicates that the rule professes to validate that there are no unused using statements. The URI describes the defect and the rule verifies the defect does not exist. The relationship between URI(defects)->rules is not 1-to-1. Many rules may profess to verify a defect. This also implies that many tools may profess to verify a defect.

Validator tool should be based on Driver skeleton

ConsoleLogger should use extension method Result.FormatForVisualStudio to create error messages

Provide an overload of the FormattedMessage ctor that takes params object[]

Schema: annotatedCodeLocation is missing properties and tags

We need to publisht the SARIF viewer to the VS gallery

Allow command line to specify runInfo properties

For examples:

DbScanner http://localhost/accounting.db /runInfo:correlationId=123abc;architecture=x86

charOffset and byteOffset need to be int? (nullable)

It is easy for log consumers to determine that they need to need to consult charOffset or byteOffset members to determine result location, they only need to consult startColumn, which is 1-indexed (if its value is 0, then charOffset or byteOffset must be consulted).

Because charOffset and byteOffset are 0-indexed, however, we cannot distinguish which property is in play for offset == 0.

The practical impact is that object models (such as provided by the SARIF .NET SDK) must provide nullable values for these properties, that directly express whether the property was present in source JSON.

We should apply category from result properties bag in viewer, if available

See SarifSnapshot.cs for where this is populated

We are not serializing/deserializing hash algorithm enum correctly

FxCop converter should support project files

Sarif-Sdk has MEF dependency that needs to be added to nuspec

Provide formattedMessage validation (in release builds)

We have some debug handling for verifying that formattedMessages have appropriate # of arguments. This doesn't help callers who take a dependency on release builds.

Since the generation of a result is a reasonably low-frequency occurrence, we should put this validation into the release path. For now, it probably belongs in the RuleUtilities.BuildResult API (since we are still auto-generating the SDK, otherwise validation could exist in FormattedMessage itself).

Putting validation in the SDK API would allow someone to reauthor their own version if the validation was perceived as not helpful for some reason.

File/Open dialog should validate file existence with every directory change

When attempting to remap a file location, it is nice when the code automatically tries to locate the remapped file, rather than requiring a use to navigate to it completely. Making this improvement would entail getting a callback from File/Open on any directory change. We'd then need to examine all directory suffixes on the target file path to see if any of them resolve via File.Exists.

Viewer should provide simple application of code fixes

We should be able to take a code fix for a test file and to apply it using VS text buffer manipulating capabilities. Complexity here would involve remembering what edits have been made in order to permit subsequent application of other code fixes within the file.

We need a telemetry plan for a right-click/up or down vote issue features

We want users to have an easy way to send telemetry that says, minimally, this is a good result vs. tool noise.

We need documentation on how to use the Sarif.Sdk to produce Sarif files

We took a crack at integrating Sarif into two of our prototyping tools and were able to do so, but through trial and error. Some sample code in the README file for generating a Sarif output file using the SDK would be helpful!

Someone needs to emit/test code fixes.

SqlSkim could provide a code fix here for the 'don't use small variably typed column definitions' rule.

Viewer should automatically invoke converters, if available

We should add new options to File/Open for our tool converters. E.g., we have an entry for Fortify files that would invoke the converter for Fortify, convert to SARIF, and then populate via regular code paths.

When we have this functionality, the viewer is a kind of universal SA results viewer, might warrant some renaming of commands/change of metadata associated with package.

Fields which refer to a rule Id is sometimes named 'ruleId' and other times 'id'.

In the results node, a rule is identified by the “ruleId” field. In the ruleInfo node a rule is identified by the “id” field. It would be nice if these were named the same since they refer to the same value.

Can't find GeneratedKey.snk in a new enlistment

Repro:

1. Fork this repo and clone it.
2. From the root directory, run BuildAndTest.cmd.

Expected: It builds.

Actual:

CSC : error CS7027: Error signing output with public key from file 'GeneratedKey.snk' -- File not found. [e:\Repos\sarif-sdk\src\Sarif\Sarif.csproj]

Cause: The script creates GeneratedKey.snk in the root directory, but the project files don't refer to it at the correct path.

Fix: Modify the project files to refer to the key at the correct path. (The project file Sarif.Viewer.VisualStudio.csproj has it right. Sarif.csproj and Sarif.Driver.csproj have it wrong.)

Write command line tool to validate SARIF files against schema

IsSuppressedInSource needs to be a bool?

In order to express three conditions:

in source suppression information not available (e.g., when CODE_ANALYSIS is not defined)
in-source suppression info available and item is suppressed in source
in-source suppression info available and item is not suppressed in source

Unit test failures

There are several test failures. Console output attached:

sarif-sdk-test-errors.txt

Not clear how many distinct errors this represents. Will open separate issues as appropriate.

Schema: description on definitions.fix.fileChanges is wrong

Validate that all logging utilities can accept input with null context URI.

Some error messages (such as unhandled engine exception) may occur with no populated analysis target URI. We should explicit review these cases and ensure logging succeeds.

SARIF validator should emit SARIF

URI serialization should be done with a Json.NET converter

... so that customers don't have to discover the obscure helper method CreateUriForJsonSerialization.

ERR0998 should call out the id of the rule that failed

Since ERR0998 is raised for rule exceptions, the rule id of the offending rule should be explicitly stated in the ERR0998 result node. Currently, the rule name is listed in the format args, but the rule id does not exist.

Viewer should populate VS 'Category' column from properties bag

If we have a non-null property bag and it contains a 'Category' entry, we should populate the VS column with this information.

We need to populate IsSuppressedInSource in SARIF and test functionality in viewer

One way to accomplish this would be to enable FxCop ability to emit suppressed messages to log. Then populate SARIF from a converted FxCop log file and open in VS. If we have the automated FxCop conversion working in VS, this intermediate step could be skipped.

validation assemblies are delay-signed

We should package/download signed versions instead.

investigate improved hover pop-ups for all error list columns

Can we add the tool short-name to the tool column and provide tool full name, if present on pop-up, for example?