github / go-spdx

Golang implementation of a checker for determining if an SPDX ID satisfies an SPDX Expression.

go-spdx

Golang implementation of a checker for determining if a set of SPDX IDs satisfies an SPDX Expression.

Installation

There are several ways to include a go package. To download and install, you can use go get. The command for that is:

go get github.com/github/go-spdx@latest

Packages

• spdxexp - Expression package validates licenses and determines if a license expression is satisfied by a list of licenses. Validity of a license is determined by the SPDX license list.

Public API

NOTE: The public API is initially limited to the Satisfies and ValidateLicenses functions. If there is interest in the output of the parser or license checking being public, please submit an issue for consideration.

Function: Satisfies

Satisfies(testExpression string, allowedList []string, options *Options)

Parameter: testExpression

testExpression is an SPDX expression describing the licensing terms of source code or a binary file.

Example expressions that can be used for testExpression:

"MIT"
"MIT AND Apache-2.0"
"MIT OR Apache-2.0"
"MIT AND (Apache-1.0 OR Apache-2.0)"
"Apache-1.0+"
"DocumentRef-spdx-tool-1.2:LicenseRef-MIT-Style-2"
"GPL-2.0 WITH Bison-exception-2.2"

See satisfies_test.go for more example expressions.

Parameter: allowedList

allowedList is an array of single licenses describing what licenses can be used to satisfy the testExpression.

Example allowedList:

[]string{"MIT"}
[]string{"MIT", "Apache-2.0"}
[]string{"MIT", "Apache-2.0", "ISC", "GPL-2.0"}
[]string{"MIT", "Apache-1.0+"}
[]string{"GPL-2.0-or-later"}

N.B. If at least one of expressions from allowedList is not a valid SPDX expression, the call to Satisfies will produce an error. Use ValidateLicenses function to first check if all of the expressions from allowedList are valid.

Examples: Satisfies returns true

Go Playground for Satisfies

Satisfies("MIT", []string{"MIT"})
Satisfies("MIT", []string{"MIT", "Apache-2.0"})
Satisfies("Apache-2.0", []string{"Apache-1.0+"})
Satisfies("MIT OR Apache-2.0", []string{"Apache-2.0"})
Satisfies("MIT OR Apache-2.0", []string{"MIT", "Apache-2.0"})
Satisfies("MIT AND Apache-2.0", []string{"MIT", "Apache-2.0"})
Satisfies("MIT AND Apache-2.0", []string{"MIT", "Apache-2.0", "GPL-2.0"})

Examples: Satisfies returns false

Satisfies("MIT", []string{"Apache-2.0"})
Satisfies("Apache-1.0", []string{"Apache-2.0+"})
Satisfies("MIT AND Apache-2.0", []string{"MIT"})

ValidateLicenses

func ValidateLicenses(licenses []string) (bool, []string)

Function ValidateLicenses is used to determine if any of the provided license expressions is invalid.

parameter: licenses

Licenses is a slice of strings which must be validated as SPDX expressions.

returns

Function ValidateLicenses has 2 return values. First is bool which equals true if all of the provided licenses provided are valid, and false otherwise.

The second parameter is a slice of all invalid licenses which were provided.

Examples: ValidateLicenses returns no invalid licenses

valid, invalidLicenses := ValidateLicenses([]string{"Apache-2.0"})
assert.True(valid)
assert.Empty(invalidLicenses)

Examples: ValidateLicenses returns invalid licenses

valid, invalidLicenses := ValidateLicenses([]string{"NON-EXISTENT-LICENSE", "MIT"})
assert.False(valid)
assert.Contains(invalidLicenses, "NON-EXISTENT-LICENSE")
assert.NotContains(invalidLicenses, "MIT")

Examples: ValidateLicenses works with SPDX expressions

valid, invalidLicenses := ValidateLicenses([]string{"MIT AND APACHE-2.0"})
assert.True(valid)
assert.NotContains(invalidLicenses, "MIT AND APACHE-2.0")

ExtractLicenses

func ExtractLicenses(expression string) ([]string, error)

Function ExtractLicenses is used to extract licenses from the given expression without duplicates.

parameter: expression

expression is an SPDX expression string.

returns

Function ExtractLicenses has 2 return values. First is []string which contains all of the SPDX licenses without duplicates.

The second return value is an error which is not nil if the given expression is not a valid SPDX expression.

Example

licenses, err := ExtractLicenses("(MIT AND APACHE-2.0) OR (APACHE-2.0)")
assert.Equal(licenses, []string{"MIT", "Apache-2.0"})

Background

This package was developed to support testing whether a repository's license requirements are met by an allowed-list of licenses.

Dependencies are defined in go.mod.

Contributions and requests are welcome. Refer to the Contributing section for more information including how to set up a test environment and install dependencies.

License

This project is licensed under the terms of the MIT open source license. Please refer to MIT for the full terms.

Maintainers

• @elrayle
• @ajhenry

Support

You can expect the following support:

• bug fixes
• review of feature request issues
• review of questions in discussions

Contributing

Contributions in the form of bug identification Issues, bug fix PRs, and feature requests are welcome. See CONTRIBUTING.md for more information on how to get involved and set up a testing environment.

NOTE: The list of valid licenses is maintained manually. If you notice a missing license, an excellent way to contribute to the long term viability of this package is to open an Issue or PR addressing the missing license.

Acknowledgement

The process for parsing and evaluating expressions is a translation from JavaScript to Go based heavily on the JavaScript implementation defined across several repositories.

• spdx-satisfies.js
• spdx-expression-parse.js
• spdx-ranges
• spdx-compare
• spdx-license-ids
• spdx-exceptions

go-spdx's Issues

Use of spdxexp as a library

👋 Thanks for the awesome implementation and Satisfies functionality.

Over in syft we're trying to upgrade how we handle the complex license statements from SPDX.

I noticed that parse.go and scan.go has a lot of really good private functionality for reading these complex SPDX expressions.

Would you all accept a PR that exposed a few methods of this package so we could incorporate the parsing logic into how we identify and parse license in our SBOM output?

DOCUMENTATION: Review public API documentation

Description

As this package moves to an open source release, it is important that documentation reads well. The doc generator should be run and resulting documentation reviewed for clarity.

Including examples

While looking through documentation docs, I noticed some packages (e.g. json) have examples that are hidden until clicked (e.g. json Indent function). It would be great to use this for the spdxexp package, especially for the Satisfies method which has lots of examples.

Looking at the code for json's Indent function, it looks like the examples are defined in example_test.go. I don't know if this is only available to standard go packages or if the general documentation generator will process an example file in any package.

left is mutated in mergeLeftRight

In satisfies.go, the following code likely will mutate left...

func mergeLeftRight(left, right [][]*Node) [][]*Node {
	for _, r := range right {
		for j, l := range left {
			left[j] = append(l, r...)

The process doesn't use left again, but it is probably cleaner to address this. Since left is created by the parser, it is conceivable that in the future the results of the parser may be re-used somewhere. This would be a hard bug to track down.

CI: Update golang lint ci to remove deprecated rules

There are currently some deprecated rules in the .golangci.yaml config that need to be removed

add unit tests for exceptions.go

I would like to add unit tests for cmd/exceptions.go

DOCUMENTATION: add package level documentation and files level documentation for all files

Add documentation explaining major concepts and purpose for each file.

Add package level documentation that describes interconnections between major concepts in different files and how to use public API.

_NOTE: Use Go for identifying the go language. There is at least one use of GO in the code documentation.`

Improve error reporting for invalid licenses

Description

If a license is invalid, the scan process is returning an error message showing only the first letter of the invalid license.

Expected

For a license named "BAD-LICENSE", the error message should be something like... "unknown license 'BAD-LICENSE' at offset 0".

Actual

For a license named "BAD-LICENSE", the error message is... "unexpected 'B' at offset 0".

REFACTOR: do not convert operators to lower case

The original javascript converted incoming operators (e.g. AND) to lower case (e.g. and) during the tokenization process. The initial translation stayed as close to the original javascript as possible and reasonable with assumptions that a later process would require the output to be in the format created in the javascript.

I do not see the need for this case conversion in later processing. Recommending a refactor to not convert operators case.

BUG: operator error - parenthetical without match fails silently

Description

An open parenthesis in an expression should always have a matching close parenthesis. In the initial implementation in PR #1 , it was noted that a mismatch was silently ignored and swallowed at least the next following token. This leads to an invalid set of tokens instead of an error message.

Expected Behavior

When a matching close parenthesis is not in an expression, an error message should be returned from the scan function.

Proposed New Test

Add the following test (and perhaps other variations) to test for unmatched parentheses to the TestScan function in scan_test.go.

		{"operator error - malformed parenthetical", "(MIT",
		    []token{}, errors.New("open parenthesis does not have a matching close parenthesis")},

Results with current code

expected: *errors.errorString(&errors.errorString{s:"open parenthesis does not have a matching close parenthesis"})
actual  : <nil>(<nil>)

expected: <nil>(<nil>)
actual  : []spdxexp.token{spdxexp.token{role:0x0, value:"("}, spdxexp.token{role:0x3, value:"MIT"}}