oasis-open / cti-stix-validator

OASIS TC Open Repository: Validator for STIX 2.0 JSON normative requirements and best practices

cti-stix-validator

NOTE: This is an OASIS TC Open Repository <https://www.oasis- open.org/resources/open-repositories/>__. See the Governance section for more information.

The STIX Validator checks that STIX JSON content conforms to the requirements specified in the STIX 2.1 specification. In addition to checking conformance with the JSON schemas <https://github.com/oasis- open/cti-stix2-json-schemas>, the validator checks conformance with requirements that cannot be specified in JSON schema, as well as with established "best practices". This validator is non-normative; in cases of conflict with the STIX 2.1 specification, the specification takes precedence.

The STIX 2.1 specification contains two types of requirements: mandatory "MUST" requirements, and recommended "SHOULD" best practice requirements. The validator checks documents against the "MUST" requirements using JSON schemas. Some of these mandatory requirements cannot be implemented in JSON Schema, however, so the validator uses Python functions to check them. The "SHOULD" requirements are all checked by Python functions, and options may be used to ignore some or all of these recommended "best practices."

The only exception to this is the mandatory requirement that an object's 'type' be one of those defined by a STIX Object in the specification. This rules out custom objects, so this check was made optional.

The validator also color-codes its output to make it easier to tell at a glance whether validation passed.

Installation

The easiest way to install the STIX validator is with pip:

$ pip install stix2-validator

Usage

As A Script

The validator comes with a bundled script which you can use to validate a JSON file containing STIX content:

$ stix2_validator <stix_file.json>

As A Library

You can also use this library to integrate STIX validation into your own tools. You can validate a JSON file:

from stix2validator import validate_file, print_results

results = validate_file("stix_file.json")
print_results(results)

You can also validate a JSON string using validate_string(), or a Python dictionary representing a STIX object using validate_instance(). For more information, see the full documentation here.

Governance

This GitHub public repository ( https://github.com/oasis-open/cti-stix-validator ) was proposed <https://lists.oasis- open.org/archives/cti/201609/msg00001.html> and approved [bis] by the OASIS Cyber Threat Intelligence (CTI) TC <https://www.oasis- open.org/committees/cti/> as an OASIS TC Open Repository to support development of open source resources related to Technical Committee work.

While this TC Open Repository remains associated with the sponsor TC, its development priorities, leadership, intellectual property terms, participation rules, and other matters of governance are separate and distinct <https://github.com/oasis-open/cti-stix- validator/blob/master/CONTRIBUTING.md#governance-distinct-from-oasis- tc-process> from the OASIS TC Process and related policies.

All contributions made to this TC Open Repository are subject to open source license terms expressed in the BSD-3-Clause License <https://www.oasis-open.org/sites/www.oasis-open.org/files/BSD-3- Clause.txt>. That license was selected as the declared "Applicable License" <https://www.oasis-open.org/resources/open- repositories/licenses> when the TC Open Repository was created.

As documented in "Public Participation Invited" <https://github.com/oasis-open/cti-stix- validator/blob/master/CONTRIBUTING.md#public-participation-invited>, contributions to this OASIS TC Open Repository are invited from all parties, whether affiliated with OASIS or not. Participants must have a GitHub account, but no fees or OASIS membership obligations are required. Participation is expected to be consistent with the OASIS TC Open Repository Guidelines and Procedures <https://www.oasis- open.org/policies-guidelines/open-repositories>, the open source LICENSE <https://github.com/oasis-open/cti-stix- validator/blob/master/LICENSE> designated for this particular repository, and the requirement for an Individual Contributor License Agreement <https://www.oasis-open.org/resources/open- repositories/cla/individual-cla> that governs intellectual property.

Maintainers

TC Open Repository Maintainers <https://www.oasis- open.org/resources/open-repositories/maintainers-guide>__ are responsible for oversight of this project's community development activities, including evaluation of GitHub pull requests <https://github.com/oasis-open/cti-stix- validator/blob/master/CONTRIBUTING.md#fork-and-pull-collaboration- model> and preserving <https://www.oasis-open.org/policies- guidelines/open-repositories#repositoryManagement> open source principles of openness and fairness. Maintainers are recognized and trusted experts who serve to implement community goals and consensus design preferences.

Initially, the associated TC members have designated one or more persons to serve as Maintainer(s); subsequently, participating community members may select additional or substitute Maintainers, per consensus agreements <https://www.oasis-open.org/resources/open- repositories/maintainers-guide#additionalMaintainers>.

Current Maintainers of this TC Open Repository

• Emily Ratliff; GitHub ID: https://github.com/ejratl; WWW: IBM
• Jason Keirstead; GitHub ID: https://github.com/JasonKeirstead; WWW: IBM
• Rich Piazza; GitHub ID: https://github.com/rpiazza; WWW: MITRE

About OASIS TC Open Repositories

• TC Open Repositories: Overview and Resources
• Frequently Asked Questions
• Open Source Licenses
• Contributor License Agreements (CLAs)
• Maintainers' Guidelines and Agreement

Feedback

Questions or comments about this TC Open Repository's activities should be composed as GitHub issues or comments. If use of an issue/comment is not possible or appropriate, questions may be directed by email to the Maintainer(s) listed above. Please send general questions about TC Open Repository participation to OASIS Staff at [email protected] and any specific CLA-related questions to [email protected].

cti-stix-validator's Issues

Check for invalid timestamps

Timestamps with invalid dates like November 31, or February 29 on non-leap years should raise an error.

Tests fail in tox, not with pytest outside of tox or on Travis-CI

When I try to run tox, there are four tests that fail in every environment:

$ tox -r -e py27
GLOB sdist-make: /Users/gback/code/cti-stix-validator/setup.py
py27 recreate: /Users/gback/code/cti-stix-validator/.tox/py27
py27 installdeps: pytest
py27 inst: /Users/gback/code/cti-stix-validator/.tox/dist/stix2-validator-0.0.1.zip
py27 installed: colorama==0.3.7,functools32==3.2.3.post2,jsonschema==2.5.1,py==1.4.32,pytest==3.0.5,requests==2.12.4,requests-cache==0.4.13,six==1.10.0,stix2-validator==0.0.1
py27 runtests: PYTHONHASHSEED='2876203125'
py27 runtests: commands[0] | py.test
========================================================= test session starts =========================================================
platform darwin -- Python 2.7.13, pytest-3.0.5, py-1.4.32, pluggy-0.4.0
rootdir: /Users/gback/code/cti-stix-validator, inifile: tox.ini
collected 134 items

lib/python2.7/site-packages/stix2validator/test/attack_pattern_tests.py ......
lib/python2.7/site-packages/stix2validator/test/bundle_tests.py ....
lib/python2.7/site-packages/stix2validator/test/custom_obj_tests.py .........
lib/python2.7/site-packages/stix2validator/test/identity_tests.py ....
lib/python2.7/site-packages/stix2validator/test/indicator_tests.py ................
lib/python2.7/site-packages/stix2validator/test/intrusion_set_tests.py ....
lib/python2.7/site-packages/stix2validator/test/malware_tests.py .....
lib/python2.7/site-packages/stix2validator/test/marking_definition_tests.py ......
lib/python2.7/site-packages/stix2validator/test/network_traffic_tests.py .F.F...
lib/python2.7/site-packages/stix2validator/test/observed_data_tests.py ...F..F..................................
lib/python2.7/site-packages/stix2validator/test/relationship_tests.py .........
lib/python2.7/site-packages/stix2validator/test/report_tests.py ..
lib/python2.7/site-packages/stix2validator/test/sighting_tests.py ....
lib/python2.7/site-packages/stix2validator/test/threat_actor_tests.py ......
lib/python2.7/site-packages/stix2validator/test/tool_tests.py .....
lib/python2.7/site-packages/stix2validator/test/vulnerability_tests.py ......

============================================================== FAILURES ===============================================================
__________________________________________ ObservedDataTestCases.test_network_traffic_ipfix ___________________________________________

self = <stix2validator.test.network_traffic_tests.ObservedDataTestCases testMethod=test_network_traffic_ipfix>

    def test_network_traffic_ipfix(self):
        net_traffic = copy.deepcopy(self.valid_net_traffic)
        net_traffic['objects']['1']['ipfix'] = {
            "minimumIpTotalLength": 32,
            "maximumIpTotalLength": 2556,
            "foo": "bar"
        }
>       self.assertFalseWithOptions(json.dumps(net_traffic))

lib/python2.7/site-packages/stix2validator/test/network_traffic_tests.py:95:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
lib/python2.7/site-packages/stix2validator/test/__init__.py:57: in assertFalseWithOptions
    self.assertEqual(results.is_valid, False)
E   AssertionError: True != False
________________________________________ ObservedDataTestCases.test_network_traffic_protocols _________________________________________

self = <stix2validator.test.network_traffic_tests.ObservedDataTestCases testMethod=test_network_traffic_protocols>

    def test_network_traffic_protocols(self):
        net_traffic = copy.deepcopy(self.valid_net_traffic)
        net_traffic['objects']['1']['protocols'].append('foobar')
>       self.assertFalseWithOptions(json.dumps(net_traffic))

lib/python2.7/site-packages/stix2validator/test/network_traffic_tests.py:82:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
lib/python2.7/site-packages/stix2validator/test/__init__.py:57: in assertFalseWithOptions
    self.assertEqual(results.is_valid, False)
E   AssertionError: True != False
_________________________________________ ObservedDataTestCases.test_directory_character_set __________________________________________

self = <stix2validator.test.observed_data_tests.ObservedDataTestCases testMethod=test_directory_character_set>

    def test_directory_character_set(self):
        observed_data = copy.deepcopy(self.valid_observed_data)
        observed_data['objects']['2'] = {
          "type": "directory",
          "path": "C:\\Windows\\System32",
          "path_enc": "blablabla"
        }
>       self.assertFalseWithOptions(json.dumps(observed_data))

lib/python2.7/site-packages/stix2validator/test/observed_data_tests.py:518:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
lib/python2.7/site-packages/stix2validator/test/__init__.py:57: in assertFalseWithOptions
    self.assertEqual(results.is_valid, False)
E   AssertionError: True != False
____________________________________________ ObservedDataTestCases.test_file_character_set ____________________________________________

self = <stix2validator.test.observed_data_tests.ObservedDataTestCases testMethod=test_file_character_set>

    def test_file_character_set(self):
        observed_data = copy.deepcopy(self.valid_observed_data)
        observed_data['objects']['0']['name_enc'] = "blablabla"
>       self.assertFalseWithOptions(json.dumps(observed_data))

lib/python2.7/site-packages/stix2validator/test/observed_data_tests.py:506:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
lib/python2.7/site-packages/stix2validator/test/__init__.py:57: in assertFalseWithOptions
    self.assertEqual(results.is_valid, False)
E   AssertionError: True != False
================================================ 4 failed, 130 passed in 9.13 seconds =================================================
ERROR: InvocationError: '/Users/gback/code/cti-stix-validator/.tox/py27/bin/py.test'
_______________________________________________________________ summary _______________________________________________________________
ERROR:   py27: commands failed

They all seem related to the code that gets data from the network and (optionally) caches it. I tried to ensure the cache is never used during testing, but that didn't seem to fix anything.

The tests all pass when I run pytest outside of tox, and this doesn't seem to affect Travis-CI builds. @clenk , can you take a look at this, and see if the same thing is happening for you or if it's just my machine?

Push latest version to PyPi

Push latest version of master with unpinned jsonschema to PYPI

Duplicate UUIDs in bundle pass validation

Attached is a file showing a bundle w/ duplicate UUID's, but the validator says that it passes validation fine. It would be nice if the tool warned about such errors, where if the modified is the same, but properties differ, that it throws an error.

It is expected that a bundle could have multiple different versions of the same UUID, but in that case, the modified must be different.

Attached is a sample file.
test.txt

why didn't check 'name' property in process? (OBSERVABLE_PROPERTIES)

cti-stix-validator/stix2validator/v21/enums.py

Line 852 in ed380c1

'process': [

The validator which version is '2.0' have checked name property in OBSERVABLE_PROPERTIES 'process'.
But the validator which version is '2.1' didn't check this. The 'name' property is missed in 'v21/enums.py' file.
Is it intended? or bug?

Convert README and CONTRIBUTING to Markdown

Malformed hash values are not caught by the validator

It appears that the grammar would need to be changed. Not sure it is worth it.

Better error message when observed-data.objects is a list instead of dictionary

If you give the validator a bundle with an Observed Data object that has a list for its objects property you get the following:

Fatal error occurred
Traceback (most recent call last):
  File "/home/lu/Projects/test/cti-stix-validator/stix2validator/scripts/stix2_validator.py", line 27, in main
    results = run_validation(options)
  File "/home/lu/Projects/test/cti-stix-validator/stix2validator/validator.py", line 348, in run_validation
    results = validate(options.files, options)
  File "/home/lu/Projects/test/cti-stix-validator/stix2validator/validator.py", line 421, in validate
    results = validate_parsed_json(obj_json, options)
  File "/home/lu/Projects/test/cti-stix-validator/stix2validator/validator.py", line 398, in validate_parsed_json
    results = validate_instance(obj_json, options)
  File "/home/lu/Projects/test/cti-stix-validator/stix2validator/validator.py", line 685, in validate_instance
    error_gens += _schema_validate(sdo, options)
  File "/home/lu/Projects/test/cti-stix-validator/stix2validator/validator.py", line 629, in _schema_validate
    for key, obj in iteritems(sdo['objects']):
  File "/home/lu/.virtualenvs/test-stix-validator/lib/python3.6/site-packages/six.py", line 587, in iteritems
    return iter(d.items(**kw))
AttributeError: 'list' object has no attribute 'items'

We should give a better error to point users to the real issue.

Support additional schemas (custom extensions)

We should add an option to allow custom additional schemas. The validator should then ensure the STIX input validates against both the official specification (via the OASIS schemas) and these additional schemas.

Consider changing strict types/properties to warnings

We should think about changing --strict-types and --strict-properties to issue warnings instead of errors. Then --strict could be used in addition if needed to change them to errors like they are currently.

error on -ov values

The pattern matcher presents an error when a value is used that is NOT in an -ov. An -ov is an open vocab and is allowed to have ANY value, and so the validator MUST NOT show it as an error, as it is valid and well formed to have non vocab value:
[!] indicator--38f66507-4e25-4cd1-8a43-52bd45e18c37: {214} labels contains a value not in the indicator-label-ov vocabulary.

Look into using appdirs for cache location

https://github.com/ActiveState/appdirs

the validator should be able to be called as a library method

The elevator "calls" the stix2 validator as sort of a library. The elevator should just use the validator via its API (command line args). Currently, the elevator is parsing the validator's args and setting some of them directly in ValidatorOptions - which should be hidden.

The main method should be rewritten, so you just pass in the command line arg string. Then the new main method of the validator is just a wrapper around that library method.

The validator logger has introduced problems for the elevator logger

Once the validator 1.0.1 was used with the elevator, each elevator message was output twice.

This temporary fix (oasis-open/cti-stix-elevator@b7ce45b) in the elevator solved the problem, but we should look into how loggers work together when stix utilities call each other.

warn instead of error for SHOULD

When providing a custom property, it errors about it (prints in red), instead of warning about it.
[!] course-of-action--d6e1dbd5-104f-4a09-b76c-3531d3398ecb: {103} Custom property 'obs_ref' should have a type that starts with 'x_' followed by a source unique identifier (like a domain name with dots replaced by dashes), a dash and then the name.

Validator requires version property for observed-data

For the attached file
observ.txt

It will pass even though the version property is present though it should not be (other objects do not require it).

If you do have the "required" version property present, it then warns about it being a custom property and should have a prefix of x_.

Support later versions of STIX 2

We need to figure out how to support later versions of STIX 2 in the validator. It should probably default to the latest published version of the spec, with an option to set the version to validate against. People may want to validate against multiple versions of the spec if they are supporting multiple versions at once. Each version of the validator could have a maximum supported STIX version, or we could do something with subpackages like oasis-open/cti-python-stix2#79. I'm leaning towards the latter.

Test for type of the object ref network-traffic is wrong

This is the error I got when testing the elevator.

observed-data--65492fbc-5d48-49bf-83b8-33c965e36066: 'opened_connection_refs' in observable object '0' must refer to an object of type 'ntwork-traffic'.

The output of the elevator:

        "4": {
                    "dst_ref": "3",
                    "type": "network-traffic"
                }

The generated file is in cti-stix-elevator/idioms-json/isuee-62.json.

Seeing error w/ indicator w/ a pattern

With the attached file, I see:
[X] Fatal Error: expected string or buffer

It appears that it is trying to parse/understand the pattern, but it fails.
test.txt

This is at 63a339b.

A previous version of the validator was known to work properly on a similar indicator w/ a pattern. I don't remember what version that was.

handle options --verbose and --silent correctly

Related to #50

The correct combination of these two should be as follows:

--verbose	--silent	desired behavior
absent (default is False)	absent (default is False)	all messages except those printed by info
absent (default is False)	present (True)	no messages printed
present (True)	absent (default is False)	all messages, including info are printed
present (True)	present (True)	error

Current behavior is:

--verbose	--silent	current behavior
absent (default is False)	absent (default is False)	all messages except those printed by info
absent (default is False)	present (ignored, so the default - False)	all messages except those printed by info
present (True)	absent (default is False)	all messages, including info are printed
present (True)	present (ignored, so the default - False)	all messages, including info are printed

Add option to silence all output written to stdout

Would be nice if all output to stdout could be silenced.

validate_instance appears to not be defined

With version 1.0.1 of the validator on Python 3.6:

>>> from stix2validator import validate_instance
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'validate_instance'

Per the usage documentation it seems like that should work.

cpe:2.3:o:microsoft:windows_xp:-:sp2:*:*:*:*:*:* Fails

I got the official CPE name for this version of windows from https://nvd.nist.gov/products/cpe/detail/70394 but it is failing the validator script.

It looks like it has a problem with the "-" between windows_xp and sp2 which is supposed to be the version number. It makes sense that "-" is a bad version number but if that is what is listed as the official cpe name it should pass right?

Don't call sys.exit when used as a library

Only call sys.exit when used as a CLI utility.

See #67 (comment).

Validate that indicator patterns use valid observable types and property names

Issues like oasis-open/cti-stix2-json-schemas#55 are caused because we don't validate the contents of Indicator pattern expressions, just the syntax. We should be able to combine the stix2patterns.inspector module with the data in enums.py to check whether object types and properties (including via referenced objects) are valid.

For custom types and properties, we can follow the same rules as the rest of the validator:

• allow them regardless if they follow the correct naming conventions, and
• raise an error if they don't, which can be ignored/silenced.

To scope this, I'll assume the following are out of scope for now:

• Checking if the values are the right type (since this isn't yet tracked in enums.py)
• Allowing users to specify names of properties of custom objects in or nested property names in custom properties. So basically, as soon as a (valid) custom object name or property name is encountered, checking that component of the pattern stops.

relationships & sighting objects

The following STIX 2 bundle currently passes validation. Is the "sighting-of" relationship between a Sighting and Malware objects valid? Seems the validator currently does not enforce the relationships between the SDOs. Will this be enforced in the future?

{
  "type": "bundle",
  "spec_version": "2.0",
  "id": "bundle--8f361d59-3d22-47c4-bab1-67560aecb34a",
  "objects": [
    {
      "type": "sighting",
      "sighting_of_ref": "malware--cad074cf-d120-456a-98b5-236d4e3aa025",
      "count": 0,
      "summary": false,
      "id": "sighting--1232725b-6211-4c91-bc71-c9159bd487b1",
    },
    {
      "type": "relationship",
      "target_ref": "malware--cad074cf-d120-456a-98b5-236d4e3aa025",
      "source_ref": "sighting--1232725b-6211-4c91-bc71-c9159bd487b1",
      "id": "relationship--5b6c5dc9-5d0b-4f0b-be23-626531b250f3",
      "relationship_type": "sighting-of",
      "created": "2018-07-11T11:17:48-04:00",
      "modified": "2018-07-11T11:17:48-04:00",
    },
    {
      "type": "malware",
      "name": "malware--cad074cf",
      "labels": ["ransomware"],
      "description": "",
      "id": "malware--cad074cf-d120-456a-98b5-236d4e3aa025"
    }
  ]
}

"STIX JSON: Valid" but "[X] Fatal Error: Input must be an object with a 'type' property."

We are wondering about the following error:

STIX JSON: Valid
    [X] Fatal Error: Input must be an object with a 'type' property.

Here is the JSON file in STIX 2:

{"package": {
    "id": "bundle--59ca11e2-1810-4f97-b3f1-4d13950d210f",
    "objects": [
        {
            "created": "2017-10-06T09:38:18.744698Z",
            "id": "identity--57ada0c5-be9c-4e58-960c-74db950d210f",
            "identity_class": "organization",
            "modified": "2017-10-06T09:38:18.744698Z",
            "name": "CERT-BPOL",
            "type": "identity"
        },
        {
            "created": "2017-10-06T09:38:18.748813Z",
            "created_by_ref": "identity--57ada0c5-be9c-4e58-960c-74db950d210f",
            "id": "report--59ca11e2-1810-4f97-b3f1-4d13950d210f",
            "labels": [
                "ecsirt:malicious-code=\"ransomware\"",
                "tlp:white"
            ],
            "modified": "2017-10-06T09:38:18.748813Z",
            "name": "RedBoot Ransomware/Wiper",
            "object_refs": [
                "observed-data--59ca138b-8f08-49b2-a9fb-44be950d210f",
                "observed-data--59ca1356-e4cc-440a-ab58-47ce950d210f",
                "observed-data--59ca1326-8d68-49a2-ace5-ff74950d210f",
                "observed-data--59ca1307-e754-4d33-bd50-4c5a950d210f",
                "observed-data--59ca12f3-3a74-45ab-9071-42b0950d210f",
                "observed-data--59ca12de-dca4-4998-b629-4cab950d210f",
                "observed-data--59ca12c6-4ed8-4642-b190-4937950d210f",
                "observed-data--59ca1221-41b8-4d3e-ab8c-4535950d210f",
                "observed-data--59ca12a3-4e70-413f-bb74-4af9950d210f"
            ],
            "published": "2017-09-26T08:51:08Z",
            "type": "report"
        },
        {
            "created": "2017-10-06T09:38:18.744823Z",
            "created_by_ref": "identity--57ada0c5-be9c-4e58-960c-74db950d210f",
            "first_observed": "2017-09-26T08:44:59Z",
            "id": "observed-data--59ca138b-8f08-49b2-a9fb-44be950d210f",
            "last_observed": "2017-09-26T08:44:59Z",
            "modified": "2017-10-06T09:38:18.744823Z",
            "number_observed": 1,
            "objects": {
                "0": {
                    "name": "https://www.bleepingcomputer.com/news/security/ransomware-or-wiper-redboot-encrypts-files-but-also-modifies-partition-table/",
                    "type": "file"
                }
            },
            "type": "observed-data"
        },
        {
            "created": "2017-10-06T09:38:18.745665Z",
            "created_by_ref": "identity--57ada0c5-be9c-4e58-960c-74db950d210f",
            "first_observed": "2017-09-26T08:44:06Z",
            "id": "observed-data--59ca1356-e4cc-440a-ab58-47ce950d210f",
            "last_observed": "2017-09-26T08:44:06Z",
            "modified": "2017-10-06T09:38:18.745665Z",
            "number_observed": 1,
            "objects": {
                "0": {
                    "name": "This computer and all of it's files have been locked! Send an email to [email protected] containing your ID key for instructions on how to unlock them. Your ID key is 79E7794CEEBBDF34EE595914D968AAAD2E355904",
                    "type": "file"
                }
            },
            "type": "observed-data"
        },
        {
            "created": "2017-10-06T09:38:18.74609Z",
            "created_by_ref": "identity--57ada0c5-be9c-4e58-960c-74db950d210f",
            "first_observed": "2017-09-26T08:43:18Z",
            "id": "observed-data--59ca1326-8d68-49a2-ace5-ff74950d210f",
            "last_observed": "2017-09-26T08:43:18Z",
            "modified": "2017-10-06T09:38:18.74609Z",
            "number_observed": 1,
            "objects": {
                "0": {
                    "hashes": {
                        "SHA-256": "2a5305369edb9c2d7354b2f210e91129e4b8c546b0adf883951ea7bf7ee0f2b2"
                    },
                    "type": "file"
                }
            },
            "type": "observed-data"
        },
        {
            "created": "2017-10-06T09:38:18.746724Z",
            "created_by_ref": "identity--57ada0c5-be9c-4e58-960c-74db950d210f",
            "first_observed": "2017-09-26T08:42:47Z",
            "id": "observed-data--59ca1307-e754-4d33-bd50-4c5a950d210f",
            "last_observed": "2017-09-26T08:42:47Z",
            "modified": "2017-10-06T09:38:18.746724Z",
            "number_observed": 1,
            "objects": {
                "0": {
                    "hashes": {
                        "SHA-256": "e61a8382f7293e40cb993ddcbcaa53a4e5f07a3d6b6a1bfe5377a1a74a8dcac6"
                    },
                    "type": "file"
                }
            },
            "type": "observed-data"
        },
        {
            "created": "2017-10-06T09:38:18.747082Z",
            "created_by_ref": "identity--57ada0c5-be9c-4e58-960c-74db950d210f",
            "first_observed": "2017-09-26T08:42:27Z",
            "id": "observed-data--59ca12f3-3a74-45ab-9071-42b0950d210f",
            "last_observed": "2017-09-26T08:42:27Z",
            "modified": "2017-10-06T09:38:18.747082Z",
            "number_observed": 1,
            "objects": {
                "0": {
                    "hashes": {
                        "SHA-256": "f2bc5886b0f189976a367a69da8745bf66842f9bba89f8d208790db3dad0c7d2"
                    },
                    "type": "file"
                }
            },
            "type": "observed-data"
        },
        {
            "created": "2017-10-06T09:38:18.747426Z",
            "created_by_ref": "identity--57ada0c5-be9c-4e58-960c-74db950d210f",
            "first_observed": "2017-09-26T08:42:06Z",
            "id": "observed-data--59ca12de-dca4-4998-b629-4cab950d210f",
            "last_observed": "2017-09-26T08:42:06Z",
            "modified": "2017-10-06T09:38:18.747426Z",
            "number_observed": 1,
            "objects": {
                "0": {
                    "hashes": {
                        "SHA-256": "1001a8c7f33185217e6e1bdbb8dba9780d475da944684fb4bf1fc04809525887"
                    },
                    "type": "file"
                }
            },
            "type": "observed-data"
        },
        {
            "created": "2017-10-06T09:38:18.747762Z",
            "created_by_ref": "identity--57ada0c5-be9c-4e58-960c-74db950d210f",
            "first_observed": "2017-09-26T08:41:42Z",
            "id": "observed-data--59ca12c6-4ed8-4642-b190-4937950d210f",
            "last_observed": "2017-09-26T08:41:42Z",
            "modified": "2017-10-06T09:38:18.747762Z",
            "number_observed": 1,
            "objects": {
                "0": {
                    "hashes": {
                        "SHA-256": "2ae74130ac809fb54f12e72f589069ad7b5e1f56e68cee00db8867b02112c4b9"
                    },
                    "type": "file"
                }
            },
            "type": "observed-data"
        },
        {
            "created": "2017-10-06T09:38:18.748104Z",
            "created_by_ref": "identity--57ada0c5-be9c-4e58-960c-74db950d210f",
            "first_observed": "2017-09-26T08:38:57Z",
            "id": "observed-data--59ca1221-41b8-4d3e-ab8c-4535950d210f",
            "last_observed": "2017-09-26T08:38:57Z",
            "modified": "2017-10-06T09:38:18.748104Z",
            "number_observed": 1,
            "objects": {
                "0": {
                    "type": "email-address",
                    "value": "[email protected]"
                }
            },
            "type": "observed-data"
        },
        {
            "created": "2017-10-06T09:38:18.748456Z",
            "created_by_ref": "identity--57ada0c5-be9c-4e58-960c-74db950d210f",
            "first_observed": "2017-09-26T08:41:07Z",
            "id": "observed-data--59ca12a3-4e70-413f-bb74-4af9950d210f",
            "last_observed": "2017-09-26T08:41:07Z",
            "modified": "2017-10-06T09:38:18.748456Z",
            "number_observed": 1,
            "objects": {
                "0": {
                    "hashes": {
                        "SHA-256": "1001a8c7f33185217e6e1bdbb8dba9780d475da944684fb4bf1fc04809525887"
                    },
                    "type": "file"
                }
            },
            "type": "observed-data"
        }
    ],
    "spec_version": "2.0",
    "type": "bundle"
}}

Is it something intended? or an issue from the validator itself? or our STIX 2 file?

Validation should fail when `*_bin` values aren't base64 encoded

When the cti-pattern-matcher library tries to match a pattern that matches on a value such as payload_bin on observed data where payload_bin contains unencoded text, the library raises an exception because it can't decode it as base64. However, that same observed data doesn't show up as a validation error when run through the validator. So it seems like this is something that should be checked by the validator.

print_results: 'ObjectValidationResults' object is not iterable

from stix2validator import validate_string, print_results    
results = validate_string(sample.stix2)      
print_results(results)     

Produces:

File "lib/python3.6/site-packages/stix2validator/output.py", line 152, in print_results
for file_result in sorted(results, key=operator.attrgetter("filepath")):
TypeError: 'ObjectValidationResults' object is not iterable

Using:

stix2-validator==1.0.1
Python 3.6.3

Implement output with logging

Use logging instead of print.

stix2patterns.pattern module not found in latest tag version

Importing stix2patterns.pattern throws exception as the location of pattern is moved in the latest tag version of stix2-patterns==1.2.0.
Here's the stacktrace of the error:

Traceback:
<file path>:5: in <module>
    from stix2validator import validate_instance, print_results
virtualenv/lib/python3.6/site-packages/stix2validator/__init__.py:9: in <module>
    from .validator import (run_validation, validate, validate_file,
virtualenv/lib/python3.6/site-packages/stix2validator/validator.py:15: in <module>
    from . import musts, output, shoulds
virtualenv/lib/python3.6/site-packages/stix2validator/musts.py:8: in <module>
    from stix2patterns.pattern import Pattern
E   ModuleNotFoundError: No module named 'stix2patterns.pattern'

@clenk can you please verify? i can see Pattern(object) is copied into couple of directories v20 and v21 in stix2patterns

Enable custom schemas to be used without requiring empty schemas

As noted in #47, validating certain objects using additional schemas requires adding empty dummy files. This is silly and we should fix this.

The option --strict-types also causes custom properties to be flagged

it would be nice to have another option: --strict-properties, so you can control if custom properties are flagged with warning messages. A custom object type is a much bigger deal that a custom property.

Enforce referenced objects in relationship objects

This bundle also passed the validator, and I do not think it should:

{
  "type": "bundle",
  "id": "bundle--44af6c39-c09b-49c5-9de2-394224b04982",
  "spec_version": "2.0",
  "objects": [
    {
      "type": "indicator",
      "id": "indicator--a932fcc6-e032-176c-126f-cb970a5a1ade",
      "created": "2014-02-20T09:16:08.989Z",
      "modified": "2014-02-20T09:16:08.989Z",
      "name": "File hash for Poison Ivy variant",
      "description": "This file hash indicates that a sample of Poison Ivy is present.",
      "labels": [
        "malicious-activity"
      ],
      "pattern": "[file:hashes.'SHA-256' = 'ef537f25c895bfa782526529a9b63d97aa631564d5d789c2b765448c8635fb6c']",
      "valid_from": "2014-02-20T09:00:00.000000Z"
    },
    {
      "type": "malware",
      "id": "malware--fdd60b30-b67c-11e3-b0b9-f01faf20d111",
      "created": "2014-02-20T09:16:08.989Z",
      "modified": "2014-02-20T09:16:08.989Z",
      "name": "Poison Ivy",
      "labels": [
        "remote-access-trojan"
      ]
    },
    {
      "type": "relationship",
      "id": "relationship--f191e70e-1736-47c3-b0f9-fdfe01387eb1",
      "created": "2014-02-20T09:16:08.989Z",
      "modified": "2014-02-20T09:16:08.989Z",
      "relationship_type": "indicates",
      "source_ref": "indicator--b932fcc6-e032-176c-126f-cb970a5a1ade",
      "target_ref": "malware--fdd60b30-b67c-11e3-b0b9-f01faf20d111"
    }
  ]

Note that I modified the uuid of the indicator object in source_ref of the SRO. I think we also want to make sure all of the SDO ids in every SROs are valid.

Test Individual Validation Methods

From @gtback:

It would be great to test individual validation methods, in addition to (or possibly just in place of) doing a full validation. This came up when looking at the code that validates strict relationships. I'd like to avoid having tests that "succeed" but are passing a different validation check than what we expected. The checks are already pretty isolated in that they can mostly be called directly with instance data (and don't need ValidationOptions or the like).

Validator must raise error if identifier doesn't follow the form "object-type--UUIDv4"

The following is a custom object which identifier doesn't follow the form "object-type--UUIDv4".

{
  "created_by_ref": "identity--55f6ea5e-2c60-40e5-964f-47a8950d210f",
  "created": "2019-08-19T13:28:33.874Z",
  "x_misp_timestamp": "2018-04-04 22:00:00",
  "x_misp_value": "Licenser.pdb",
  "labels": [
    "misp:type=comment",
    "misp:category=Other",
    "misp:to_ids=False"
  ],
  "modified": "2019-08-19T13:28:33.874Z",
  "type": "x-misp-object-comment",
  "x_misp_category": "Other",
  "id": "x-misp-object--5a6f3acb-08d4-4861-ae24-43aa950d210f"
}

Validator must raise error but it didn't.

stix validator does not handle list of stix objects in a file

If you attempt to validate a file which contains a list of stix objects (for testing w/ the stix pattern matcher code), you cannot, and will get the error:
[X] Fatal Error: Input must be an object with a 'type' property.

Please support files w/ a list of stix objects, validating each object in the list.

Thanks.

Remove 30 character check

In the final 2.0 spec, the hashes type and the dictionary type in cyber observables are the only places that says the keys SHOULD be no longer than 30 characters. We need to remove this check from everywhere else (e.g. custom property names).

Validate has unhandled exception with bad patterns

Rather than returning the appropriate error message, there is an unhandled exception.

Using Python3.

Tested with "validate-pattern" and sending in "file:size = 1280"

This pattern is failing to be validated

This STIX pattern fails validation.

[emailaddr:value MATCHES '.+\@IBM\.com$' OR file:name MATCHES '^Final Report.+\.exe$']

Error message is

FAIL: Error found at line 1:25. mismatched input ''' expecting StringLiteral

Running the validator from the elevator with different options causes an exception

From the 2.1-wd05 branch of the elevator, run both of the followiing:

cli.py /Users/rpiazza/git/stix/cti-stix-elevator/idioms-xml/issue62.xml -v 2.1 --validator-args "--version 2.1"
cli.py /Users/rpiazza/git/stix/cti-stix-elevator/idioms-xml/issue62.xml -v 2.1

The first will run and generate STIX 2.1. The second one causes this exception:

Traceback (most recent call last):
  File "/Users/rpiazza/py-envs/python3.7/lib/python3.7/site-packages/stix2_patterns-1.1.0-py3.7.egg/stix2patterns/grammars/STIXPatternParser.py", line 1226, in propTest
    la_ = self._interp.adaptivePredict(self._input,15,self._ctx)
  File "/Users/rpiazza/py-envs/python3.7/lib/python3.7/site-packages/antlr4_python3_runtime-4.7.2-py3.7.egg/antlr4/atn/ParserATNSimulator.py", line 342, in adaptivePredict
    alt = self.execATN(dfa, s0, input, index, outerContext)
  File "/Users/rpiazza/py-envs/python3.7/lib/python3.7/site-packages/antlr4_python3_runtime-4.7.2-py3.7.egg/antlr4/atn/ParserATNSimulator.py", line 414, in execATN
    raise e
antlr4.error.Errors.NoViableAltException: None

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/rpiazza/git/stix/cti-stix-elevator/stix2elevator/cli.py", line 200, in <module>
    main()
  File "/Users/rpiazza/git/stix/cti-stix-elevator/stix2elevator/cli.py", line 192, in main
    result = elevate_file(elevator_args.file_)
  File "/Users/rpiazza/git/stix/cti-stix-elevator/stix2elevator/__init__.py", line 82, in elevate_file
    validation_results = validate_stix2_string(json_string, validator_options, fn)
  File "/Users/rpiazza/git/stix/cti-stix-elevator/stix2elevator/__init__.py", line 36, in validate_stix2_string
    results = validate_string(json_string, validator_options)
  File "/Users/rpiazza/git/stix/cti-stix-validator/stix2validator/validator.py", line 486, in validate_string
    return validate(stream, options)
  File "/Users/rpiazza/git/stix/cti-stix-validator/stix2validator/validator.py", line 421, in validate
    results = validate_parsed_json(obj_json, options)
  File "/Users/rpiazza/git/stix/cti-stix-validator/stix2validator/validator.py", line 398, in validate_parsed_json
    results = validate_instance(obj_json, options)
  File "/Users/rpiazza/git/stix/cti-stix-validator/stix2validator/validator.py", line 755, in validate_instance
    warnings = [pretty_error(x, options.verbose) for x in warnings]
  File "/Users/rpiazza/git/stix/cti-stix-validator/stix2validator/validator.py", line 755, in <listcomp>
    warnings = [pretty_error(x, options.verbose) for x in warnings]
  File "/Users/rpiazza/git/stix/cti-stix-validator/stix2validator/validator.py", line 62, in _iter_errors_custom
    for err in _iter_errors_custom(obj, checks, options):
  File "/Users/rpiazza/git/stix/cti-stix-validator/stix2validator/validator.py", line 52, in _iter_errors_custom
    for x in result:
  File "/Users/rpiazza/git/stix/cti-stix-validator/stix2validator/v21/shoulds.py", line 1069, in types_strict
    p = Pattern(pattern)
  File "/Users/rpiazza/py-envs/python3.7/lib/python3.7/site-packages/stix2_patterns-1.1.0-py3.7.egg/stix2patterns/pattern.py", line 34, in __init__
    self.__parse_tree = self.__do_parse(pattern_str)
  File "/Users/rpiazza/py-envs/python3.7/lib/python3.7/site-packages/stix2_patterns-1.1.0-py3.7.egg/stix2patterns/pattern.py", line 117, in __do_parse
    real_exc)
  File "<string>", line 3, in raise_from
stix2patterns.pattern.ParseException: 1:897: no viable alternative at input 'unconverted_term:WinExecutableFileObj.exportsAND'

Fix UUIDv4 regex

The regex used in the validator and in python-stix2 should be consistent but is not currently. See https://github.com/oasis-open/cti-python-stix2/blob/master/stix2/properties.py#L21.

We'll also need to update apt1.json, which includes indicator--b3b6b540-d838-11e2-853b-005056c00008 and which python-stix2 cannot parse because it should be 41e2 instead of 11e2.

Disable some warnings by default

Warnings not tied to SHOULDS in the spec should be disabled by default, but still available if desired. For example, 202 which warns if content contains a relationship not defined in the spec, and 210 which warns if properties contain values not in an open vocab.

Add Documentation to ReadTheDocs

1.1.1 on PyPI is missing schemas directory

With stix2-validator==1.1.1
Every attempt at validation fails with result.errors containing following SchemaError: Cannot locate a schema for the object's type, nor the base schema (core.json).

work fine with stix2-validator==1.1.0

when spec_version is 2.0 on bundle, do not warn

On a stix bundle that has a spec_version of 2.0, the latest version of the validator returns a warning:
[!] Warning: bundle--59afb48d-0f9c-434d-be6a-69515424b0c3: {101} Custom property 'spec_version' should have a type that starts with 'x_' followed by a source unique identifier (like a domain name with dots replaced by hyphen), a hyphen and then the name.

It should NOT return a warning, as that is properly formatted STIX 2.0.

This was found when checking against https://github.com/pan-unit42/playbook_viewer/blob/master/playbook_json/darkhydrus.json

The pattern validator should not be invoked for non-STIX patterns

If the pattern type is snort, the STIX validator will still try to see if it is valid for the STIX Pattern language - which of course it isn't. We should not output a warning in these cases.

Update to STIX 2.1 WD06

List of changes to be accounted for:

• SCO ids:

  • If the contributing properties are all optional, and none are present on the SCO, then a UUIDv4 MUST be used. (Some example SCOs where this is possible: Email Message, Process, X509 Certificate)
  • Producers not following these rules MUST NOT use a namespace of 00abedb4-aa42-466c9c01-fed23315a9b7 and SHOULD use UUIDv4 in cases where the id would not be unique

• indicator.pattern_type: new vocab: pattern-type-ov

• indicator.pattern_version: if stix patterning lang used, default is spec_version of the object

• empty dictionaries are now prohibited just like empty lists (can be handled by schemas)

• socket-ext.options now has some requirements around its keys/values (can be handled by schemas)

• x509-certificate objects MUST have at least 1 object specific property (non-common, non-custom) other than 'type' (can be handled by schemas)

• hashes properties: The value MUST be a string in the appropriate format defined by the hash type indicated in the dictionary key.

• TLSH added to hash-algorithm-ov

pattern_lang has wrong error

The property pattern_lang has been removed from the Indicator, but it still says:
{217} pattern_lang contains a value not in the pattern-lang-ov vocabulary.

External References whose source_name includes 'capec' are invalid

The following should be valid STIX. Notice that the source_name includes the string 'capec', but that should be legal

       {
            "source_name": "reference_from_capec",
            "description": "The Official Social Engineering Portal, Social-Engineer.org, Tick Tock Computers, LLC",
            "url": "http://www.social-engineer.org",
            "external_id": "REF-348"
        }

Here is the error message:

[X] attack-pattern--0a19a0a2-1b7b-4dde-99b9-624edc549928: external_references[0]: If the external reference is a CVE, 'source_name' must be 'cve' and 'external_id' must be in the CVE format (CVE-YYYY-NNNN+). If the external reference is a CAPEC, 'source_name' must be 'capec' and 'external_id' must be in the CAPEC format (CAPEC-N+). If the external reference is neither, it must contain the 'source_name' property and at least one of the 'external_id', 'url', or 'description' properties.

Changing the source_name to reference_from_CAPEC avoids this issue.