turn external_references into Reports about cti

[jburns12]

Hi @VladimirAlexiev -

From @johnwunder - we put the references on the actual STIX object that they occur on, which isn’t always the object for the page that you’re viewing. For example, all of the “Examples” are captured as relationship objects linking the technique to a threat group or software instance that uses that technique. So in this case, there are a lot of references that are just on examples…for example, the reference for “APT37 has used 8080 for C2” occurs on the relationship object relationship--d0ed6995-15c9-4221-9783-4b61281d1d02 linking T1043 to APT37. Hopefully that makes sense.

So in summary, the external_references are all in the STIX content, but they're purposefully not all put in the individual attack-pattern objects. Please let us know if you have any further concerns!

[VladimirAlexiev]

That makes a lot of sense. I grepped these 32 refs https://gist.github.com/VladimirAlexiev/8a67976633db2f6b7a96fd82b12c3cd8 and they all appear in a lot of objects: between 4 and 37.

This is the most used reference;

Dell SecureWorks Counter Threat Unit Threat Intelligence. (2015, August 5)
intrusion-set/intrusion-set--fb366179-766c-4a4a-afa1-52bff1fd601c.json
malware/malware--56f46b17-8cfa-46c0-b501-dd52fef394e2.json
malware/malware--5a3a31fe-5a8f-48e1-bff0-a753e5b1be70.json
malware/malware--64fa0de0-6240-41f4-8638-f4ca7ed528fd.json
malware/malware--a60657fa-e2e7-4f8f-8128-a882534ae8c5.json
malware/malware--e066bf86-9cfb-407a-9d25-26fd5d91e360.json
relationship/relationship--0b36c1d0-d016-4c12-bf61-6dc14b29c7e0.json
relationship/relationship--0dee5507-6e61-4244-86a8-c7e8a34469da.json
relationship/relationship--0e12d7d1-5c46-4314-97fb-263853eed6af.json
relationship/relationship--1088fc27-2de5-4b73-83fd-6741ab3ff4d6.json
relationship/relationship--13d8aec7-3e49-41f8-b57c-475cdc0d9632.json
relationship/relationship--291b7fbf-5b5f-460a-8009-cadb383b3262.json
relationship/relationship--2a93ea80-d0f6-4b81-887d-8911f7573245.json
relationship/relationship--2c48f039-61f7-4af4-974b-f0e0fcf95f58.json
relationship/relationship--2eaea386-ee0f-42c4-bca1-ce2d22062f98.json
relationship/relationship--33caa1a2-8465-47b9-89c4-94f4e9a899c7.json
relationship/relationship--358047bf-1dd3-4fc4-bc1a-b7004bd54b8d.json
relationship/relationship--36b9f594-9a27-4281-a18e-9a5e7df70ad9.json
relationship/relationship--37804b22-63b4-4b24-846e-6541688d9213.json
relationship/relationship--380db9ad-f6ad-4988-8a28-b773313f07b7.json
relationship/relationship--464ce0ed-31a5-4a99-9791-9ce5bb987f58.json
relationship/relationship--660d09ce-8722-42b3-8503-911dff37bf22.json
relationship/relationship--69682171-e717-4af7-a24a-06a39f381641.json
relationship/relationship--6a1693a7-1e85-48b6-9097-11339a987099.json
relationship/relationship--6b5c6fc2-615a-46fc-80a4-9ab332159722.json
relationship/relationship--6bf4098c-7667-44df-bdaa-076b9099f851.json
relationship/relationship--71ede2de-7e5f-49fa-ac07-9322ef4857ae.json
relationship/relationship--79106ad4-28d3-4f67-a2c3-116d138ec84a.json
relationship/relationship--a3251b26-7012-4f26-9c5d-1fb9d69b8569.json
relationship/relationship--a83992e1-5be5-433e-b3f1-d9ccde98c9ca.json
relationship/relationship--ab6a19e4-ce00-46cd-ae83-0798471e4a4a.json
relationship/relationship--ab7faed6-3c50-4b04-a31b-ac2c933a51ef.json
relationship/relationship--b052a076-6d4e-49f5-95ac-16264ef05b1d.json
relationship/relationship--b1e7f787-2d43-442b-8bd1-4fa064f089b2.json
relationship/relationship--c1884e62-7b2e-45a1-89fd-c76b1b717f50.json
relationship/relationship--d53d1e84-f4de-4e6a-bc84-5edfce84b055.json

In fact I'd go further: since the refs appear commonly and you use them as reference data, turn them into Report objects. Some of them may not have official status but all of them are used multiple times. Even Wikipedia is used 4 times:

Wikipedia. (2016, June 15)
relationship/relationship--05352dad-ecbb-477c-a05c-5eb3d67ae9ae.json
relationship/relationship--ac3ee298-bef0-4a52-9050-3dcef1701408.json
tool/tool--cf23bf4a-e003-4116-bbae-1ea6c558d565.json

[VladimirAlexiev]

Actually that last one "Wikipedia. (2016, June 15). File Transfer Protocol. Retrieved July 20, 2016." is a bad example. That describes FTP in general, not any adverse use of FTP.
Whereas Report is defined as "threat intelligence focused on one or more topics, such as a description of a threat actor, malware, or attack technique".
But you get my point: these are shared objects, so you better make them out as objects.

[VladimirAlexiev]

We're already doing something similar in our semantic exploits:

But this doesn't quite comply with STIX:

• the report URLs should be .../report--... and the target should be in url
• the link should be "report - object - object" instead of "object - external_reference - report"
• the type should be stix:report not stix:REPORT (I made it uppercase to emphasize it currently does not comply)

[VladimirAlexiev]

Here is a conforming model:

[johnwunder]

Thanks for thinking about this, @VladimirAlexiev!

Using the report object to avoid duplicating the references is an interesting concept, but that object was really designed to be used to convey reports described using STIX. It just has a title, description, and a list of report contents. So I think using it in this way would not be compatible with how other STIX tools use report.

External references are really designed for this use case (referencing non-STIX content by URL). The content does end up being repeated, but because we include the URL that should provide you with a unique ID to be able to correlate how they're used across different objects.

I'd like to close this issue in our repository, because what you're describing is really a feature request for STIX. If you want to make the same suggestion there you can use that tracker: https://github.com/oasis-tcs/cti-stix2/issues.