CVE Research

This is a research project of CVE fixes.

Files in this repo

script-upstream.py - process CVE in a given upstream python project
cve.json - analysis for respective python projects
openstack-backports.json.xz - backport data for nova, horizon, keystone, ironic, neutron...

In the current state we are only interested in Python projects and CVE fixes that take place in a single code file. More complex fixes are skipped for now.

How this works?

We clone high-profile python projects and try to find CVE string in the commit messages.

JSON structure (cve-data)

cve_id: CVE ID, in the form of CVE-{number}-{number}, empty if not detected

Upstream project metadata

upstream_patch: diff of the upstream code fix (fixes composed of multiple commits are not detected right now)
upstream_commit_hash: upstream commit hash that fixes the CVE (can be used in the upstream git repo)
upstream_commit_message: commit message of the single commit that fixed the CVE
upstream_code_before: source file before applying the patch
upstream_code_after: source file after applying the patch
upstream_project: name of the open-source project where the CVE occurred

Downstream project metadata

Not available yet.

downstream_patch: TBD

JSON structure (django and cpython backport data)

Running the script with get-django-backports or get-cpython-backports will produce a json output containing data on how django developers backported some of the fixes. Both are amazing sources for this data as they maintain multiple parallel streams. These are not just CVE backports, but also regular bug fixes.

backport_patch: diff of the backported code fix (fixes composed of multiple commits are not detected right now)
backport_commit_hash: backport commit hash that fixes a bug (can be used in the upstream git repo)
backport_commit_message: commit message of the single commit that fixed the bug: it contains string Backport of {commit_hash} and this is how we discover upstream_commit_hash, kudos django maintainers.
backport_before: source file before applying the backported patch: this is the code that developers need to update once the problem is fixed in the main development branch
upstream_patch: diff of the upstream code fix: upstream in this context mean the main development branch of django - main
upstream_commit_hash: upstream commit hash
upstream_commit_message: commit message of the original fix
upstream_before: source file before applying the fix in the main development branch

Example run

You need to have python3 and git available on your system to run it.

$ ./script-upstream.py
[
  {
      "cve_id": "CVE-2023-...

This will print a json-formatted metadata described above. The script will clone selected high-profile python projects and analyze them. It creates a directory called "workspace" where the analysis will happen. Please be aware the git repos take several gigabytes on a disk.

TomasTomecek/cve-data

README