fraunhofer-aisec / gallia

Extendable Pentesting Framework

Gallia

Gallia is an extendable pentesting framework with the focus on the automotive domain. The scope of the toolchain is conducting penetration tests from a single ECU up to whole cars. Currently, the main focus lies on the UDS interface. Acting as a generic interface, the logging functionality implements reproducible tests and enables post-processing tasks. The rendered documentation is available via Github Pages.

Keep in mind that this project is intended for research and development usage only! Inappropriate usage might cause irreversible damage to the device under test. We do not take any responsibility for damage caused by the usage of this tool.

Quickstart

See the setup instructions.

First create a config template with --template, store it to a file called gallia.toml, and adjust it to your needs. gallia reads this file to set the defaults of the command line flags. All options correspond to a command line flag; the only required option for scans is gallia.scanner.target, for instance isotp://can0?src_addr=0x123&dst_addr=0x312&tx_padding=0xaa&rx_padding=0xaa.

$ gallia --template > gallia.toml

You are all set to start your first scan, for instance read the diagnostic trouble codes:

$ gallia primitive uds dtc read

The target can also be specified by the --target option on the command line. For the format of the --target argument see the transports documentation.

Acknowledgments

This work was partly funded by the German Federal Ministry of Education and Research (BMBF) as part of the SecForCARs project (grant no. 16KIS0790). A short presentation and demo video is available at this page.

gallia's Issues

Mention conventional commits in contributing

Add this to the contributing documentation.

https://www.conventionalcommits.org/en/v1.0.0/

Implement a proper tester-present background task

The current tester-present background task is broken and it lives at the wrong place, see #21.

• It uses the suppress response flag: Several ECUs ignore this flag and respond with a message. -> boom, the UDS stack is out of sync.
• The background tasks uses the transport directly and bypasses some parts of the stack.
• There are some hacks according to the mutex, e.g. when the ECU responds with BusyResponsePending.

Session check in scan_identifiers shows slight misbehaviour

Starting scan_identifiers.py with --check-session 3 was intended to check the session after every 3rd request.
For service 0x31 the current implementation checks for every third identifier, which results in the following:

Mar 16 22:45:22.066 {ecu     } [message ]: Read current session not supported: requestOutOfRange, skipping check_session
Mar 16 22:45:22.076 {gallia  } [message ]: 0xfff9: requestOutOfRange
Mar 16 22:45:22.086 {ecu     } [message ]: Read current session not supported: requestOutOfRange, skipping check_session
Mar 16 22:45:22.096 {gallia  } [message ]: 0xfff9: requestOutOfRange
Mar 16 22:45:22.106 {ecu     } [message ]: Read current session not supported: requestOutOfRange, skipping check_session
Mar 16 22:45:22.116 {gallia  } [message ]: 0xfff9: requestOutOfRange
Mar 16 22:45:22.126 {gallia  } [message ]: 0xfffa: requestOutOfRange
Mar 16 22:45:22.137 {gallia  } [message ]: 0xfffa: requestOutOfRange
Mar 16 22:45:22.146 {gallia  } [message ]: 0xfffa: requestOutOfRange
Mar 16 22:45:22.157 {gallia  } [message ]: 0xfffb: requestOutOfRange
Mar 16 22:45:22.167 {gallia  } [message ]: 0xfffb: requestOutOfRange
Mar 16 22:45:22.176 {gallia  } [message ]: 0xfffb: requestOutOfRange
Mar 16 22:45:22.187 {ecu     } [message ]: Read current session not supported: requestOutOfRange, skipping check_session
Mar 16 22:45:22.197 {gallia  } [message ]: 0xfffc: requestOutOfRange
Mar 16 22:45:22.207 {ecu     } [message ]: Read current session not supported: requestOutOfRange, skipping check_session
Mar 16 22:45:22.217 {gallia  } [message ]: 0xfffc: requestOutOfRange
Mar 16 22:45:22.226 {ecu     } [message ]: Read current session not supported: requestOutOfRange, skipping check_session
Mar 16 22:45:22.236 {gallia  } [message ]: 0xfffc: requestOutOfRange
Mar 16 22:45:22.246 {gallia  } [message ]: 0xfffd: requestOutOfRange
Mar 16 22:45:22.256 {gallia  } [message ]: 0xfffd: requestOutOfRange
Mar 16 22:45:22.266 {gallia  } [message ]: 0xfffd: requestOutOfRange
Mar 16 22:45:22.276 {gallia  } [message ]: 0xfffe: requestOutOfRange
Mar 16 22:45:22.286 {gallia  } [message ]: 0xfffe: requestOutOfRange
Mar 16 22:45:22.296 {gallia  } [message ]: 0xfffe: requestOutOfRange
Mar 16 22:45:22.306 {ecu     } [message ]: Read current session not supported: requestOutOfRange, skipping check_session
Mar 16 22:45:22.316 {gallia  } [message ]: 0xffff: requestOutOfRange
Mar 16 22:45:22.326 {ecu     } [message ]: Read current session not supported: requestOutOfRange, skipping check_session
Mar 16 22:45:22.336 {gallia  } [message ]: 0xffff: requestOutOfRange
Mar 16 22:45:22.346 {ecu     } [message ]: Read current session not supported: requestOutOfRange, skipping check_session
Mar 16 22:45:22.356 {gallia  } [message ]: 0xffff: requestOutOfRange

Maybe we want to change this to every identifier, but only one sub-function instead.

/cc @ferdinandjarisch

Rename `_setup_transport()` to `load_transport()`

According to load_ecu(). Make the naming consistent.

dump-seeds: provide a flag to powercycle the ecu regularly

pcap: Use zstd

It is supported by default in wireshark. Wait until this version is available in debian stable.

wireshark/wireshark@6fe6899

Wire up CI with virtual ECU

Depends on #82

Replace penrun shellscript with a builtin solution in python

Let's deprecate bin/penrun and provide a built in solution.

Idea so far is:

• Create a reusable python module in order to provide a standalone script and a sophisticated solution within gallia.
• Use TOML for a config file. This will become availabe in the stdlib: https://docs.python.org/dev/library/tomllib.html
• Configure post and pre hooks in toml using a mulitline shell srcipt string.
• Use the same config discovery mechanism as penrun; support inheritence.

Setup blocks if Dumpcap process has terminated

If the parameters for Dumpcap are wrong, the process terminates right after start.
Subsequent calls to sync will deadlock.

How to reproduce:

$ gallia simple-ping --target "isotp://can99" 2>&1|hr
May  9 17:03:53.519 {gallia  } [preamble]: Storing artifacts at /tmp/ping_20220509-170353_Vcy69U4z
May  9 17:03:53.520 {gallia  } [preamble]: Starting "gallia simple-ping" (1.0.0a3) with [gallia simple-ping --target isotp://can99]
May  9 17:03:53.525 {penlab.d} [preamble]: Started "dumpcap": dumpcap -q -i can99 -w - -f 'link[1] == 0x01'
[deadlock]

Dumpcap is started in setup:

gallia/src/gallia/udscan/core.py

Lines 285 to 286 in 4988e4a

	self.dumpcap = await Dumpcap.start(args.target, self.artifacts_dir)
	await self.dumpcap.sync()

Rename `BaseTransport.terminate()` to `BaseTransport.close()`

.close() for closing connections is the lingua franca. .terminate() stems from history…

Find a way to reduce duplicate log messages

Several helpers are called multiple times and they tend to log the same warnings over and over again. Find a way to silence them. Maybe the penlog module needs a log_once() function or similar. Have a look at systemd; they have an implementation of such a function.

Restructure udscan package

The udscan package has history. It is time for a restructuring. Especially the scanner modules and varios “utlils” should live somewhere else.

Don't access entry_points via dict

This avoids snippets like this here #60. But we have to wait until we drop support for Python 3.9.

Rework `bind()` in CANRaw

Rethink the design of this transport according to bind().

Implement and document consistent exit codes

https://www.freebsd.org/cgi/man.cgi?query=sysexits

Fully Implement Unified Discovery Scanners

Track in flight requests/SID Multiplexing

We often have these bugs:

663	2020-10-20 15:46:28,173432	169.254.100.1	169.254.100.68	UDS	67	Request   Read Data By Identifier                0xf186
666	2020-10-20 15:46:28,424213	169.254.100.68	169.254.100.1	UDS	67	Reply     Error                                  Read Data By Identifier (SID: Request correctly received, but response is pending)
668	2020-10-20 15:46:28,430752	169.254.100.68	169.254.100.1	UDS	68	Reply     Read Data By Identifier                0xf186   02
670	2020-10-20 15:46:28,435925	169.254.100.1	169.254.100.68	UDS	66	Request   Tester Present                         Sub-function 0   (Reply suppressed)
673	2020-10-20 15:46:28,458779	169.254.100.1	169.254.100.68	UDS	67	Request   Read Data By Identifier                0x0015
676	2020-10-20 15:46:28,666417	169.254.100.68	169.254.100.1	UDS	67	Reply     Error                                  Read Data By Identifier (SID: Request Out of Range)
678	2020-10-20 15:46:28,672764	169.254.100.1	169.254.100.68	UDS	67	Request   Read Data By Identifier                0xf186
681	2020-10-20 15:46:28,918713	169.254.100.68	169.254.100.1	UDS	67	Reply     Error                                  Read Data By Identifier (SID: Request correctly received, but response is pending)
683	2020-10-20 15:46:28,928805	169.254.100.68	169.254.100.1	UDS	68	Reply     Read Data By Identifier                0xf186   02
685	2020-10-20 15:46:28,933829	169.254.100.1	169.254.100.68	UDS	67	Request   Read Data By Identifier                0x0016
688	2020-10-20 15:46:29,170610	169.254.100.68	169.254.100.1	UDS	67	Reply     Error                                  Read Data By Identifier (SID: Request Out of Range)
690	2020-10-20 15:46:29,176109	169.254.100.1	169.254.100.68	UDS	66	Request   Tester Present                         Sub-function 0   (Reply suppressed)
693	2020-10-20 15:46:29,199441	169.254.100.1	169.254.100.68	UDS	67	Request   Read Data By Identifier                0xf186
696	2020-10-20 15:46:29,210863	169.254.100.68	169.254.100.1	UDS	67	Reply     Error                                  Tester Present (SID: Request correctly received, but response is pending)
698	2020-10-20 15:46:29,416685	169.254.100.68	169.254.100.1	UDS	66	Reply     Tester Present                         Sub-function 0

Find a way to multiplex/track in flight request/responses.

Create Scanners for DLT

• https://www.autosar.org/fileadmin/user_upload/standards/classic/4-3/AUTOSAR_SWS_DiagnosticLogAndTrace.pdf
• https://github.com/COVESA/dlt-daemon

I have found this implementation by @menschel which could potentially be used as a baseline. Since it is licensed under CC we need to clarify if we are allowed to (re)use parts of it.

Configure pypi deployment via GH Actions

Separate `GalliaBaseSync` from `GalliaBaseAsync`

#85 suggest providing a more basic GalliaBase class. In several use cases it is not needed to provide an async def main(…) …. Provide a base class which is not async.

Decode xcp response in discovery scanner

The XCP discovery scanner looks at very few bytes…

Create a more basic `GalliaBase`

GalliaBase includes a lot of stuff. For just adding commands to gallia it is a bit too much. Create a very basic GalliaBase class.

Insufficient penrun docu

Penrun documentation lacks information about how to use the template, e.g.

# current working dir
cp misc/penrun.config ./.penrun.sh

# global user config
cp misc/penrun.config ~/.config/penrun/config.sh

docs build with errors/warnings

See: https://github.com/Fraunhofer-AISEC/gallia/runs/6224701583?check_suite_focus=true

[autosummary] generating autosummary for: api/gallia.cursed_hr.rst, api/gallia.db.rst, api/gallia.penlab.rst, api/gallia.rst, api/gallia.services.rst, api/gallia.transports.rst, api/gallia.uds.core.rst, api/gallia.uds.rst, api/gallia.udscan.rst, api/gallia.udscan.scanner.rst, api/modules.rst, contributing.md, index.rst, power_supply.md, setup.md, transports.md
Failed to import gallia.penlab.opennetzteil.
Possible hints:
* ModuleNotFoundError: No module named 'gallia.penlab.opennetzteil'; 'gallia.penlab' is not a package
* AttributeError: module 'gallia.penlab' has no attribute 'opennetzteil'
Failed to import gallia.db.db_importer.
Possible hints:
* ModuleNotFoundError: No module named 'gallia.db.db_importer'
* AttributeError: module 'gallia.db' has no attribute 'db_importer'
myst v0.17.2: MdParserConfig(commonmark_only=False, gfm_only=False, enable_extensions=[], linkify_fuzzy_links=True, dmath_allow_labels=True, dmath_allow_space=True, dmath_allow_digits=True, dmath_double_inline=False, update_mathjax=True, mathjax_classes='tex2jax_process|mathjax_process|math|output_area', disable_syntax=[], all_links_external=False, url_schemes=('http', 'https', 'mailto', 'ftp'), ref_domains=None, highlight_code_blocks=True, number_code_blocks=[], title_to_header=False, heading_anchors=None, heading_slug_func=None, footnote_transition=True, sub_delimiters=('{', '}'), words_per_minute=200)
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 16 source files that are out of date
updating environment: [new config] 16 added, 0 changed, 0 removed
reading sources... [  6%] api/gallia
reading sources... [ 12%] api/gallia.cursed_hr
reading sources... [ 18%] api/gallia.db
reading sources... [ 25%] api/gallia.penlab
reading sources... [ 31%] api/gallia.services
reading sources... [ 37%] api/gallia.transports
reading sources... [ 43%] api/gallia.uds
reading sources... [ 50%] api/gallia.uds.core
reading sources... [ 56%] api/gallia.udscan
reading sources... [ 62%] api/gallia.udscan.scanner
reading sources... [ 68%] api/modules
reading sources... [ 75%] contributing
reading sources... [ 81%] index
reading sources... [ 87%] power_supply
reading sources... [ 93%] setup
reading sources... [100%] transports

/home/runner/work/gallia/gallia/docs/api/gallia.rst:7: WARNING: toctree contains reference to nonexisting document 'api/gallia.analyzer'
/home/runner/work/gallia/gallia/docs/api/gallia.rst:7: WARNING: toctree contains reference to nonexisting document 'api/gallia.complex_args'
/home/runner/work/gallia/gallia/docs/api/gallia.rst:7: WARNING: toctree contains reference to nonexisting document 'api/gallia.penreport'
/home/runner/work/gallia/gallia/src/gallia/cursed_hr/cursed_hr.py:docstring of gallia.cursed_hr.cursed_hr.CursedHR.parse_structure:6: ERROR: Unexpected indentation.
WARNING: autodoc: failed to import module 'db_importer' from module 'gallia.db'; the following exception was raised:
No module named 'gallia.db.db_importer'
WARNING: autodoc: failed to import module 'opennetzteil' from module 'gallia.penlab'; the following exception was raised:
No module named 'gallia.penlab.opennetzteil'; 'gallia.penlab' is not a package
/home/runner/work/gallia/gallia/docs/api/gallia.uds.rst:7: WARNING: toctree contains reference to nonexisting document 'api/  :maxdepth: 4'
/home/runner/work/gallia/gallia/src/gallia/uds/core/client.py:docstring of gallia.uds.core.client.UDSClient.read_dtc_information_report_dtc_by_status_mask:7: WARNING: Field list ends without a blank line; unexpected unindent.
/home/runner/work/gallia/gallia/src/gallia/uds/core/client.py:docstring of gallia.uds.core.client.UDSClient.read_dtc_information_report_emissions_related_obd_dtc_by_status_mask:8: WARNING: Field list ends without a blank line; unexpected unindent.
/home/runner/work/gallia/gallia/docs/api/gallia.udscan.rst:7: WARNING: toctree contains reference to nonexisting document 'api/gallia.udscan.plugins'
/home/runner/work/gallia/gallia/src/gallia/udscan/utils.py:docstring of gallia.udscan.utils.range_diff:7: ERROR: Unexpected indentation.
/home/runner/work/gallia/gallia/src/gallia/udscan/utils.py:docstring of gallia.udscan.utils.range_diff:19: WARNING: Field list ends without a blank line; unexpected unindent.
/home/runner/work/gallia/gallia/docs/index.rst:4: WARNING: toctree contains reference to nonexisting document 'testing'
/home/runner/work/gallia/gallia/docs/setup.md:15: WARNING: duplicate label setup:setup, other instance in /home/runner/work/gallia/gallia/docs/setup.md
looking for now-outdated files... none found
pickling environment... done
/home/runner/work/gallia/gallia/docs/api/gallia.services.rst: WARNING: document isn't included in any toctree
/home/runner/work/gallia/gallia/docs/contributing.md: WARNING: document isn't included in any toctree

scan-reset: handle timeout during `wait_for_ecu()`

scan-reset uses wait_for_ecu() to await the ECU coming online again. The case when the ECU dies is not handled. Allow wait_for_ecu() to raise a TimeoutException.

gallia console/tui

Bump version correctly

Maybe this can help: https://github.com/mtkennerly/poetry-dynamic-versioning

python penrun: Provide a TOML config

Scan dump seeds should inform about progress after time rather than after modulo seeds

When dumping seeds is real slow, one has to wait for an eternity to get the feedback of actual dumping speed.

/cc @ferdinandjarisch

Rework scanners and use more advanced features from the stack

A few scanners are already old and might benefit from a rework. In the meanwhile our UDS stack has gained quite substancial cababilities. It might be beneficial to rework some old scanners and use new features.

Reuse Code in Github Actions

The current Github Actions configs share some copy & pasted code. Clean this up and find out how to reuse code snippets.

(Sub)Command structure

I hit the “Create Issue from Discussion Button”, since discussion could not be linked into projects.

Discussed in #76

^{Originally posted by rumpelsepp May 9, 2022}
I worked a bit on #9 and reached a blocker. Gallia is going to become a framework which is not limited to uds only. Therefore, it make sense to structure the CLI interface better. The challenge is to create a somehow self describing CLI interface which is easy to remember and easy to use.

I had a few ideas for structuring the commands. The structure of the code base could follow this accordingly. Personally, I think “Subcommands grouped by operation” is cool.

Subcommands grouped by service

• gallia
  • uds
    • scan
      • identifiers
    • discover
    • primitive
      • dtc
      • read-data-by-identifier
  • xcp
    • discover
  • dlt
    • discover

Examples

$ gallia uds scan identifiers
$ gallia xcp discover
$ gallia uds primitives dtc

Subcommands grouped by operation

• gallia
  • scan
    • uds
      • identifiers
  • discover
    • xcp
    • uds
    • dlt
  • primitives
    • uds
      • dtc
      • read-data-by-identifier

Examples

$ gallia scan uds identifiers
$ gallia discover xcp
$ gallia primitives uds dtc

No subcommands grouped by prefix

• gallia

  • dlt-discover
  • uds-discover
  • uds-primitives-dtc
  • uds-primitives-read-data-by-identifier
  • uds-scan-identifiers
  • xcp-discover

Examples

$ gallia uds-scan-identifiers
$ gallia xcp-discover
$ gallia uds-primitives-dtc
```</div>

Filter for idle CAN IDs in find-can-ids scanner

When there are idle-can-messages, the scanner might not work.

Move `check_and_set_session()` to ECU class

Move check_and_set_session() to ECU class; might be helpful for #13

Improve README

Log and tag UDS traffic in UDSClient

Currently uds traffic is logged in the transports which causes code duplication. Move this to the UDSClient.

Full UDS primitive coverage

Primitives such as ReadDataByPeriodicIdentifier are missing in the UDS Stack.

Configure transport specific address scan range for discover-endpoints scanner

discover-endpoints supports multiple transports. Choose the address range depending on the chosen transport automatically.

Log session transitions in DB

Implement Lazy Loading

import importlib.util
import sys


def lazy_import(name):
    if name in sys.modules:
        return sys.modules[name]
    spec = importlib.util.find_spec(name)
    loader = importlib.util.LazyLoader(spec.loader)
    spec.loader = loader
    module = importlib.util.module_from_spec(spec)
    sys.modules[name] = module
    loader.exec_module(module)
    return module

UDS max_retry value misleading

max_retry is used in a loop like this:

for i in range(0, max_retry):

Setting max_retry to 0 e.g. causes the UDS request to be silently dropped. A value of 1 means that the request just gets sent without a retry. This is a bug.

Insufficient dependencies in docu

Original reporter: @ferdinandjarisch

From what I see at least the following are missing in the docu:

• jq (penrun)
• dumpcap (über package tshark oder wireshark-common)

@fkglr @VeroSec @emedav Please add from your recent experiences…

Maybe one should also give instructions on how to install poetry, even if only a link to their installation wiki

Move session up the hierarchy

Currently (nearly) all scanners offer a --session argument, which allows to set the session. Shouldn't we move the session argument up to the UDSScanner class then?

/cc @fkglr

Rework mandatory src_addr/dst_addr

find-endpoints is very buggy due to some constraints with the src_addr and dst_addr

Move tester present to ECU

Currently, the tester present worker is in the scanner class, which is unfortunate. Move it to the ECU class. Is required for #18

While retrying request_pdu the mutex should be released or TesterPresent should be sent

While retrying in request_pdu_unsafe method the mutex is locked, resulting in exponentially increasing times where no other communication is allowed (e.g. TesterPresent).

While waiting, the mutex should be unlocked to allow these messages.

/cc @ferdinandjarisch

Find out how to perform scans with security access

penlab: enable multiple netzteil servers

Maybe a change in pyopennetzteil is needed.

Do check_session inside set_session

Pseudocode:

def set_session(session) -> bool:
    if check_session(session):
        return True
    _set_session(session)
    return check_session(session)

Extend dump-seeds with automatic recognition of NRCs

It might be useful to trigger a reset on specific NRCs. This can be used to detect a brute-force protection and only reset the ecu when the threshold is reached