jstop / wind.tunnel

FHIR server performance testing

Windtunnel (beta)

Performance & stress testing for your FHIR server.

This is the first release to gauge the community interest and we'd love to hear your feedback on it. For feedback and known issues, see here.

Setup

First, prepare the server under test with sample data. Either upload all Synthea bundles yourself or do:

1. Zip up all of the resource to upload with atool: apack upload.zip performance-data/
2. Upload zip with Vonkloader: vonkloader -file:upload.zip -collectionHandling:Split -server:http://<my server>

Install Python dependencies

1. Install Python dependencies using pip (if you don't have it, install first): pip install influxdb jsonpath_rw statistics psutil pathlib fhirclient numpy monotonic

Extra - own private dashboard

It's also possible to setup your own, private dashboard by hosting your own InfluxDB+Grafana and importing the dashboard used.

Running performance tests

Run start_performance_test.sh - parameters supported are:

• --backend (required): backend in use by target host (mongo, postgres, memory, sqlite, or sqlserver)
• --host (optional): system under test (eg. http://localhost)
• --influxdb (required): InfluxDB intake to sends results to (eg. http://grafana-locust-firely.westeurope.cloudapp.azure.com:9086)
• --duration (optional): customise how long to run each test for. By default, each test is run for 5mins. The format is that of the --run-time option of Locust itself, e.g. like '1h30m'

# example: run performance tests against localhost:4080
./start_performance_test.sh --backend mongo --host http://localhost:4080 --influxdb http://grafana-locust-firely.westeurope.cloudapp.azure.com:9086

Viewing performance test results

Results are available on online:

1. Go to inspect-particular-test-run
2. In the Select run dropdown, select your particular test run
3. Copy the year+time timestamp and paste it into the From: field in the time range top-right
4. Paste the timestamp plus two hours into the To: field

See gif of the process.

Aggregated results are available at overview-of-all-runs-results.

Running stress tests

Stress testing is different from performance testing: whereas performance testing will help you see how better (or worse) you're doing in different scenarios, stress testing will push your configuration to the max to see what you can handle.

Run start_stress_test.sh - parameters supported are:

• --host (required): system under test (eg. http://localhost)
• --ignore-lock (optional): allow stress test to run even while another performance or stress test is running

 ./start_stress_test.sh --host http://localhost

Viewing stress test results

Open up the results on http://localhost:8089.

Built With

• Python
• Locust - An open source load testing tool
• InfluxDB - Scalable datastore for metrics, events, and real-time analytics
• Grafana - The tool for beautiful monitoring and metric analytics & dashboards for Graphite, InfluxDB & Prometheus & More

Authors

• Lilian Minne
• Vadim Peretokin

License

BSD 3-clause.