acooks / jittertrap

JitterTrap is a measurement and impairment tool for developers of latency-sensitive applications and devices.

JitterTrap

JitterTrap is a network measurement and impairment tool for developers of real-time applications and devices.

It has two broad areas of use:

• detection and measurement of unexpected delays, introduced by the device or application under test. That is, characterising the source behaviour with respect to throughput, packet rates, jitter.
• introducing and controling known network impairment conditions (eg. delay, jitter, packet loss) to verify the correct operation of the device or application under test. That is, characterising the behaviour of the destination, with respect to delay, jitter and loss.

The user interface is implemented as a web application. Have a look at the demo hosted on a t2.micro instance at AWS Sydney. (Performance is highly variable.)

Or try this old demo video:

Help is available from [email protected], or create a github issue.

Installing JitterTrap

We're aiming to release packages for Fedora, Ubuntu and OpenWRT and would appreciate help with that.

Building JitterTrap

Dependencies

• libnl >= 3.2.24
• libwebsockets >= 1.6
• libjansson >= 2.6

Fedora

Build dependencies:

sudo dnf install libnl3-devel jansson-devel libwebsockets-devel libpcap-devel

Run-time dependencies:

sudo dnf install libnl3 jansson libwebsockets libpcap

Ubuntu

Build dependencies:

sudo apt-get install libnl-3-dev libnl-route-3-dev libnl-genl-3-dev libjansson-dev libwebsockets-dev libncurses5-dev libpcap-dev pkgconf

Run-time dependencies:

sudo apt-get install libnl-3-200 libnl-route-3-200 libnl-genl-3-200 libjansson4 libwebsockets6

Compiling JitterTrap

Fetch:

git clone https://github.com/acooks/jittertrap.git

Build:

cd jittertrap
make

Run:

sudo ./server/jt-server --port 8080 --resource_path html5-client/output/

Now point your web browser to the user interface, eg. http://localhost:8080/

jittertrap's Issues

Add a "show static chart" action to traps.

Background
Currently the Traps module does little more than change the colour of a virtual LED, but it could do so much more! The next, simple action to implement is to pop up a static chart that supports zooming, panning and export to png/jpeg file.

Tasks

1. Add ability to select a trap action to the "Add trap template".
2. Add a UI container for the "Show Chart" action.
3. Add mechanism to jittertrap-charts.js to make a copy of a series and draw the series in the traps module's container.

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Remove mallocs in the fast path

In jt_server_message_handler.c, the unsent_frame_count_mutex acquired in the stats thread's event handler is also acquired in stats_filter_and_write(), which does a whole bunch of mallocs and other slow operations.

One possible way to fix this, is to change the struct jt_msg_stats so that we can allocate a buffer on the stack and make a deep copy into it in the critical section.

Rework chart displays

Background
Not all charts are relevant or interesting under all circumstances and can cost substantial cpu time to draw. Similarly, there are use cases where it would be useful to have both ingress and egress displayed simultaneously.

Tasks

1. Rework index.html so that each possible chart has a unique container div.
2. Rework jittertrap-charts.js so that drawing of each chart can be toggled on or off independently.
3. Rework jittertrap-core.js so that all chartable series are available as needed, ie. remove selectedSeries and add filteredData arrays.

Triage toptalk chart load rendering quirk [$100]

When opening http://demo.jittertrap.net, the toptalk chart renders incorrectly. Sometimes the curve drops below zero on the y-axis or look as if elements are rendered out of sequence. The attached example screenshots illustrate the issue:

I have not been able to reproduce the issue when running on the local machine (connecting to localhost).

---

Did you help close this issue? Go claim the $100 bounty on Bountysource.

could't find netem qdisc for iface: eth1

I compiled jittertrap from git (rev 185dacd), but when I run the server it fails to start;

root@jittertrap:~/workspace/jittertrap# ./server/jt-server --resource_path html5-client/output/
Setting resource path to "html5-client/output/"
lwsts[3425]: jittertrap server
Using resource path "html5-client/output/"
lwsts[3425]: Initial logging level 7
lwsts[3425]: Libwebsockets version: 1.6.0 4939a70
lwsts[3425]: IPV6 not compiled in
lwsts[3425]: libev support not compiled in
lwsts[3425]:  Threads: 1 each 1024 fds
lwsts[3425]:  mem: platform fd map:  4096 bytes
lwsts[3425]:  mem: per-conn:          284 bytes + protocol rx buf
lwsts[3425]:  canonical_hostname = jittertrap
lwsts[3425]:  Compiled with OpenSSL support
lwsts[3425]:  Using non-SSL mode
lwsts[3425]: ERROR on binding to port 80 (-1 98)
lwsts[3425]: lws_context_destroy
creating new queue for 31 messages of 3000 bytes each
switching to iface: [eth1]
monitoring iface: [eth1]
get netem for iface: [eth1]
could't find netem qdisc for iface: eth1
couldn't get netem parameters.
iface: eth1
iface: eth0
iface: eth2
iface: br0
4 ifaces
iface: eth1
iface: eth0
iface: eth2
iface: br0
get netem for iface: [eth1]
could't find netem qdisc for iface: eth1
couldn't get netem parameters.
lwsts[3425]: libwebsocket init failed
pthread_setaffinity_np: Invalid argument

Any idea why it would fail to get the netem parameters ?

Write a blog post about jittertrap [$35]

Write a public blog post about Jittertrap. Points for...

• What is it?
• What is the problem to solve?
• Show screenshot

---

Did you help close this issue? Go claim the $35 bounty on Bountysource.

Create a package for OpenWRT [$30]

• Create a package for OpenWRT
• Document the process and progress in this issue
• Publish this work somewhere (preferably upstream in OpenWRT)

---

There is a $30 open bounty on this issue. Add to the bounty at Bountysource.

Investigate Chart.js as an alternative to CanvasJS

Background
There are a few issues with CanvasJS that makes it desireable to look for alternatives.

1. License.
   1.1 CanvasJS is available under commercial and CC-BY-NC licenses, but the NC license plasters an ugly CanvasJS.com tag on every chart.
   1.2 The license may also cause problems when packaging JitterTrap for Fedora and Ubuntu.
   1.3 The license does not encourage me to fix CanvasJS problems.
2. Performance.
   2.1 Drawing the charts is by far the most expensive part of the frontend. Most of the cost seems to be associated with fonts and labels - which don't really change once a data full series has been collected!
3. Code Quality.
   3.1 Having looked at the source, I have some concerns about the long-term maintainability of CanvasJS. For example, the current implementation generates a lot of garbage, but using a CBuffer as the backing store for chart data would be quite difficult.
4. Missing features.
   4.1 It would be preferable to draw the chart such that the X-axis has -X on the left and 0 on the right, with the data still moving from right to left across the screen. CanvasJS does not seem to be able to do this properly.

Tasks

1. Investigate whether Chart.js charts can easily be updated continuously (it seems possible, but slightly awkward).
2. Check what the Chart.js performance is like.
3. Some of the features of CanvasJS (type: "rangeArea") might not be available. Check them.
4. To use Chart.js, abstractions in jittertrap-core.js and jittertrap-charts.js would have to change.

Plot heat maps of per-connection resonse times

Once per-connection response times are collected/calculated (in #48), plot the response times as heat maps.

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Lacks Tests

JitterTrap is desperately lacking some tests.

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Drop root privs, use capabilities [$15]

This is the current set of capabilities to get things working:

sudo setcap "cap_sys_admin,cap_sys_resource,cap_syslog,cap_sys_nice,cap_block_suspend,cap_net_bind_service,cap_net_raw,cap_net_admin+eip" server/jt-server

This is quite a lot!

Plan:

1. This might be more than the minimum caps required. Find the absolute minimum set of capabilities needed to run right now.
2. Fix checking of the return codes for the calls that require caps.
3. Comment the code so that it's clear what calls need what caps.
4. Drop caps when no longer needed.

---

There is a $15 open bounty on this issue. Add to the bounty at Bountysource.

Turn the evil vendor tracking code off by default

In html5-client/src/js/jittertrap.js there is a bit of code that posts the version info to https://www.jittertrap.net/tracker. This code makes it possible to track whether anyone is running or hacking on jittertrap, but user tracking is unpopular and must be disabled by default in releases (it will remain enabled on http://demo.jittertrap.net).

TODO:

• Add a compile-time variable to make.config to set tracking on/off
• Use the variable in html5-client/Makefile to do a simple string replacement in html5-client/src/js/jittertrap.js using sed.
• Test the replaced value in html5-client/src/js/jittertrap.js to enable/disable the POST.

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Add a rate limit to Impairments module

Background
Most people think of throughput as the defining characteristic of any network transport, rather than delay and delay variation. JitterTrap currently lacks the ability to specify the maximum throughput.

Tasks

1. Add support for the 'rate' parameter of the netem qdisc to libnl.
2. Work with upstream to merge this.
3. Add support for specifying the rate to index.html.
4. Add support for rate parameter to netem config messages.

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Show response times in top talk talkers view

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

upgrade d3.js from 3.5.13 to 4.7.4 or later [$50]

D3.js is an external JS library used to draw the charts. It's a dependency that is bundled with Jittertrap and kept in the repository at html5-client/deps/js/d3.min.js.

It is included into the JS application in html5-client/src/html/index.tpl.html and used in:

• src/js/jittertrap-charting.js
• src/js/jittertrap-chart-tput.js
• src/js/jittertrap-chart-toptalk.js
• src/js/jittertrap-chart-pgaps.js

The v4 API has changed and the changes affect the chart drawing code in jittertrap. Changing to the new API may or may not be a simple mechanical search and replace process.

Todo:

• Investigate the scope of API changes
• Upgrade the bundled d3.min.js to the latest release version (4.7.4 or greater) and convert to the new v4 API

---

Did you help close this issue? Go claim the $50 bounty on Bountysource.

jt-server assumes SMP

When running on a single-cpu vm, jt-server exits immediately with "pthread_setaffinity_np: Invalid argument".

Do not assume an SMP system, but warn user and continue instead.

Implement Tests module

Background
The current Impairments, Measurements and Traps modules lack a mechanism to do repeatable test scenarios. A Test module is needed!

Task
Tests specify:

• a tuple of impairment parameters for a particular duration,
• a set of Traps for that duration (or the overall progression of impairments)
• a collection of measurements for each tuple of impairments, as well as the whole test.

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Make location of external js dependencies configurable [$15]

To comply with Fedora packaging guidelines, dependencies like d3.js need to be installed system-wide, away from the rest of the web assets.

However, some use cases require the js dependencies to be served by the jittertrap server, like running on an isolated test network. (As it works currently)

To do:
Review html5-client/src/html/index.tpl.html and make the location of each of the 3rd-party javascript dependencies configurable, so that it's possible to comply with Fedora packaging guidelines. A simple compile-time 'sed' or 'perl' template replacement in html5-client/Makefile may be sufficient.

---

There is a $15 open bounty on this issue. Add to the bounty at Bountysource.

Add capability to export/save measurements, traps & test results.

The ability to save (or export) measurements, test results and raised traps (with trigger values) is needed for user records and benchmark comparisons.

Tasks

1. Define a file format.
2. Add "save" button to measurements tab.
3. Add "save" button to traps.
4. Add "save" button to tests module.

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Cleanup server-side debugging output [$50]

The server application is currently very noisy.

Todo:

• Cleanup the application to be silent by default
• Add a '-d' option to keep the current debugging output.

Example of current output on startup:

$ sudo ./server/jt-server --resource_path html5-client/output/
[sudo] password for acooks:
Setting resource path to "html5-client/output/"
lwsts[25295]: jittertrap server
Using resource path "html5-client/output/"
lwsts[25295]: Initial logging level 7
lwsts[25295]: Libwebsockets version: 2.1.0 unknown-build-hash
lwsts[25295]: IPV6 not compiled in
lwsts[25295]: libev support not compiled in
lwsts[25295]: libuv support not compiled in
lwsts[25295]: Threads: 1 each 1024 fds
lwsts[25295]: mem: platform fd map: 8192 bytes
lwsts[25295]: Compiled with OpenSSL support
lwsts[25295]: SSL disabled: no LWS_SERVER_OPTION_DO_SSL_GLOBAL_INIT
lwsts[25295]: Creating Vhost 'default' port 80, 2 protocols, IPv6 off
lwsts[25295]: Listening on port 80
lwsts[25295]: mem: per-conn: 512 bytes + protocol rx buf
lwsts[25295]: canonical_hostname = t460s.rationali.st
lwsts[25295]: lws_protocol_init
creating new queue for 15 messages of 4096 bytes each
switching to iface: [enp0s31f6]
RT thread [jt-toptalk] priority [3] CPU affinity: CPU0
creating new queue for 15 messages of 2288 bytes each
creating new queue for 15 messages of 160 bytes each
RT thread [jt-compute] priority [1] CPU affinity: CPU0
consumer 42424242 joined
consumer 42424242 joined
available ifaces: enp0s31f6RT thread [jt-intervals] priority [2] CPU affinity: CPU0
RT thread [jt-sample] priority [2] CPU affinity: CPU0
wlp4s0 virbr0 virbr0-nic vm-br0

---

Did you help close this issue? Go claim the $50 bounty on Bountysource.

Dynamically adjust interface list

The interface list doesn't handle dynamically adding/removing interfaces. For example, new tap devices created by qemu-bridge-helper don't show up in the the dropdown and, if these interfaces are removed after if jt-server is started it causes invalid state and crashed.

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Create a distro package for Debian [$65 awarded]

There's a half-baked debian package in the source tree to use as a starting point.

Todo:

• Build a debian package
• Submit to debian upstream for inclusion
• Document and track the process in this github issue

---

The $65 bounty on this issue has been claimed at Bountysource.

Interface change crash

Assertion failed on build of master @ commit 2d36223, when changing the selected interface:

jt-server: intervals.c:404: void tt_get_top5(struct tt_top_flows *, struct timeval): Assertion `0' failed.

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.