epc002 / cablemodemdata

Cable Modem Data, how to record and plot, simplified

Cable Modem Data, how to record and plot, simplified

2017-04-13 paul chevalier

Select the image above to present a live graph instance using example data files within this repository (give it a while to display, there're a lot of datapoints to render). Try out the rolling average entries(lower left of each graph, default=10), and use the range selector bars under each graph(far left and right) to zoom in and move around the timeline. (note that the page auto-refreshes every 5 minutes)

---

The Goal:
Persistently capture and record statistics from an Arris SB6183 cable modem, and display historical plots in dygraphs.

Method Overview:

• Periodically run the cableModemCapture-SB6183.sh shell script via cron.
• The script uses the lynx text-based web browser to capture statistical data from the cable modem.
• It parses the data to extract target numerical data.
• Then appends the timestamped numerical data to running CSV data files.
• Graphic plots (using Dygraphs) of the accumulated data are produced in realtime.

Requirements/Dependencies:

• An Arris SB6183 cable modem to capture data from
• A linux instance, or equivalent
• dygraph-combined.js
• A local web server(optional) running on the linux instance

Operational files:

• cableModemCapture-SB6183.sh (typically run every 5 minutes via cron)
• index.html
• dygraph-combined.js
• crontab entry

Data files: three running csv data files are appended to every time the shell script is run:

• DownPowerLevels.csv
• SNR-Levels.csv
• UpPowerLevels.csv

The index.html file references those csv files, and uses dygraphs to produce graphic plots.

A recommended implementation is to run this off a Raspberry-pi with a small UPS, as it’s useful for portability, persistence, and low power consumption. (most any host system can use this method, it only uses a basic shell, lynx, sed, grep, cut, dygraphs, and a scheduler)

Example file structure on Raspberry-pi:

/var/www/html

├── dygraph-combined.js
├── index.html
├── SB6183
│ ├── CableModemCapture-SB6183.sh
│ ├── DownPowerLevels.csv
│ ├── SNR-Levels.csv
│ └── UpPowerLevels.csv

example crontab -l :
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /bin/bash /var/www/html/SB6183/CableModemCapture-SB6183.sh

While the parsing parameters in the capture script are specific to an Arris SB6183 cable modem, they can easily be modified for other modems or devices that can produce a repeatable pattern of data via a URI. To do that, use lynx to dump a text file of the target device data. Then open the text file in an appropriate editor (e.g. notepad++) to easily determine the line numbers and field positions of the target data, and then adjust the parsing parameters in the capture script.

A typical parse event in the script performs a piped sequence of operations against the lynx-captured dump file similar to this:

• use sed to trim to a target range of line numbers:

   sed -n '23,38 p' 

• use grep to trim to lines that contain a target text identifier:

   grep dBmV 

• use sed to shrink multiple whitespace in each line:

   sed 's/   */ /g' 

• use cut with a whitespace delimiter and extract the target field of numerical data:

   cut -d" " -f 10 

• use sed to transpose the extracted data into a single csv line:

   sed -e :a -e '/$/N; s/\n/,/; ta'  

• append the timestamped csv line to the target cumulative csv file

Note that running a web server is only needed if you intend to view the plots via a browser from any system on your lan(preferred). You instead could just view the plots on the hosting system by pointing firefox to the local index.html file. (note that the non-web server method doesn’t work with Chrome browser because it disallows cross origin requests)