mozilla / telemetry-dashboard

Web-frontend for aggregated telemetry data

Telemetry Dashboard

This repository contains the source code for telemetry.mozilla.org. The dashboards on this site can be used for everything from checking measure values to figuring out common causes of Firefox hangs.

The main dashboards on telemetry.mozilla.org consume data from Telemetry's backend using Telemetry.js.

This repository also contains the source code for Telemetry.js. The specific files can be found under the v2/ directory.

Deploying Telemetry Dashboard

The telemetry.mozilla.org site is hosted in Github Pages, so it may also be accessed via mozilla.github.io/telemetry-dashboard. In front of Github Pages, there is also a GCP CDN (managed by DSRE).

Updates to the gh-pages branch (also the default branch) will be reflected on telemetry.mozilla.org after a few moments.

Using Telemetry.js

Check out the documentation!

• Telemetry.js v2

Adding Telemetry Probes

See this MDN article, which outlines the process and details for adding new Telemetry probes to Firefox which can be used with the dashboards.

For setting histogram properties, make sure to check out the histogram simulator, which might help with designing histograms that fit the expected data well.

Contributing to the Telemetry Dashboard

This project is entirely open source, and licensed under the MPL to boot. Contributions welcome!

Getting started

Looking for some task to get started on? Check the list of mentored issues. If you're unsure what to choose, just get in touch.

Communication happens mostly on Github in the comments for issues or pull requests, but we're also active in IRC in #tmo on irc.mozilla.org.

Working on the dashboard

A local webserver is really helpful for running a version of the site on the same machine you're developing on.

• This can be done as follows (requires Python):

      cd /PATH_TO_REPOSITORY_ROOT
      python -m http.server
      # now visit localhost:8000 in your browser to see the page
  

• It can also be done with NPM:

      cd /PATH_TO_REPOSITORY_ROOT
      npm install http-server -g
      # now visit localhost:8080 in your browser to see the page
  

Shortlink buttons (in the top right hand corner of the main dashboards) will not work when running the site on local servers. This is because they are shortened with bit.ly, which doesn't allow local links.

Submitting pull requests for the dashboard

For pull requests, it is recommended that you set up a live site hosting your branch. This makes it a lot easier for reviewers to check out the changes.

GitHub Pages is great for this, you can enable this in your forked repository under "Settings", "Github pages" and then share the link in your pull requests comments. The Github help pages have more information.

If you need more than one test site up at a time, try Divshot, a static website hosting service.

Working on Telemetry.js

Telemetry.js (both versions) is pretty straightforward to work on. However, note that sites that use Telemetry.js generally hotlink to the source files - make sure to preserve API backwards compatibility wherever possible.

telemetry-dashboard's Issues

Ignore leading or trailing space in histogram name box

Pie charts "show table" is broken

http://bit.ly/1NDx5ZL

"Compare by" for boolean histograms shouldn't show a 3-point line

These should be bar graphs or multiple pie charts or something else, definitely not a 3-point curve http://bit.ly/1NDvfIn

CORS request blocked

Is this error unique to the divshot.io deployement?

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://aggregates.telemetry.mozilla.org/aggregates_by/submission_date/channels/nightly/?version=42&dates=20150818,20150817,20150816,20150815,20150814,20150813,20150812,20150811,20150810,20150808,20150807,20150806,20150805,20150804,20150803,20150802,20150801,20150731,20150727,20150726,20150725,20150724,20150723,20150722,20150721,20150720,20150719,20150718,20150717,20150716,20150715,20150714,20150713,20150712,20150711,20150710,20150709,20150708,20150707,20150706,20150705,20150704,20150703,20150702,20150701&metric=JS_TELEMETRY_ADDON_EXCEPTIONS. (Reason: CORS header 'Access-Control-Allow-Origin' missing).

Keyed histogram dash shouldn't sanitize key list?

This should have hundreds of keys by default: http://mzl.la/1fokpdu

Telemetry docs need to cover keyed histograms

loadEvolutionOverBuilds should always call the callback

If the caller calls for measures with no data, the callback is never reached. This makes it difficult to do error-handling. There's also no return value to help the caller not hang.

Same for loadEvolutionOverTime.

JSFiddle doesn't work on Nightly

This works in Chrome but not Firefox Nightly:

http://jsfiddle.net/mkXLg/

It looks like innerText doesn't exist on the DOM node, re-implementing it with querySelector ( for speed ) and textContent works though:

http://jsfiddle.net/G7BZS/4/

Histogram simulator dash errors out

"uncaught exception: sum must be a number"

https://telemetry.mozilla.org/histogram-simulator/

Histogram descriptions in the dashboard are converted to all-uppercase

Histogram descriptions from Histograms.json are mixed-case, recently the dashboard started showing them as all-uppercase.
This makes readability much worse, can we keep them mixed-case or is there any important reason behind this?

Bug in Histogram table-view

Go to http://bit.ly/1LileQP and select between "Sanitize" and "Don't sanitize".

First 2 table rows with "Sanitize":

Start	End	Windows Vista Count	Windows 8.1 Count	Windows 7 Count	Windows 10 Count
1	22	568.66k	1.18M	817.7k	98.15k

First 2 table rows with "Don't sanitize":

Start	End	Windows Vista Count	Windows 8.1 Count	Windows 7 Count	Windows 10 Count	Windows 8 Count
1	22	4.09k	568.66k	1.18M	817.7k	98.15k

Note how the last column is no longer "Windows 10 count" and is instead "Windows 8 count", but the sample count in the last column is unchanged?

Keyed histogram shown as unkeyed

STR:

1. Open Histogram dashboard
2. Select histogram PLUGIN_ACTIVATION_COUNT

Expected: shown as a keyed histogram
Actual: shown as a regular count histogram! Generated URL: https://telemetry.mozilla.org/new-pipeline/dist.html#!cumulative=0&end_date=2015-09-30&keys=flash!__none__!__none__!__none__&max_channel_version=nightly%252F44&measure=PLUGIN_ACTIVATION_COUNT&min_channel_version=null&product=Firefox&sanitize=1&sort_keys=submissions&start_date=2015-09-29&table=0&trim=1&use_submission_date=0

However, if I disable data sanitization, then I get the correct graph:

https://telemetry.mozilla.org/new-pipeline/dist.html#!cumulative=1&e10s=true&end_date=2015-09-30&keys=flash!npvlc!nplwaplugin!npbecon&max_channel_version=nightly%252F44&measure=PLUGIN_ACTIVATION_COUNT&min_channel_version=null&os=Windows_NT&product=Firefox&sanitize=0&sort_keys=median&start_date=2015-09-27&table=0&trim=1&use_submission_date=0

The version selector text box should always be blank when opened

STR:

1. Type some text in the histogram name text box (in any view), select a histogram
2. Open the histogram name box again

Expected: The histogram name text box should be empty and showing all selectable options
Actual: The currently selected histogram name is in the textbox and the suggestion (options) list is empty, text box needs to be cleared first

support ClientID

Now that telemetry includes ClientID, it should be supported in telemetry.js

http://robertovitillo.com/2014/12/19/clientid-in-telemetry-submissions/

Change "data missing" message

Change the "Data currently missing or unavailable" string to "No data matching these filters and settings". And suggest "Turn off data sanitization in Advanced Settings?" if sanitization is enabled

Label for enumerated histograms should be a single value, not a range

e.g. http://bit.ly/1HW9ZZe

Label should say "11", not "11 <= sample < 12"

Probe selector regularly switches to a different probe when changing parameters e.g. version pickers

IMO the probe name should never change on me as it's very annoying have to change it back and it's not always obvious that it's happening.

Remove "Select All" option from some filters

The product + arch + e10s + process type filters don't need a "select all" option. It just makes it more confusing to select or unselect items. They should be regular checklists

Don't use a modal progress overlay

A modal progress overlay (with the gear) is quite a bad UX (which the Pentaho dashboard also used to have) as it makes me suffer and wait while I may want to change parameters or after accidentally clicking the wrong probe name.

Security warning

on http://bit.ly/1KwpOYO , probably related to Disqus

"Password fields present on an insecure (http://) page. This is a security risk that allows user login credentials to be stolen.[Learn More]"

Modal (evo) progress indicator doesn't go away/notify upon fatal error

http://aggregates.telemetry.mozilla.org/aggregates_by/build_id/channels/nightly/?version=42&dates=20150813%2C20150811%2C20150810%2C20150809%2C20150808%2C20150807%2C20150806%2C20150805%2C20150804%2C20150803%2C20150802%2C20150801%2C20150731%2C20150730%2C20150729%2C20150728%2C20150727%2C20150726%2C20150725%2C20150724%2C20150723%2C20150722%2C20150721%2C20150720%2C20150719%2C20150718%2C20150717%2C20150716%2C20150715%2C20150714%2C20150713%2C20150712%2C20150711%2C20150710%2C20150709%2C20150708%2C20150707%2C20150706%2C20150705%2C20150704%2C20150703%2C20150702%2C20150701%2C20150630%2C20150629&metric=GHOST_WINDOWS&application=Firefox
HTTP/1.1 504 GATEWAY_TIMEOUT

uncaught exception: Could not obtain evolution: status 0 (http://aggregates.telemetry.mozilla.org/aggregates_by/build_id/channels/nightly/?version=42&dates=20150813%2C20150811%2C20150810%2C20150809%2C20150808%2C20150807%2C20150806%2C20150805%2C20150804%2C20150803%2C20150802%2C20150801%2C20150731%2C20150730%2C20150729%2C20150728%2C20150727%2C20150726%2C20150725%2C20150724%2C20150723%2C20150722%2C20150721%2C20150720%2C20150719%2C20150718%2C20150717%2C20150716%2C20150715%2C20150714%2C20150713%2C20150712%2C20150711%2C20150710%2C20150709%2C20150708%2C20150707%2C20150706%2C20150705%2C20150704%2C20150703%2C20150702%2C20150701%2C20150630%2C20150629&metric=GHOST_WINDOWS&application=Firefox)

The gear kept spinning indefinitely and wouldn't allow retrying, changing parameters or reloading the page (as the page doesn't seem to keep state and would reset to default values; filing a separate issue on this)

"Give Feedback" button should link to GitHub issue tracker

Caption should be changed to "Report bug", no one was complementing us anyways ;)

Histogram name box should not resize itself

Open histogram name, it's X units wide, paste the name of a histogram (e.g. GC_MINOR_REASON_LONG), box resizes itself so that it partially hides the pasted text.

Mouse-over legend for a histogram bar is clipped by the edge of the chart area

http://mzl.la/1fom6aV

"Settings (modified)" set if URL has a customization even when the Advanced Settings panel doesn't

Probe selector loses its filter upon re-opening

When I re-open the probe dropdown after having selected a new probe, the popup initially opens with the list pre-filtered or at least scrolled but then switches to a unfiltered view from the top. Ideally the filter the should be remembered since one normally wants to look at related probes e.g. all PWMGR_ ones

I think this may be a regression since I don't think this happened a month ago.

When an Advanced Setting option is not applicable disable it

When looking at GC_MINOR_REASON_LONG in Table View, I shouldn't be able to change "Don't/Trim buckets" options since they do nothing in that view

Web app names in Product list

Example: http://mzl.la/1fctBS2

Only the Firefox/Fennec/B2G/OTHER product names should be in that list.
Fix can be server-side or client-side.

Histogram table-view doesn't show 0 bucket

The 0 to 1 bucket is missing: http://bit.ly/1LilLSM

$checkbox is undefined bootstrap-multiselect.js:729:20 when aggregates are selected that don't exist for a probe

STR:

1. Load a probe that allows you to choose only Mean/Median/etc. (not just the submission count) e.g. PWMGR_NUM_SAVED_PASSWORDS.
2. Uncheck some of the checkboxes of data to see on the graph e.g. only leave submissions, mean and median. The url parameter updates to include &aggregates=Submissions!Mean!median
3. Switch to a probe that doesn't let you graph Mean/Median e.g. A11Y_CONSUMERS

Expected result:
The URL should update to indicate the selected probe name (e.g. A11Y_CONSUMERS) and the histogram should update.

Actual result:
A JS error occurs: TypeError: $checkbox is undefined bootstrap-multiselect.js:729:20
The URL doesn't update.
The histogram doesn't update (still shows the data from the last probe).

Showing the wrong histogram for a probe is a big problem if decisions are getting made based on the data.

Telemetry Dashboard no longer supports Thunderbird

The telemetry dashboard seems to no longer support viewing Thunderbird application details. It used to when Metrics ran it on their server.

AFAICT there's now no longer a way to view Thunderbird stats, so I can't even tell if they are being sent & received or not :-(

Layout bug in anthony-zhang.me v4 dash. Is t.m.o dash affected?

Example: http://bit.ly/1W2Wv96

Line color being re-used in Evo view

Same color being used for different versions: http://bit.ly/1Np1rSM

Better visualization for boolean histograms

True and False or 0 and 1 are better labels for a boolean hgram pie chart than 0 <= sample value < 1 and 1 <= sample value < 2

Histogram table view should show % column

No progress indicator is shown when switching probe names for the evolution view

Sometimes XHRs are very slow and there is no indicator showing it seems until the first request has some kind of progress? At that point "Updating evolution for nightly 42... 0%" shows up but that was about 15-20 seconds after I changed the dropdown.

Am I supposed to see keys in keyed evo view?

am i supposed to see keys here? http://bit.ly/1Np0dHe

Keyed histograms not scaled vertically properly

See http://bit.ly/1KwpR6U

telemetry.mozilla.org needs a favicon

:)

Mixed content in Histogram Dashboard prevents it from loading over HTTPS

If I load https://telemetry.mozilla.org/new-pipeline/dist.html (HTTPS URL), I see the following in the devtool console:

Blocked loading mixed active content "http://aggregates.telemetry.mozilla.org/aggregates_by/build_id/channels/"

This is breaking the page.

Visualization bug on histogram compare-by-OS-version

http://mzl.la/1TWCX2d

"Compare by" doesn't trim buckets?

The 0.00% buckets on the right side of this view aren't getting trimmed: http://bit.ly/1LikR8Q

V4 histogram dash shouldn't have "key sorting" option for unkeyed histograms

Keyed histogram view shouldn't duplicate graphs

If a keyed histogram only has say 2 keys, it should only show 2 graphs, not 4 graphs with each graph duplicated

http://mzl.la/1fokpdu

Boolean evolution graph should plot the proportion over time

Thanks for working on boolean and enumerated evolution!

Looking at PWMGR_SAVING_ENABLED (boolean) in the evolution view, the obvious thing I would want to see is the proportion of true vs. false over time. Instead I get a mean plotted with a consistent value of 1.01. How am I supposed to interpret this graph? The histogram for beta 40 shows as 95.79/4.21 split.

Selecting key "(none)" does nothing

clicking "select all" on an Enumerated Histogram Evo view hangs browser for a minute

e.g. click Select All on http://bit.ly/1HWaiU6

TypeError: selectedKeys is undefined evo.js:657:21

I see this regularly in my console and I suspect it's related to the many issues I'm seeing e.g. switching probes not updating the UI.

Boolean pie chart labels are inverted

For PWMGR_SAVING_ENABLED on Beta 40:
Start End Count
0 1 1.55M (4.21%)
1 2 35.24M (95.79%)

Where 0 is false as (one would normally expect) yet the pie graph tells the opposite story saying that 95.79% have password saving disabled.

Mouse-over label "blinks" (changes style?)

STR:

1. load an evo view
2. move the cursor along the curve
3. stop moving the cursor.

Notice how the label's text style changes subtly as soon as cursor stops moving