Comments (4)
create_graph_json.py
handles existing metadata by adding it to the datapoint, e.g.
Meta | Datapoint |
---|---|
Iteration 1:CheckpointFoo | blah/explicit/js |
becomes Iteration 1/CheckpointFoo/blah/explicit/js
The reason these fields aren't just baked into the datapoint by the test to begin with is the silly string de-duplication stuff. We don't want to store each reporter name iterations * checkpoints
times per run in sqlite. A database with proper transparent compression would solve this, so we could just prefix whatever onto the reporter name and it would work.
Also, the website is hardcoded to mark certain 'depths' of the tree as special in a few spots, but these are easy to increment if we add another level for process:
https://github.com/mozilla/areweslimyet/blob/master/html/slimyet.js#L820
https://github.com/mozilla/areweslimyet/blob/master/html/slimyet.js#L821
https://github.com/mozilla/areweslimyet/blob/master/html/slimyet.js#L928
from areweslimyet.
For the record, if you change the JSON format, the procedure to re-export the old databases is roughly:
# Run this over a weekend, it wont be quick, these DBs are huge and sqlite slow
for compressedMonthDB in db/areweslimyet-*.sqlite.xz; do
monthDB="${compressedMonthDB%.xz}"
seriesName="$(basename "${monthDB%.sqlite}")"
monthJSON="html/data/$seriesName.json.gz"
util/unarchive_db.sh "$compressedMonthDB"
util/update_db.py "$monthDB"
mv -v "$monthJSON" "$monthJSON.oldFormat"
./create_graph_json.py "$monthDB" "$seriesName" html/data/
util/archive_db.sh "$monthDB"
done
(do this one at a time with a script because having them all uncompressed at once will fill the disk)
from areweslimyet.
As of ae88a34 create_graph_json.py
handles the new DB format but doesn't actually use anything but the main process. The next step is to update the script to use the process names (and annotate kinds).
from areweslimyet.
This landed in a11f3ec on the e10s branch.
from areweslimyet.
Related Issues (20)
- Using non-existent proxy no longer works, breaks AWSY HOT 2
- Handle ftp.mozilla.org migration
- Update to new mozdownload API
- Properly normalize process names HOT 1
- Try builds are no longer working HOT 1
- FTP builds are no longer working
- Support posting perf data to both prod and stage HOT 1
- Attach logs to perfherder data HOT 1
- Track source repo HOT 1
- Support processing minidumps HOT 1
- Fix about:memory exporter HOT 2
- Add a test that opens and closes browser windows (as opposed to tabs) HOT 1
- Make AWSY scroll during testing, or at least add some tests that scroll HOT 1
- Just display the past year's results by default HOT 3
- Handle rename of the marionette-client package to marionette-harness
- archive.m.o stopped updating, no tests run since 1/18/2017
- Use pyup.io to keep Python dependencies up to date
- Update treeherder-client from 3.0.0 to 4.0.0
- Auto-populate the first value(s) for a custom series
- How come ? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from areweslimyet.