mwater / mwater-visualization

mwater-visualization

Requires C3, D3, Bootstrap and Font Awesome

Also requires that server has hexgrid functions (src/maps/Hexgrid Functions.sql)

Expressions

Expressions

Principles

• In design, things must be valid or null. Design can be incomplete but not invalid
• After any change design is cleaned before being stored
• Stored design is always cleaned before being edited or displayed

Quick start

Run npm run watch to start the demo

mwater-visualization's Issues

Finalize schema

• Include baseTableId in expressions?
• Use tableId, joinIds, etc. or table, joins?

Implement data point clicking for LayeredChartView

PHP server for JsonQL

Create a PHP server page that produces SQL from JsonQL. Bonus marks for having it run the resulting SQL on a PostgreSQL connection and return either JSON results or an intelligible error code.

• Inputs should be the query as JsonQL string + a file on disk that describes the schema. Schema is a list of tables, each with an id and table name. Each also contains a list of columns which are id and sql expression to replace with. This is the whitelist of what to allow queries on. e.g.

[{ id: "table1", sql: "table1", columns: [
  { id: "col1", sql: "{alias}.col1" },
  { id: "col2", sql: "({alias}.col3 + {alias}.col4)" },
  ...
]},
...
]

{alias} is replaced with the current table alias. See SchemaMap.coffee

• Node code should compile to SQL + parameters, returning both if PHP easily supports parameterized queries, otherwise an SQL string with it all embedded.
• Check escaping and length limits, so that we don't end up crashing or allowing arbitrary code execution on the server

Cannot resize pie charts

c3js/c3#1205 I believe.

Group by aesthetics should have simple filter (multi-select box)

Display scope as bar at top of dashboard, not in charts

Add primary key to schema table definition

Options:

• as column with id type
• as property of table (since order, name, type? don't matter and can't have zero or many)

Don't autofill RHS of inequality expressions

Check that extra files are not included in distribution

Browserify seems to pull in React for some reason and possibly lodash.

Dragging resize handle sometimes moves instead

For Annie on Chrome, cannot reproduce locally

Dragging and resizing very slow

Recreating chart each time?

Hide legend if one series

Add line chart

Figure out radius mapping

How can maps like safe water access be done properly in a single tile call?

Add printing functionality

Switching from integer to enum type left < as operator (hidden)

Throttle queries to prevent overquerying when changing a filter

Add text widget

Add pie chart

Count(_id) has wrong summary and type

Fix remaining missing React keys

Open and scroll to currently selected tree item in ScalarExprEditor

Right now it can be offscreen and/or collapsed

JsonQL query does not appear to be sorting

Null color axis values make for ugly layer0:null:x type display

Add support for count(*)

• Remove id type

Two problems to solve:

1. scalar query with joins and aggregate should have no expression (undefined) and aggr set to count. This is to return a scalar of type integer

2. entire aggregate aesthetic has aggr of count and no expression. This is to do a count(*) as part of a grouping

Can 2) have a type scalar, but with undefined expression? i.e. a) no expression at all (not even a scalar, just undefined) or b) scalar with undefined expr

Add x axis and y axis labels

Dashboard with filter on bar then pie has bar chart all grey

Due to using complete row as data of scope. Should use subset.

Add undo/redo stack

Add to designer at top

Pop open scalar expression editor immediately on add filter

Data Source dropdown select cursor incorrect

Appearing as a bar, not a hand as one would expect. I've seen this in other places as well.

Scatter chart has odd results

Check query to make sure correct.

Investigate printing dashboards and widgets

Printing is done through window.print(). Allow time first for widget to render.

Put widget to render under body in print_xyz div, apply stylesheet:

@media print {
   body {
      visibility: hidden; /* Necessary? */
      margin: 0;
      padding: 0;
    }

   body > * {
    display: none;
   }

   #print_xyz {
     display: block !important;
     visibility: visible;
   }

 }
 #print_xyz {
  display: none;
 }

@page  {
size: 8.5in 11in; 
margin: 0.5in 0.5in 0.5in 0.5in; 
}

Page breaks depend on media. If letter with 0.5in margins, contents are 7.5in by 10in.

So, render with width of minimum 96dpi (7.5*96=720). Page will break at 960px down.

Page break items need to be non-absolute positioned:

.mwater-visualization-page-break {
    width: 100%;
    border-bottom: dashed 1px #AAA;
}

@media print {
    .mwater-visualization-page-break {
        border-bottom: none;
        page-break-after: always;
    }
}

When rendering page breaks in screen, adjust to scale down. If control is 1000px wide, put page break at 1000/7.5*10 px down on screen.

Initially all tree sections should be closed

Address react warnings in browser console

libs.js:41864 Warning: Failed propType: Required prop `width` was not specified in `DashboardViewComponent`.
libs.js:41864 Warning: Failed propType: Required prop `width` was not specified in `ChartWidgetComponent`. Check the render method of `DashboardViewComponent`.
libs.js:41864 Warning: Failed propType: Required prop `height` was not specified in `ChartWidgetComponent`. Check the render method of `DashboardViewComponent`.
libs.js:41864 Warning: Each child in an array or iterator should have a unique "key" prop. Check the render method of DashboardDesignerComponent. See https://fb.me/react-warning-keys for more information.
libs.js:41864 Warning: Each child in an array or iterator should have a unique "key" prop. Check the render method of DashboardViewComponent. See https://fb.me/react-warning-keys for more information.
libs.js:41864 Warning: Each child in an array or iterator should have a unique "key" prop. Check the render method of LayeredChartDesignerComponent. See https://fb.me/react-warning-keys for more information.
libs.js:41864 Warning: Each child in an array or iterator should have a unique "key" prop. Check the render method of LayerDesignerComponent. See https://fb.me/react-warning-keys for more information.
libs.js:41864 Warning: Failed propType: Invalid prop `frame` supplied to `ScalarExprTreeTreeComponent`, expected a ReactElement. Check the render method of `ScalarExprTreeComponent`.
libs.js:41864 Warning: Each child in an array or iterator should have a unique "key" prop. Check the render method of ScalarExprTreeTreeComponent. See https://fb.me/react-warning-keys for more information.

Add table widget

Test date rendering, adding binning by year and year/month

Is binning something that's done at expression level? i.e. as part of xExpr of design? Or is it a statistical x-form afterwards (xBin with options of year, year/month, year/month/date for dates and # of bins for decimal/integer)?

We need to build in binning as post-xform for decimal anyway. It would be easiest to do in two phases: get max/min/stdev of the x expression, as filtered. Then determine bins and group by case expression and then map then to known strings.

It makes x categorical if binned and right now, categories are not assembled in strict order when multiple layers are present.

In fact, how can binning be done with multiple layers, as each might have their own ranges? I think we could disallow multiple layers with decimal binning in x-axis?

Can't click on unprotected dug well

in:

{"items":{"fc8d82bc-c485-4bc7-bc6d-b6c351f33813":{"layout":{"x":0,"y":0,"w":12,"h":12},"widget":{"type":"LayeredChart","version":"0.0.0","design":{"type":"pie","layers":[{"xExpr":null,"yExpr":{"type":"scalar","table":"entities.water_point","joins":[],"expr":null},"colorExpr":{"type":"scalar","table":"entities.water_point","joins":[],"expr":{"type":"field","table":"entities.water_point","column":"type"}},"filter":null,"table":"entities.water_point","yAggr":"count"}],"titleText":"Water Points by Type"}}},"34e1f95a-30f5-4d63-b90f-bd3d310db850":{"layout":{"x":12,"y":0,"w":12,"h":12},"widget":{"type":"LayeredChart","version":"0.0.0","design":{"type":"bar","layers":[{"xExpr":{"type":"scalar","table":"entities.water_point","joins":["source_notes"],"expr":{"type":"field","table":"source_notes","column":"status"},"aggr":"last"},"yExpr":{"type":"scalar","table":"entities.water_point","joins":[],"expr":null},"colorExpr":null,"filter":null,"table":"entities.water_point","yAggr":"count"}],"titleText":"Water Points by Status","transpose":true}}}}}

Add overrides for type of each layer

Each layer can be a different type. Just need to expose in UI

Vertical axis duplicated in 0.4.11rc

Bug in rendering of a vertical axis:

@daxfohl can you have a look? We can't proceed with the upgrade if we can't resolve this bug and will have to draw the title manually in svg.

Number of should not display for aggregate aesthetic

Make non-editable view of charts

When designer is not open, no dragging, moving, selecting or removing.

Show ghost of last valid design and data when invalid design

If adding filter, greyed out version of last valid one should remain as display

Dragging charts is slow

I've been dragging widgets around and notice that if you drag and then move your mouse in circles or whatever real fast for a while, the border can't keep up and just keeps going around that same path, trailing further and further behind. Sometimes takes a good 15 sec before finally settling into wherever you ended up.

This seems to only occur with chart widgets.

Show localized enum values in bar chart, not raw values

Add export to svg file functionality

Easy filters on LHS

Without popup, choose:

• Water type
• Group
• Latest E.Coli risk
• Latest Functionality

Tabs on design pane to clean up?

Basics:

Indicator? Important to use the word.
Basic filters
Download as CSV

Advanced tab: filter list
Customize/Appearance: color, etc.

Handle null x axis for bar/line charts

Should be possible to have no x axis? Use ""? null (current) means that chart doesn't display

Missing filter control

Add date literal and ops