timwis / vizwit

An interactive data visualization tool

VizWit

An interactive data visualization tool. VizWit uses a JSON config file to generate interactive charts that cross-filter one another. It currently supports data hosted in Socrata and Carto, which includes open data provided by cities such as Philadelphia, Chicago, and San Francisco). Data that’s hosted elsewhere can be supported by adding a new data provider.

Features

• Clicking on a chart cross-filters the other charts
• Multiple datasets on one page
• Configurable via GUI (work in progress)
• Fork configurations with a gist
• Pie chart
• Callout metric
• Scrolling bar charts
• Responsive / mobile-friendly
• Embeddable (and cross-filtering still works)
• Free-text search on table
• 100% client-side
• Open source / extendable

Examples

• Business Licenses in Chicago (config)
• Philadelphia Parking Violations (config)
• Philadelphia Property Assessments (config)
• City-owned Properties in Philadelphia (config)
• Chicago Crime Incidents (config)
• Callout Cards (config)
• Philadelphia Crime Incidents (From Carto) (config)

Configuration

VizWit cards are configured via JSON objects. The layout is defined using size and order properties on those objects, generated by the Config Builder (work in progress). The resulting JSON file is then hosted on gist.github.com, a free and easy service to share code snippets. The Gist ID is then used in the VizWit URL (ie. vizw.it/?gist=a304ce8fac9c14dfbf16). This allows viewers to fork the Gist and build their own VizWit page.

Configuration Documentation

Build your own

Visit the builder and add a few cards, then configure them using the wrench icon. When you're finished, click Export and copy the configuration code into a new "gist" at gist.github.com. Once you click Create, you should see your new "gist" and the "gist id" of random characters in the URL (ie. 813483da72ac781f8b13). Use that gist id to go to http://vizw.it?gist=YOUR_GIST_ID

Alternatively, use the live editor to edit the raw JSON alongside a live representation of it.

If you'd like to share what you've created, post an issue with a link to it.

Technology

VizWit is a client-side JavaScript application. The application is structured using Backbone. Charts are built using AmCharts. Maps are built using Leaflet. Tables are built using DataTables. Tests (of which there are not nearly enough!) are run using Mocha. The interactive layout is generated by Gridstack. The application is compiled using Browserify. See the full list of dependencies in package.json

How it works

vizwit.js is the primary chart-generating module. It takes a container selector and a config object and uses them to initialize a view (bar chart, map, table, etc.). The views are passed a collection and a filteredCollection, which are identical and allow the view to query the data provider. Views generate the chart/map/table/etc. and listen for user interactions on them. On an interaction, the view triggers the global event object with the filter (ie. state=PA), and all views that use the same dataset receive the event and use their filteredCollection to query the data provider with the filter passed through the event. By using a separate collection, VizWit can show the "filtered value" and the "original value" side-by-side.

The actual entry point is either vizwit-loader.js or vizwit-embed.js. vizwit-loader.js fetches a gist or local file, reads its configuration, and passes it to layout.js to create a layout on the page. Then for each chart in the configuration, layout.js calls init() from vizwit.js as described above. vizwit-embed.js simply finds the parent container of the <script> tag that was embedded, and calls init() from vizwit.js inside of it. This way of embedding allows the VizWit library to only be loaded once on the page with as many charts embedded in any element (example).

Development

• After cloning the repo, use npm install to install dependencies
• While developing, use npm start to automatically recompile when changes are made and run a web server at localhost:8080
• Use npm test to verify code style and run unit tests
• Use npm run build for a production build, and then use npm run deploy to push the dist/ directory to the gh-pages branch (first make the deploy script executable via chmod +x deploy.sh)

License

GPL-2 (create an issue if that doesn't work for someone)

vizwit's Issues

Decide whether to hide grid lines

Pretty sure this is a Tufte thing but not sure the result looks that nice.

Before:

After:

To enable, add settings:

• chart.valueAxes[0].gridColor = '#fff'
• chart.valueAxes[0].gridAlpha = 0.6
• chart.gridAboveGraphs = true

Allow panels to use other aggregation functions

ie. sum() (instead of just count())

Leaflet map renders before template is visible

Despite appending the template to the el before initializing Leaflet, it seems Leaflet's container is not visible upon initialization. As a temporary workaround, I'm using setTimeout() to cause a slight delay. Obviously this is not a permanent solution.

Map should zoom to extent of boundaries

It currently zooms to Philly's lat lng with an hard-coded zoom

Add link options from config to header

ie. download buttons, links to other datsets, etc.
Should be configurable via config.json

Make chart labels more legible & not overlap

Hide table stats until total record count is known

Cross-filtering maps from other maps doesn't seem to colorize properly

Probably because it uses quantiles instead of equalIntervals

Decrease margins of panels

Or perhaps do away with the xenon theme all together?

Map should show currently selected polygon

Just as other charts do

Add date range chart

amcharts has a really nice one

Add GUI-based configuration

Dashing uses gridster. gridstack is a gridster inspired bootstrap responsive version. Here's how it serializes. Also, there's got to be some client-configurable bootstrap dashboard template/plugin out there. But how would we update the config file if the application is 100% client-side?

Table should infinite scroll

Depends on #1

Add total record count to table

Table should know how many records there are total. It should probably also need to know how the total with the filters applied. I imagine the collection should handle this, but not sure best practice (should there be a model or two on the collection? Or just an AJAX request? When should they kick off? etc.)

docs

Map clicks throwing inconsequential error

Uncaught TypeError: Cannot read property 'mouseEventToContainerPoint' of null on most clicks. Doesn't appear to impact functionality, but didn't start happening until today.

Improve usability of table on small screens

Github does this pretty well with tables rendered in readme files, ie on turf-count

Chart & map colors should be configurable via config.json

Legend/baseline can change when showing filtered amounts

If an "original amount" bar chart has a legend from 100k to 1M and it gets filtered by another chart, and one of its bars has an amount of 5, it will change its legend to 0 to 1M to accommodate the new, smaller filtered amount, which changes how all of the original bars appear.

This is even more evident in the choropleth chart, which colorizes based on the min/max of either the original collection or the filtered collection.

The question is whether this is okay, or if it's too confusing.

Use amCharts via NPM

Use field definitions in table

Columns should have human readable names tied to their field names. Panels should have descriptions. This could either be defined manually in a config file or fetched via a metadata API.

Add pie chart

this may come in handy (minimum size of slice)

Bar charts should indicate that not all columns are displayed

Currently it shows the first 10. It should indicate that there are others that aren't displayed, perhaps via an "other" bar.

Bar chart can change original values when being cross filtered

When plate_id on parking violations is filtered by another chart, the original bar amounts change

Add legend to map

https://www.mapbox.com/mapbox.js/example/v1.0.0/choropleth/

Cross-filtering should alter the viz data instead of re-rendering it

Probably more performant (especially w/map of census tracts); hopefully it's possible

idea:

init:
    collection.fetch()

on collection sync:
    render chart w/collection

on filtered:
    filterCollection.fetch()

on filterCollection sync:
    graph1 color = grey
    forEach graph1 column, find match in filterCollection, add value
    refresh chart

on click:
    highlight clicked column
    trigger filter

Cross filters should respect dataset id

If a chart representing dataset A was clicked, it should only crossfilter charts representing dataset A and not dataset B

Add tooltips to charts

Add download buttons to charts

To export grouped values. Should just be a matter of changing the endpoint format. Not sure how this example might be helpful.

Tutorial

Should maintain current filter, and work for table & map

Default zoom on bar charts should be the left side

Right now the workaround is calling a zoom method after the chart loads. Ideally there'd be a default zoom in the config of the chart. The current method appears not to work consistently on mobile.

Allow cards to have base filters

For example, if the underlying dataset contains 5 years of data but the visualization should only show one year, we should be able to set a base filter on every card/chart/map/etc. that sticks.

Add a way to clear a filter

For the datetime chart, this should probably make use of the label

Switch to amcharts

Looks like using "clustered": false inside the graphs objects puts the bars on top of each other.

Add a "clear filters" buttons

Or at least the ability to clear filters from the .filters panel.

Switch to config.json file

Currently configuring via HTML attributes. Switching to config.json configuration will require implementing templates.

Bar chart should show fewer bars on small screens

Use URL hash to switch dataset config files

ie. domain.com/#parking-violations uses config/parking-violations.json

Add accessibility plugin to amcharts

http://paypal.github.io/amcharts-accessibility-plugin/

Add loading indicator to each chart

Datetime chart has occasional error on chartCursor.addListener()

Occasionally I'll get an error about chartCursor.addListener not being a function on line 61

this.chart.chartCursor.addListener('selected', this.onClick);

Debugger suggests this.chart.chartCursor has not been initialized at the time of the error, and simply refers to the chartCursor definition passed to the AmCharts initialization method. The infrequency would suggest it's an async call not finishing quickly enough, but there don't appear to be any async calls that could be the culprit. Cannot reproduce so difficult to diagnose.

Give it a name

Improve chart horizontal scroll

Yes it works but it's a little confusing, slightly buggy, and doesn't seem to work on mobile

Perhaps use external buttons to navigate? example

Also, legend probably shouldn't reset as it's pretty confusing what's going on

Choropleth chart should listen to crossfilter

Add indication of currently filtered value on charts

ie. if you select a bar and it filters the other charts, the bar you selected should be highlighted. Same thing for date range.

Cleanup basechart

It may not even make sense to have a base chart. At the moment, it makes it harder to understand what's going on. It should probably be renamed "serial" since it's really only common for bar & datetime (not pie).

Map layer adding occasionally produces error

Cannot read property 'addLayer' of undefined on:

addTo: function (map) {
        map.addLayer(this);
        return this;
    },

This is probably occurring because the collection/boundaries finish syncing before the 100 ms timeout renders the map, so likely dependent on #24

Add indication of what filters are currently applied

Per @themightychris, either show on each chart or show a fixed bar across the top that lists the filters currently being applied.

Given the desire for these pages to support multiple datasets, the former suggestion may make more sense.

EDIT: Also, a "clear filters" button

Add distribution chart

Cross-filtering should filter itself too

If you click a bar on a chart, it should set all the other bars' filtered amounts to 0

Add choropleth map panel

Color of polygons should reflect the number of points residing within them. Not sure if we can use leaflet or turf.js for this or if we'll need to rely on server side processing with a high-volume dataset.

Idea: use collection.min() and .max() to get the floor & ceiling of the color options.

Clickable areas of chart should show pointer cursor