glue-viz / glue

Linked Data Visualizations Across Multiple Files

Glue

Glue is a python project to link visualizations of scientific datasets across many files.

This repository contains the glue-core package which includes much of the core functionality of glue that is used for the different front-ends, including the Qt-based application and the Jupyter-based application. Other key repositories include:

• glue-qt: the original Qt/desktop application for glue
• glue-jupyter: a Jupyter front-end for glue

In addition to these, there are a number of plugin packages available. For a full list of repositories, see https://github.com/glue-viz/.

Features

• Interactive, linked statistical graphics of multiple files.
• Support for many file formats including common image formats (jpg, tiff, png), ascii tables, astronomical image and table formats (fits, vot, ipac), and HDF5. Custom data loaders can also be easily added.
• Highly scriptable and extendable.

Installation

For installation documentation, visit glueviz.org.

Contributing

If you are interested in contributing to glue, please read our Code of Conduct and Contribution Guidelines.

Support

Please report problems to [email protected], or open an issue.

License

Glue is licensed under the BSD License.

glue's Issues

get icons in mac app

Define subsets via text commands

Let users do the equivalent of

subset = (subset1 & subset 2) || (ra > 5 and ra < 6 and dec > 10 and dec < 20)

Integrate into GUI

Window layout for QT MDI

The Multiple Document Interface widget in QT stores multiple moveable, resizable widgets on a canvas. It provides a tile method in an attempt to move/resize each window so they use all available space and fill the screen. However, it is buggy (it moves the entire Glue Application partially off-screen).

Perhaps we should implement our own window layout method to address the "screen real estate" problem

Enhanced interchangeability of clients

I think the current interface for the data class is too restrictive for our future goals. The simple list of regularly-gridded, in-memory numpy arrays for each component will prove problematic for, e.g., larger-than-memory data and variable resolution data sets (perhaps backed by YT data objects).

This poses a problem for viz clients, which use the current Data interface pretty heavily (i.e. they directly access the numpy component arrays). As it stands, if we were to build support for the above data types, we'd have to either

1. Force these data to conform to the current interface (awkward, if not impossible), or
2. Re-build all the desired viz clients for each new data class (time consuming, and difficult to maintain as we improve upon the clients)

As an alternative to this, I propose we create a higher-level API for the data class, so that viz clients can extract the information they need without knowing the low-level representation of the data. In other words, instead of an image client extracting a component numpy array directly, it would call a method like data.extract_image(image_properties). The various data types each have their own class, and override these methods.

I'm getting this idea by looking over how yt works -- yt has general methods to, e.g., created fixed resolution images from various simulations for generic visualization. If we adopt a similar approach, we can guarantee that our viz clients will continue to work with whatever data classes we dream up.

There are still downsides to this approach -- the current clients would need to be updated, of course. Also, the subsets would need some thought (how does the ElementSubset work?)

Thoughts? I can spend some time thinking about how this might work specifically, but wanted to test the water first. Maybe I'm missing something. If we go down this road, I think it would be a good idea to study yt's API, since I bet we could borrow heavily from their interface.

script for building glue app

alphabetize long component lists

Component lists (in scatter widget, image widget, component link widget) are given in random order. This is confusing for big catalogs with many columns. Alphabetize them

subset saving

Subsets should be saveable. They should be readable by Glue, and easily interpreted for use in other environments.

This functionality should be connected to the GUI

Give component link widget a title

the component link widget's title is 'form'. Change it to something meaningful

Contrast mouse mode doesn't deactivate

When activating the contrast mouse mode and then the pan/zoom mouse mode in the image widget, the contrast mode stays active and responds to mouse gestures. It shouldn't

Painless data input

Glue should read data easily. Currently, Glue punts if it hits an error when parsing with atpy. Many times, these errors can be fixed by tweaking which catalog format to expect, what denotes a comment line, etc.

There should be a more fleshed out "data load wizard" that walks the user through loading data, letting him fiddle with these options until the data can be read.

When loading images, Glue currently punts if hit hits unexpected astrometric information. It should deal with this more gracefully. Probably, this means skipping the step where it creates world coordinates.

How should namespace be flattened?

Option for displaying world coordinates in image client

Maybe this involves coordinate grids with AplPy? Or maybe we display the world coordinates of the mouse in the Qt Information window?

Integrating RasterAxes into subset client

RasterAxes are needed for interactive plotting once a catalog grows to more than ~1000 entries.

Normal axes provide more functionality at the moment, so perhaps they should be retained for small catalogs. Can the code swap out axes when needed?

subset updates (or at least their messages) are being broadcast across data sets

Better default colors/sizes

Default plot size for edit subsets should be bigger, for emphasis. Default colors could be made more intelligent, so that similar colors are less likely to overplot

Data/Subset linking

We need to better think about how to link data and subsets when we create subsets. Also, we need to think about circular references a bit harder in general.

Non-trivial and custom link functions in link dialog

We need a way to specify other link logic besides 1:1 in the link_dialog. The code can already handle this -- we just need a way of plugging in other options to the dialog

Reorganize code

Goals include

• More thoughtful of what gets imported in init.py files
• Better partition of code into directories / submodules
• Separation of glue core, astroglue, qtglue, etc.

document how to build and/or install

discuss raster axes and other speedup ideas

qt clients should unsubscribe when windows are closed

Image client doesn't show subsets on init

The image client doesn't render pre-existing subsets when first initialized. It should

Create glueqt command line binary

Should basically do what sandbox/glueapp.py does

Documentation

For .astronomy, we should have at a minimum:

• A screencast walking through a particular use case
• A tutorial in sphinx to do the same
• A minimal cookbook, showing how to add functionality (details depend on what we want to hack on during .astronomy)

Settle on best way to communicate data linking

How should selections be applied across data? Automatically? Manually as done now? Should their be a visual indication of which data sets are linked? Should the visual properties of linked data sets sync up?

In other words, what is the most intuitive way to do cross-data linking?

Mouse mode for bias/contrast mode

The imagewidget uses right mouse gestures to adjust image bias/contrast, a la ds9. However, this conflicts with other mouse modes (namely the pan/zoom mode). The bias/contrast gestures should be moved to a mouse mode, so these gestures don't conflict

Scatter client crashes with non 1D data

Scatter client crashes when trying to plot components which aren't one dimensional.

Traceback with a 2D data set:

Traceback (most recent call last):
  File "/Users/beaumont/Library/Python/2.7/lib/python/site-packages/glue/qt/scatterwidget.py", line 116, in update_xatt
    self.client.set_xdata(component_id)
  File "/Users/beaumont/Library/Python/2.7/lib/python/site-packages/glue/scatter_client.py", line 290, in set_xdata
    self.set_xydata('x', attribute, snap=snap)
  File "/Users/beaumont/Library/Python/2.7/lib/python/site-packages/glue/scatter_client.py", line 254, in set_xydata
    map(self._update_layer, (l for l in self.managers))
  File "/Users/beaumont/Library/Python/2.7/lib/python/site-packages/glue/scatter_client.py", line 432, in _update_layer
    self.managers[layer].set_data(x, y)
  File "/Users/beaumont/Library/Python/2.7/lib/python/site-packages/glue/scatter_client.py", line 59, in set_data
    xy[:, 0] = x
ValueError: operands could not be broadcast together with shapes (13911267) (27767,501) 

Image client defines ROIs in pixel space

ROIs currently defined in coordinate space. This occasionally leads to unexpected / buggy behavior due to the spherical geometry (for example, ROI's which straddle the l=360 line are incorrect). Pixel coordinates are euclidian, and thus the ROIs should always behave sensibly

Should tests be refactored to better use py.test?

General polishing of Image Client

Play with the image client using a couple of different data sets. Walk through a few use cases. Where are the rough edges? File issues and address the most important

General polishing of Scatter Client

Play with the scatter client using a couple of different data sets. Walk through a few use cases. Where are the rough edges? File issues and address the most important

remove subsets not processed by histogram

scatter client -- edit subset on top

The editable subsets should be drawn on top of other subsets, so that they can always be seen

scatter client -- remove layers upon deletion

The scatter widget's layer manager doesn't remove data sets when they are deleted -- this is probably intrinsic to the layer manager widget

layer tree columns should be hide-able

the histogram client doesn't use point size or symbol, for example. It would be better to disable these columns for that widget

Plan for possible .astronomy hack session

What do we want to do if we get time for a .astronomy hack session? User studies? Implementing new features?

config.py for mac app

Add open data action to file menu

Should be able to navigate to file-> open data to load new data

link manager should delete links when data are deleted

Distribution for .astronomy

What is the easiest and fastest way to get Glue to install on new machines for .astronomy?

Display old links in link dialog

The link dialog widget only shows the links defined in the current invokation of the widget. It should display all links, to let the user un-glue old links

Good use cases for documentation?

Let's come up with a couple of nice use-cases that show the power behind Glue's cross-data linking. We can build tutorials around them

How to deal with circular imports?

Currently, there are several circular dependencies in the backend. One example:

• Hub contains a list of clients. Each client contains a hub.
• When someone attempts to add a client to the hub via Hub.add_client, the Hub code checks whether the input is of type Client.
• When someone creates a new Client object, they provide a Hub reference. The client code checks wither that is actually a Hub.

I attempted to add "from client import Client" to hub.py, and "from hub import Hub" to client.py. Python doesn't like this.

What's the best way to deal with this? Put the base classes for the backend (Hub, Data, Subset, Client) in a single module?

Integrate new LinkManager object into DataCollection

Should use LinkManager to keep virtual / derived components in sync. Gluing/ungluing should interface with the LM.

component_link passed an int for using

after loading irdcs.fits

Traceback (most recent call last):
File "/Users/beaumont/Library/Python/2.7/lib/python/site-packages/glue/qt/widgets/layer_tree_widget.py", line 346, in
lambda: LinkEditor.update_links(self._data_collection))
File "/Users/beaumont/Library/Python/2.7/lib/python/site-packages/glue/qt/link_editor.py", line 59, in update_links
widget = cls(collection)
File "/Users/beaumont/Library/Python/2.7/lib/python/site-packages/glue/qt/link_editor.py", line 17, in init
self._init_widgets()
File "/Users/beaumont/Library/Python/2.7/lib/python/site-packages/glue/qt/link_editor.py", line 25, in _init_widgets
self._add_link(link)
File "/Users/beaumont/Library/Python/2.7/lib/python/site-packages/glue/qt/link_editor.py", line 34, in _add_link
item = QListWidgetItem(str(link))
File "/Users/beaumont/Library/Python/2.7/lib/python/site-packages/glue/core/component_link.py", line 97, in str
result = "%s <- %s(%s)" % (self._to, self._using.name, args)
AttributeError: 'int' object has no attribute 'name'

Installation troubles

Hi!

pyqt causes trouble when I do a dumb pip install - some advice for users on the docs would be great!

Thanks,

Phil

subsets need to be synchronized upon registration to hub

If a subset is registered to a data set before a client is registered to a hub, the client misses that message. Upon registration to the hub, it should make sure it knows about all the subsets in the data.

Flesh out GUI

Iterate once more on GUI design, fleshing out some details (like the specific size of elements, menu navigation)

subsets remain in data.subsets after removing via layer tree

Scatter client doesn't snap axes when data are first loaded