samreay / chainconsumer Goto Github PK

Corner plots, LaTeX tables and plotting walks.

Home Page: https://samreay.github.io/ChainConsumer

License: MIT License

Python 98.63% TeX 0.94% Makefile 0.43%

chainconsumer's Introduction

ChainConsumer

A library to consume your fitting chains! Produce likelihood surfaces, plot your walks to check convergence, output a LaTeX table of the marginalised parameter distributions with uncertainties and significant figures all done for you, or throw in a bunch of chains from different models and perform some model selection!

Click through to the online documentation

Installation

Install via pip:

pip install chainconsumer

Python Versions

Time has ticked on, and now only python 3.10 will be supported. This is because type hints are amazing.

Developing

Clone repo
Run make install
Ensure that you set your python interpreter to the .venv/bin/python
Code away.

Contributors

I would like to thank the following people for their contribution in issues, algorithms and code snippets which have helped improve ChainConsumer:

Simon Dupourqué (for his PRs and suggestions to make ChainConsumer better for all)
Chris Davis (check out https://github.com/cpadavis/preliminize)
Joe Zuntz
Scott Dedelson
Elizabeth Krause
David Parkinson
Caitlin Adams
Tom McClintock
Steven Murray
J. Michael Burgess
Matthew Kirby
Michael Troxel
Eduardo Rozo
Warren Morningstar

Common Issues

Users on some Linux platforms have reported issues rendering plots using ChainConsumer. The common error states that dvipng: not found, and as per StackOverflow post, it can be solved by explicitly install the matplotlib dependency dvipng via sudo apt-get install dvipng.

If you are running on HPC or clusters where you can't install things yourself, users may run into issues where LaTeX or other optional dependencies aren't installed. In this case, set usetex=False in configure to request matplotlib not try to use TeX. If this does not work, also set serif=False, which has helped some uses.

Update History

1.1.1

Adding general kwargs to add_marker so one can pass show_label_in_legend and other customisations through.

1.1.0

Updating numpyro and arviz translators so you can specify variable names (var_names) to include or exclude.
Adding histogram_relative_height property to chains to control histogram height.
Adding show_label_in_legend so you can decide to give a chain a label but not have it pollute the legend.

1.0.3

Bug fix not using labels instead of column names for parameter title summaries.

1.0.2

Bug fix for chain divisions.

1.0.1

Bug fix for the truth labels.

1.0.0

Huge update to make full use of type hints, pydantic v2, and mkdocs

0.34.0

Making usetex=False and serif=False the defaults to reduce LaTeX errors.

0.33.0

Adding extra padding to bin extents for KDE and smoothing
Updating watermarking to work with matplotlib v3.0.0+

0.32.0

Fixing matplotlib axis formatter issue.

0.31.2, 0.31.3

Conda-forge updates

0.31.1

Adding ability to display plot as a prior (1D only, no 2D)

0.31.0

Linking colorbar label font size to global label font size option. Thanks Yucheng-Zhang!
Allowing chains to be passed in as a pandas DataFrame.
Statsmodel update means we are now switching to Python 3 only support.

0.30.1

Updating matplotlib dependency version for conda install. Thanks He Jia!

0.30.0

Bug fix for specifying numeric loc to legend_kwargs
Added shift_params when adding chains.

0.29.1

Potential bug fix for log_space feature.

0.29.0

Warning the user if configure is called multiple times.
Allowing parameters to be a number when calling get_latex_table
Adding log scales when plotting.
Adding the ability to plot a contour on an arbitrary axis via new method plot_contour

0.28.0

Removing rainbow option and replacing with cmap so you can specify the cmap used, not just rainbow.
Adding zorder configuration option to epxlicitly order contours.
Adding extra checks to try and catch bad chains on load.

0.27.0

Now restores default rcParams for usetex and font-family after plotting.
All logging now under logger name chainconsumer to make it easy to hide if needed.
Formula for computing shade_alpha now uses sqrt(num_chains) instead of num_chains.
get_latex_table now accepts a filename input to save the parameters to.
Adding add_covariance to compliment add_chain - useful for Fisher matrix forecasts and similar. Just invert it first for me.
Adding add_marker to allow easy inclusion of markers in the plots.

0.26.3

Adding ability to turn off chain names in plot_summary.

0.26.2

Fixing bug with plot_walks that required truth values.
Fixing flaw in configure to allow for updating values.
Fixing bug where summary values are cached without reference to the summary statistic method.

0.26.1

Adding ability to plot maximum points on 2D contour, not just global posterior maximum.
Fixing truth dictionary mutation on plot_walks

0.26.0

Adding ability to pass in a power to raise the surface to for each chain.
Adding methods to retrieve the maximum posterior point: Analysis.get_max_posteriors
Adding ability to plot maximum posterior points. Can control marker_size, marker_style, marker_alpha, and whether to plot contours, points or both.
Finishing migration of configuration options you can specify when adding chains rather than configuring all chains with configure.

0.25.2

(Attempting to) enable fully automated releases to Github, PyPI, Zenodo and conda.

0.25.1

(Attempting to) enable fully automated releases to Github, PyPI, Zenodo and conda.

0.25.0

Changing default sigma2d to False. May chance how your plots are displayed.
Allowing format specification when adding chains.
Making yule_walker (and thus all of statsmodels) a conditional import.
Updating minimum version of requirements to reduce issues with install.

0.24.3

Fixing bug in label rendering for contour sigma labels.
Improving parsing of sigma in configure, such that you don't need a leading zero.

0.24.2

Fixing bug in get_correlations.

0.24.1

Changing default colour order.
Improving behaviour of shade_gradient.

0.24.0

Refactoring project structure.
Updating colours for better legibility.
Setting shade=True automatically if shade_alpha is overriden.

0.23.2

Removing bbox_inches="tight" due to a bug in matplotlib v2.1.0.
Adding more colour shortcuts.

0.23.1

Making rainbow colours slightly more visible by darkening the yellow regions.

0.23.0

Can now pass a list of filenames to save out, to make generating a PNG and PDF option in one go easier
Adding method plot_summary

0.22.0

Adding option to specify the confidence interval (area) for parameter summaries.
Adding three extra methods for parameter summaries from Andrae 2010: max symmetric, max shortest and max central stats.

0.21.7

Fixing a bug that caused ChainConsumer to crash in some cases when you specified a number of parameters.

0.21.6

Fixing bug that made parameter ordering incorrect in some circumstances.

0.21.5

Fixing error when plotting walks with small weights.

0.21.4

Fixing issue where refactoring broke parameter blinding.

0.21.3

ChainConsumer now only finds extents of relevant parameters when plotting, instead of all parameters.

0.21.2

Updating extents so previous updates do something.

0.21.1

Adding example and code to deal with non-TeX watermarks.

0.21.0

Increasing extents again.
Updating legend defaults.
Code refactor.
Can now specify which chains to plot when plotting contours.
Adding watermark text.

0.20.0

Increase control over legend with kwargs.
Can specify legend subplot location.
Increased legend options with coloured text.
Added shade_gradient option.
Increase the amount of default extent given.

0.19.4

Adding ability to blind parameters.

0.19.3

Adding ability to plot contour levels, either in confidence levels or sigma.
Changed shading defaults.

0.19.2

Legend gets placed in top right corner now when plot_hsits is False and there are only two parameters.

0.19.1

sigma2d correctly defaults to True now.

0.19.0

Adding log_weights to the detected colour parameters.
Contours now support 1D Gaussian levels and 2D Gaussian levels (thanks @matthewkirby).

0.18.0

Adding Matched Elliptical Gaussian Kernel Density Estimator to replace statsmodels KDE.

0.17.4

Fixing bug in covariance calculation when getting the LaTeX table (did not affect contours)

0.17.3

Default figure size is now 1.5 inches per parameter, instead of 1. Also decreasing default font size, so that printing summaries is less likely to overlap surfaces.

0.17.2

Label font size now applies to legend.

0.17.1

Code quality improvements
Documentation update

0.17.0

Refactoring ChainConsumer due to growing size.
Improve bin limits to reduce overly large bins that form when some low-weight samples are located far away from the mean.
Fixed issue generating text with one sided distributions.
Adding ability to specify weights or posterior as the colour parameter.
Color scatter with uniform weights doesn't have first plot a different color.
Adding ability to control subplot spacing.
Adding method plot_distributions to quickly plot marginalised distributions.

0.16.5

Fixing bug in Gelman-Rubin diagnostic. Thanks Warren!

0.16.4

Moving rc parameters before plot creation to fix issues with parallel plot generation.

0.16.3

Fixing an integer division bug where python 2 contour shading was setting to 0 alpha.

0.16.2

Fixing bug where tick font size was only honoured when ticks were on an angle.

0.16.1

Adding ability to specify label font size, tick font size, and whether the ticks should be on an angle.

0.16.0

Bug fix for those with latest numpy which removed a deprecated method I was using.
Adding ability to get parameter covariance tables.

0.15.7

Adding ability to get parameter correlation tables.

0.15.6

Removing unnecessary debug output.

0.15.5

Can remove lists of chains properly now.

0.15.4

Adding ability to remove chains.

0.15.3

Adding ability to plot the walks of multiple chains together.

0.15.2

Removing unnecessary debug output.

0.15.1

Bugfix for python 2.7

0.15.0

Adding usetex to configure method.
When plotting walks, plots weights in log space if the mean weight is less than 0.1
Adding AIC
Adding BIC
Adding DIC
Adding method to output model comparison table.

0.14.0

Adding coloured scatter.
Disallowing grid data and KDE.
Adding more examples.
Consolidating all configures into one method.
Improved extent finding.
Updating smoothing to use reflect and not constant.
Improving max statistics being able to find ranges on cliff edges.
Printing parameter summaries without parameter labels.

0.13.3

Removing ability to having vectorised dictionary inputs for grid data due to 2.7 compatibility issues.

0.13.2

Fixing bug when smoothing grid data.
Adding more input options.
Grids can now be specified using a list of parameter vectors.

0.13.1

Better determination of extents for data with extreme weighting.
Able to scale figure size using float when plotting.

0.13.0

Modifying API defaults for smoothing with grid data.
Allowing both smoothing and bins to be passed in as lists.

0.12.0

Adding support for grid data.

0.11.3

Fixing bug in Gelman-Rubin statistic

0.11.2

Improving text labels again.

0.11.1

Improving text labels for high value data.

0.11.0

Adding Gelman-Rubin and Geweke diagnostic methods.

0.10.2

Adding options for alternate statistics.

0.10.1

Updating setup so that dependencies are automatically installed.

0.10.0

Modifying the add_chain API, so that you can pass dictionaries!

0.9.10

Smarter extent tick labels and offsets.
Adding list based line styles, widths, shading and opacity.
Adding two more examples.

0.9.9

Preconfiguring logging.

0.9.8

Adding 2D Gaussian smoothing for the contour surfaces.
Renaming contourf and contourf_alpha to shade and shade_alpha.
Updating some of the example plots.

0.9.7

Updating package setup scripts.

0.9.6

Updating package setup scripts.

0.9.5

Adding markdown paper.

0.9.4

Updating setup and package details

0.9.3

Initial zenodo release

0.9.2

Adding in smoothing, making it default
Adding extra example to show how to remove smoothing.

0.9.1

Adding in tests

0.9.0

Initial PyPi push

chainconsumer's People

Contributors

Stargazers

Watchers

Forkers

inonchiu xyh-cosmo itsnikka chto xuod caitlinadams olivares-j cmurray-astro bvgoncharov renlliang3 jiaming1999 wushichao abinashkumarshaw mr-superonion nicoborghi snwnde asantini29

chainconsumer's Issues

diagnostic_gelman_rubin compted incorrectly

I've been playing around with ChainConsumer to examine mcmc data. Its a great utility for visualizing, and processing chains, but I have noticed two issues in the Gelman Rubin diagnostic function.

The first is that when calculating the within sequence variance (W) you should take np.std()**2 to get the variance rather than the standard deviation. The variance is what Gelman & Rubin (1992: https://projecteuclid.org/euclid.ss/1177011136), use to calculate W. This distinction is important because unless your variance is very close to 1, you will either overestimate or underestimate W, which will cause you to either oversample or undersample your PDF (because the ratio B/W that is most important in determining convergence will be systematically overestimated or underestimated). This gets really bad for chains which have a small sample variance (of order 0.001); such chains can appear to converge in as few as 110 MCMC steps, when they actually need ~10000.

The Second is that when calculating R, you should not be taking the square root of Var/W. The Gelman & Rubin (1992) paper actually defines sqrt(R) = sqrt(V/W df/(df-2)), where df is some measure of the degrees of freedom (the way it is calculated is in their paper). There are two implications of this: 1) ChainConsumer is actually calculating sqrt(R), which is always less than R, so it will always make chains appear more converged than they actually are. 2) The value of R can decrease below 1 as long as V/W is sufficiently small (because it actually approaches df-2/df). This is bad, because by definition, R cannot be less than 1, although the effect on determining convergence is negligible.

The fix should be pretty easy. Just change line 1322 to be:
chain_std = np.array([np.std(c, axis=0)**2 for c in chains])

and get the formula for df from the Gelman & Rubin paper, and adjust line 1326 to be
R = var / w * df / (df-2.)

Validate chains when adding them

Walks with multiple chains

Hi Sam,

Really nice package!

I was hoping to plot walks with multiple chains, but it is not currently possible. I would think this would be easy enough -- the way I usually do it is to plot each chain with the same colour and a pretty low alpha. The resulting plot is "messy", but shows pretty clearly if there are any chains doing strange things compared to the rest.

What do you think?

Add ability to colour plot weight or posterior vales

Increase accuracy of max likelihood summaries

Make usetex a configure option

tick_font_size not working

I find that changing this option does not actually change the font size on the figure. The label size option is responding correctly.

`import numpy as np
from numpy.random import normal, multivariate_normal
from chainconsumer import ChainConsumer

np.random.seed(0)
data = multivariate_normal([0, 1, 2], np.eye(3) + 0.2, size=100000)
c = ChainConsumer()
c.add_chain(data, parameters=["$x$", "$y^2$", r"$\Omega_\beta$"])
c.configure(diagonal_tick_labels=False, tick_font_size=2, label_font_size=16, max_ticks=4)
fig = c.plot(figsize="column")`

and

`import numpy as np
from numpy.random import normal, multivariate_normal
from chainconsumer import ChainConsumer

np.random.seed(0)
data = multivariate_normal([0, 1, 2], np.eye(3) + 0.2, size=100000)
c = ChainConsumer()
c.add_chain(data, parameters=["$x$", "$y^2$", r"$\Omega_\beta$"])
c.configure(diagonal_tick_labels=False, tick_font_size=10, label_font_size=16, max_ticks=4)
fig = c.plot(figsize="column")`

produce the same figure.

Cannot match 2D figures made by ChainConsumer and getdist

I cannot get the output from getdist and chainconsumer to match, even using the same cosmomc chains. I attach figures showing what I mean.

testkids_2D.pdf
Cosmomc chains have a weight (multiplicity) and a likelihood. I include both of these in my 'add_chain' statement
fig = ChainConsumer().add_chain(final,weights=final_weights,posterior=final_post,parameters=paramnames)

Any suggestion what I might be doing wrong?

bug version 0.23.1?

Hi,

I've update chainConsumer to the latest version (0.23.1) and I'm having problem with the Plotter.plot() class (with the version 0.21.7 I I didn't experience this problem).
For instance, if I try to run the example "plot_introduction.ipynb" I get the following error:

`---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/usr/local/lib/python2.7/dist-packages/IPython/core/formatters.pyc in __call__(self, obj)
    335                 pass
    336             else:
--> 337                 return printer(obj)
    338             # Finally look for special method names
    339             method = _safe_get_formatter_method(obj, self.print_method)

/usr/local/lib/python2.7/dist-packages/IPython/core/pylabtools.pyc in <lambda>(fig)
    205 
    206     if 'png' in formats:
--> 207         png_formatter.for_type(Figure, lambda fig: print_figure(fig, 'png', **kwargs))
    208     if 'retina' in formats or 'png2x' in formats:
    209         png_formatter.for_type(Figure, lambda fig: retina_figure(fig, **kwargs))

/usr/local/lib/python2.7/dist-packages/IPython/core/pylabtools.pyc in print_figure(fig, fmt, bbox_inches, **kwargs)
    115 
    116     bytes_io = BytesIO()
--> 117     fig.canvas.print_figure(bytes_io, **kw)
    118     data = bytes_io.getvalue()
    119     if fmt == 'svg':

/usr/local/lib/python2.7/dist-packages/matplotlib/backend_bases.pyc in print_figure(self, filename, dpi, facecolor, edgecolor, orientation, format, **kwargs)
   2257                 orientation=orientation,
   2258                 bbox_inches_restore=_bbox_inches_restore,
-> 2259                 **kwargs)
   2260         finally:
   2261             if bbox_inches and restore_bbox:

/usr/local/lib/python2.7/dist-packages/matplotlib/backends/backend_agg.pyc in print_png(self, filename_or_obj, *args, **kwargs)
    505 
    506     def print_png(self, filename_or_obj, *args, **kwargs):
--> 507         FigureCanvasAgg.draw(self)
    508         renderer = self.get_renderer()
    509         original_dpi = renderer.dpi

/usr/local/lib/python2.7/dist-packages/matplotlib/backends/backend_agg.pyc in draw(self)
    428             if toolbar:
    429                 toolbar.set_cursor(cursors.WAIT)
--> 430             self.figure.draw(self.renderer)
    431         finally:
    432             if toolbar:

/usr/local/lib/python2.7/dist-packages/matplotlib/artist.pyc in draw_wrapper(artist, renderer, *args, **kwargs)
     53                 renderer.start_filter()
     54 
---> 55             return draw(artist, renderer, *args, **kwargs)
     56         finally:
     57             if artist.get_agg_filter() is not None:

/usr/local/lib/python2.7/dist-packages/matplotlib/figure.pyc in draw(self, renderer)
   1293 
   1294             mimage._draw_list_compositing_images(
-> 1295                 renderer, self, artists, self.suppressComposite)
   1296 
   1297             renderer.close_group('figure')

/usr/local/lib/python2.7/dist-packages/matplotlib/image.pyc in _draw_list_compositing_images(renderer, parent, artists, suppress_composite)
    136     if not_composite or not has_images:
    137         for a in artists:
--> 138             a.draw(renderer)
    139     else:
    140         # Composite any adjacent images together

/usr/local/lib/python2.7/dist-packages/matplotlib/artist.pyc in draw_wrapper(artist, renderer, *args, **kwargs)
     53                 renderer.start_filter()
     54 
---> 55             return draw(artist, renderer, *args, **kwargs)
     56         finally:
     57             if artist.get_agg_filter() is not None:

/usr/local/lib/python2.7/dist-packages/matplotlib/axes/_base.pyc in draw(self, renderer, inframe)
   2397             renderer.stop_rasterizing()
   2398 
-> 2399         mimage._draw_list_compositing_images(renderer, self, artists)
   2400 
   2401         renderer.close_group('axes')

/usr/local/lib/python2.7/dist-packages/matplotlib/image.pyc in _draw_list_compositing_images(renderer, parent, artists, suppress_composite)
    136     if not_composite or not has_images:
    137         for a in artists:
--> 138             a.draw(renderer)
    139     else:
    140         # Composite any adjacent images together

/usr/local/lib/python2.7/dist-packages/matplotlib/artist.pyc in draw_wrapper(artist, renderer, *args, **kwargs)
     53                 renderer.start_filter()
     54 
---> 55             return draw(artist, renderer, *args, **kwargs)
     56         finally:
     57             if artist.get_agg_filter() is not None:

/usr/local/lib/python2.7/dist-packages/matplotlib/axis.pyc in draw(self, renderer, *args, **kwargs)
   1145         self._update_label_position(ticklabelBoxes, ticklabelBoxes2)
   1146 
-> 1147         self.label.draw(renderer)
   1148 
   1149         self._update_offset_text_position(ticklabelBoxes, ticklabelBoxes2)

/usr/local/lib/python2.7/dist-packages/matplotlib/artist.pyc in draw_wrapper(artist, renderer, *args, **kwargs)
     53                 renderer.start_filter()
     54 
---> 55             return draw(artist, renderer, *args, **kwargs)
     56         finally:
     57             if artist.get_agg_filter() is not None:

/usr/local/lib/python2.7/dist-packages/matplotlib/text.pyc in draw(self, renderer)
    761             posy = float(textobj.convert_yunits(textobj._y))
    762             if not np.isfinite(posx) or not np.isfinite(posy):
--> 763                 raise ValueError("posx and posy should be finite values")
    764             posx, posy = trans.transform_point((posx, posy))
    765             canvasw, canvash = renderer.get_canvas_width_height()

ValueError: posx and posy should be finite values
`

The new class "plotter.plot_summary" works very nicely instead :)
Do you have any suggestion on how to fix this error?

Thanks a lot.

Matteo

Infer `shade` from presence of `shade_alpha`

Add BIC

Contours always being drawn as Gaussian.

Unless I am missing something, to draw the contours you use (from plotter.py)
levels = 1.0 - np.exp(-0.5 * self.parent.config["sigmas"] ** 2)
This assumes that our model parameters are always Gaussian distributed.

To draw contours more generally, you need to find the level that contains the volume you are looking for. This can be done by gridding the height of the map from 0 to zmax and then summing the volume of each bin with height greater than the current grid value (zthresh). When zthresh is 0, you expect to recover 1 (if normalized to 1). When zthresh is zmax, you expect to recover 0. If we calculate the area over a grid of zthresh, we can then interpolate to find zthresh that contains the volume that we are looking for.

I have attached an example file that does just this.
draw_contours.txt

Add installation instructions to the github.io page

Right now, the top google results for "chain consumer" with a space is the github.io page, and this repo isn't on the first page of google results. This is fine, but the installation instructions aren't on the github.io page. Either these should be added or a link back to the repo's installation instructions should be added to facilitate installation for someone making that search.

Fix typo in usage page. Add statistics keyword to usage page

Add DIC

Paper by Andrew Little 2009 comparing AIC, BIC, DIC

0903.4210

Also

https://arxiv.org/abs/astro-ph/0701113

Figure has label chopped off

The figure found here has a label chopped off at the top. This is purely a cosmetic issue with the documentation.

error in Gaussian KDE

Hi,
I'm trying to smooth my results but I get the following error:

File "plot_kde_extents.py", line 29, in
fig = c.plotter.plot(extents=[(-2, 4), (0, 9)])
File "/home/simone/anaconda2/lib/python2.7/site-packages/chainconsumer/plotter.py", line 128, in plot
fit_values = self.parent.analysis.get_summary(squeeze=False, parameters=parameters)
File "/home/simone/anaconda2/lib/python2.7/site-packages/chainconsumer/analysis.py", line 137, in get_summary
summary = self._get_parameter_summary(chain[:, i], weights, p, ind, grid=g)
File "/home/simone/anaconda2/lib/python2.7/site-packages/chainconsumer/analysis.py", line 149, in _get_parameter_summary
return method(data, weights, parameter, chain_index, desired_area=desired_area, **kwargs)
File "/home/simone/anaconda2/lib/python2.7/site-packages/chainconsumer/analysis.py", line 392, in get_parameter_summary_max
xs, ys, cs = self._get_smoothed_histogram(data, weights, chain_index, grid)
File "/home/simone/anaconda2/lib/python2.7/site-packages/chainconsumer/analysis.py", line 275, in _get_smoothed_histogram
area = simps(ys, x=kde_xs)
File "/home/simone/anaconda2/lib/python2.7/site-packages/scipy/integrate/quadrature.py", line 392, in simps
x = x.reshape(tuple(shapex))
TypeError: 'numpy.float64' object cannot be interpreted as an index

This happens both when I run my script and when I run the example https://samreay.github.io/ChainConsumer/examples/customisations/plot_kde_extents.html#sphx-glr-examples-customisations-plot-kde-extents-py

Thank you for your attention,
Simone

Plotting colour parameter with all params the same is bad. ie all weights 0

Add dvipng as a requirement in the README

Somehow ChainConsumer.plot interfaces with matplotlib in such a way that it requires dvipng. I did not have this and thus the examples failed. This was rectified when I installed dvipng

sudo apt-get install dvipng

Add WAIC and PSIS-LOO

PSIS-LOO

Pareto smoothed importance sampling. Aki Vehtari, Andrew Gelman and Jonah Gabry (2016).
https://arxiv.org/abs/1507.04544
see also https://github.com/avehtari/PSIS

Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Aki Vehtari, Andrew Gelman, Jonah Gabry 29 June 2016
http://www.stat.columbia.edu/~gelman/research/unpublished/loo_stan.pdf

WAIC
http://watanabe-www.math.dis.titech.ac.jp/users/swatanab/waicwbic_e.html
see also http://www.stat.columbia.edu/~gelman/research/published/waic_understand3.pdf

Both methods use the MCMC chain for evaluation.

conda install

I need this package in conda for another package I'm working on. Is it possible to have this done here, or would you like for me to get it pushed to conda?

Connecting to the des-mpp system

We would like to connect to the DES multiprobe pipeline system.

shading.

For some reason, the joint plots stopped shading the contours:

import numpy as np
from numpy.random import normal, multivariate_normal
from chainconsumer import ChainConsumer



np.random.seed(0)
cov = normal(size=(3, 3))
cov2 = normal(size=(4, 4))
data = multivariate_normal(normal(size=3), 0.5 * (cov + cov.T), size=100000)
data2 = multivariate_normal(normal(size=4), 0.5 * (cov2 + cov2.T), size=100000)

c = ChainConsumer()
c.add_chain(data, parameters=["$x$", "$y$", r"$\alpha$"])
c.add_chain(data2, parameters=["$x$", "$y$", r"$\alpha$", r"$\gamma$"])
fig = c.plot()

Is it something in the code?

TypeError: init() got an unexpected keyword argument 'gridspec_kw'

Hi, I am new to chainconsumer. I have installed it successfully on my system. I get the following error while testing any example given on the chainconsumer website:
TypeError: init() got an unexpected keyword argument 'gridspec_kw'

How to rectify this error?

Thanks.

Refactor ChainConsumer

trying to use \mathrm in labels but failed

Hi, I was trying to put labels on each histogram, the label is something like x_\mathrm{y}. Each time I got an error says latex can not process x_\\mathrm_{y}. Apparently, it adds extra "//" there. Could you kindly fix this?

Parameter text issues with min and mean from chain_res are the same

KL Divergence

x-axis limits

Hi Admin

Can I limit x-axis values of the triangle subplots for different parameters? E.g. I have a truth value which is not appearing in the subplot and if I can limit the x-axis of the subplot for that particular parameter it will be helpful. I haven't found such a variable in the document and if I missed it could you please point that to me?

Thanks
vinu

trace plot with multiple chains

Hi Sam,

I reckon it would be useful, when multiple walkers are specified for the chain and plotting the traceplot, to overplot different walkers, something like the following:

Instead of plotting each walker's chain appended in one long line.

Map p to purple in colours

Raise error when plotting only certain parameters if those parameters are not in the list

error in plotter.plot

Hi Sam, any ideas why I'd get the following matplotlib error all of a sudden?:

/home/steven/miniconda3/envs/HIHOD/lib/python2.7/site-packages/matplotlib/text.pyc in draw(self, renderer)
    761             posy = float(textobj.convert_yunits(textobj._y))
    762             if not np.isfinite(posx) or not np.isfinite(posy):
--> 763                 raise ValueError("posx and posy should be finite values")
    764             posx, posy = trans.transform_point((posx, posy))
    765             canvasw, canvash = renderer.get_canvas_width_height()

ValueError: posx and posy should be finite values

I'm running the same code as I always have. May have upgraded versions of either matplotlib or chainconsumer since last time, I'm not sure. Current version of MPL: 2.1.0 and ChainConsumer: 0.23.1

ticks position plot_summary

Hi,
is there a way to control the ticks position when using plot_summary?
I've tried to increase "max_ticks" in "configure, but it does not seem to change anything.
Thanks a lot

Matteo

Request: add either axes or figure as an argument to the plotter functions

I was hoping to make a figure that had two subfigures, and in each subfigure it would have an output from ChainConsumer. However, matplotlib subfigures cannot be created from matplotlib figure objects. Thus, I am unable to do this with ChainConsumer, since it creates and returns a figure object to the user.

A straightforward (I think) fix would be to optionally take in a figure or axes from the user, clear it, then do the plotting, then return the figure to the user.

The call signature in this case would look like

import matplotlib.pyplot as plt
import chainconsumer as cc

fig, ax = plt.subplots(1)

mean = [0.0, 4.0]
data = np.random.multivariate_normal(mean, [[1.0, 0.7], [0.7, 1.5]], size=100000)

c = ChainConsumer()
c.add_chain(data, parameters=["$x_1$", "$x_2$"])
c.plotter.plot(axis=ax, filename="example.png", figsize="column", truth=mean)

this way, ax can be a subplot, and we could use ChainConsumer figuers inside a pyplot.subplots() figure.

Binning should go from 99%CDF not min/max samples

Make KDE faster - 2D FFT using good kernel for boundary conditions (following getdist?)

Remove debug print statement I forgot to delete in plotter. Sigh.

Configure not recognising 'legend_kwargs'

Hey Sam, just started using this spiffing package today after hearing about how awesome it is from Caitlin for ages.

I'm trying to plot multiple contours with different colours. However for some reason legend_kwargs is not recognised. I've tried copy pasting the code directly from the Documents but it's still generating the error.

# G1 and G2 are arrays of galaxy  information.

c = ChainConsumer()
x_label = "$\mathrm{M}_\mathrm{H} \: [\mathrm{M}_\odot h^{-1}]$"
y_label = "$\mathrm{M}_* \: [\mathrm{M}_\odot h^{-1}]$"

w1 = np.where((G1.Mvir > 0.0) & (G1.StellarMass > 0.1))[0]
w2 = np.where((G2.Mvir > 0.0) & (G2.StellarMass > 0.1))[0]
halomass1 = np.log10(G1.Mvir[w1] * 1.0e10 / self.Hubble_h)
stellarmass1 = np.log10(G1.StellarMass[w1] * 1.0e10 / self.Hubble_h)
 
halomass2 = np.log10(G2.Mvir[w2] * 1.0e10 / self.Hubble_h)
stellarmass2 = np.log10(G2.StellarMass[w2] * 1.0e10 / self.Hubble_h)

c.add_chain([halomass1, stellarmass1], parameters=[x_label, y_label], name = "$\eta_\mathrm{SN} = 5 \: \mathrm{x} \: 10^{-3}$")
c.add_chain([halomass2, stellarmass2], parameters=[x_label, y_label], name = "$\eta_\mathrm{SN} = 5 \: \mathrm{x} \: 10^{-4}$")

c.configure(linestyles=["-", "--"], sigmas=[0, 1, 2, 3],
legend_kwargs={"loc": "upper left", "fontsize": 10},
legend_color_text=False, legend_location=(0, 0))

fig = c.plotter.plot(filename = OutputDir + 'XX.StellarMass_HaloMass' + OutputFormat, figsize = 1.0)

Yields

Traceback (most recent call last):
  File "allresults.py", line 1443, in <module>
    res.StellarMass_HaloMass(G1, G2, G4)
  File "allresults.py", line 1320, in StellarMass_HaloMass
    legend_color_text=False, legend_location=(0, 0))
TypeError: configure() got an unexpected keyword argument 'legend_kwargs'

I'm also getting the error for legend_color_text and legend_artists (haven't tried any other esoteric options).

Thanks!

TypeError while plotting in a for loop

First of all - thanks for the excellent package! I just started using ChainConsumer - so I am probably doing unusual/naive.

    c = ChainConsumer()
    for f, nburn, lbl in zip(chainfiles, burn_in_steps, chainlabel):
        chain = load_chain_data(f)
        chain_param_dist = chain[nburn:, 0:nparams]
        lnlike = chain[nburn:, nparams]

        # since we have flat priors, posterior \propto likelihood
        lnpost = lnlike
        c = c.add_chain(chain_param_dist, parameters=param_names,
                        posterior=lnpost, walkers=nwalkers,
                        name=lbl)
    for i, param_idx in enumerate(plot_param_combos):
        outfile = "{0}{1}.{2}".format(filebase, i, outputfiletype)
        params = [param_names[idx] for idx in param_idx]
        print("Generating plot for params names = {0}".format(params))
        c.configure(kde=[False] * nchains,
                    shade=[True] * nchains,
                    sigmas=[0, 1, 2, 3],
                    bar_shade=[True] * nchains,
                    plot_hists=[False] * nchains,
                    diagonal_tick_labels=False)
        c.plotter.plot(filename=outfile, figsize="column",
                       parameters=params)
        print("Generating plot for params names = {0}...done".format(params))

This results in the following:

Generating plot for params names = ['$\\log M_{min}$', '$\\sigma_{\\log M}$']
WARNING:chainconsumer.analysis:Parameter $\log M_{min}$ is not constrained
WARNING:chainconsumer.analysis:Parameter $\sigma_{\log M}$ is not constrained
Generating plot for params names = ['$\\log M_{min}$', '$\\sigma_{\\log M}$']...done
Generating plot for params names = ['$\\log M_1$', '$\\alpha$']
error occurred for key = max_ticks val = 5
All keys = ['max_ticks', 'linestyles', 'kde', 'tick_font_size', 'colors', 'plot_hists', 'legend_color_text', 'plot_color_params', 'cloud', 'statistics', 'legend_kwargs', 'bar_shade', 'label_font_size', 'watermark_text_kwargs', 'usetex', 'contour_labels', 'legend_location', 'legend_artists', 'shade_alpha', 'num_cloud', 'spacing', 'linewidths', 'serif', 'cmaps', 'sigmas', 'sigma2d', 'shade', 'smooth', 'flip', 'summary', 'contour_label_font_size', 'color_params', 'diagonal_tick_labels', 'shade_gradient', 'bins']
Traceback (most recent call last):
  File "xx.py", line 140, in <module>
    main()
  File "xx.py", line 134, in main
    diagonal_tick_labels=False)
  File "/Users/msinha/anaconda/lib/python2.7/site-packages/chainconsumer/chain.py", line 597, in configure
    assert len(val) >= num_chains, \
TypeError: object of type 'int' has no len()

I added the following lines into chain.py to help debug (which results in the additional info output in the preceeding block):

        for key in self.config.keys():
            val = self.config[key]
            try:
                assert len(val) >= num_chains, \
                    "Only have %d options for %s, but have %d chains!" % (len(val), key, num_chains)
            except:
                print("error occurred for key = {0} val = {1}".format(
                        key, val))
                print("All keys = {0}".format(self.config.keys()))
                raise

My read is that max_ticks is already defined and therefore does not pass the check. Might be good to add an additional boolean flag per key option that records whether or not the comparison with nchains test should run.

Thanks again!

covariance ellipses

Any plans on adding simple covariance ellipses to the plots?

Adding chain not in numpy array causes crash

When trying to make a corner plot from a chain that is not saved in a numpy array, it crashes.

trans_chain = [list(transform_chain(link)) for link in chain[:100]]
c = ChainConsumer()
c.add_chain(trans_chain, parameters=['p1', 'p2'])
fig = c.plotter.plot(figsize="column")

The error message I get is:

File "/pythonpath/lib/python2.7/site-packages/chainconsumer/chain.py", line 51, in validate_chain
(self.name, len(self.parameters), self.chain.shape[1])
AssertionError: Chain Chain 0 has 2 parameters but data has 100 columns

To fix it, I simply recast trans_chain as a numpy array. Opening issue because I felt that this would be easy to check for and then correct for when adding a chain.

Add option to remove whitespace between plots. Automatically after certain dimensionality

Add AIC

remove_chain method

Hi again,

since you're so fast, I thought I'd ask another favour. For what I'm doing, I'm using CC to do some convergence tests on a series of chains, and if they fail, I just want to skip that chain. The best way to do this, I think, is to have a remove_chain() method (or even just a remove_last_chain method) which undoes the add_chain() method. I've hacked one myself, but probably missing something important...

def remove_last_chain(cc):
    del cc._chains[-1]
    del cc._names[-1]
    del cc._walkers[-1]
    del cc._posteriors[-1]
    del cc._weights[-1]
    del cc._parameters[-1]
    del cc._num_data[-1]
    del cc._num_free[-1]

Figure-4; Axes labels are very small and the resolution of the figure needs to be improved.

And these comments were repeated for figures 5, 6 and 7. Though the figures look great on the screen, would it be possible to have a different set of defaults for printing quality figures?