jubiotech / calibr8 Goto Github PK

... or at least partically, because the build is actually current, but the HTML page title is outdated.

The following line should be replaced by a RegEx or some other mechanism to fetch the version number:

We have such regular expressions in other projects!!!

Found by Laura earlier today.

Opening this so we don't forget about it.

• Add kwarg to choose solver in scripy.minimize
• Maybe even use a better toolbox?

Thanks to @maxsiska for starting this discussion. @michaelosthege Should we discuss this soon?

This line errors, because None is not a valid option:

ax.set(
    xlabel=model.independent_key,
    xscale="log" if logscale else None,
)

We use a function that will not be available in v4:

Additionally, our current test CI is not testing with v4, so we should add compatibility with the v4 release.

Copying the code from DCS contrib server yields the error in the screenshot

⚠ This issue is about FZJ-internal code.

Should we implement a warning if scale_degree > 2 (or even starting with 2) is used? We could at least help the user question the choice this way.

We have contrib-models with StudentTNoise, but models with normally distributed noise are much more common.

I have these implementations already, just need to write docstrings and tests.

ToDo:

• Refactor the t-models into their own file
• Add a corresponding file with NormalNoise models for all three.

When printed on paper, the contrast between the green areas is hard to see.

A thin gray or dark-green edgeline could help.

While working on #12 I added tests and came up with one to check if calibr8.core._get_hdi is doing a good job.

It does not.

I believe the scipy.optimize.fmin doesn't cope well with the switching inside the hdi_objective, or at least not in combination with the step-function like area calculation by _interval_prob.

Some starting points to fix this:

• Make a detailed heatmap visualization of the objective function that's being optimized, based on a coarse x_cdf and cdf input. (Hypothesis: It's not smooth, but has step functions in one dimension.)
• Adapt test_interval_prob to an interpolation that should give much more accurate results compared to the step-function. (Also nicer conceptually!)
• Use the triangular example from test_get_hdi as a starting point, because it's HDI is easy to determine analytically (centerline of the triangle to its right flank).

• pygmo is not compatible with the latest versions of calibr8 (potentially due to pymc/pickle)
• Should we deprecate?

Here, the plot_model function tries to use plot_t_band irrespective of the actual noise model:

https://github.com/JuBiotech/calibr8/blob/master/calibr8/utils.py#L326-L365

Instead of hard-coding a plot_xyz_band, we can probabily generalize over al continuous distributions by using the DistributionMixin.scipy_dist class attribute that comes with each model.

@lhelleckes would you like to take a look at this?

In this line the BaseLogIndependentAsymmetricLogisticN.predict_dependent slices away the last parameter, thereby rendering sigma_degree=1 ineffective.

I'll open a PR to fix this right away.

• We could add some hints how to improve the fits.
• Make warnings generally more descriptive.

• Add general explanation or link a good page to explain student-t and its parameters
• Add hints on model complexity (first using linear, then exponential, then logistic)
• Add hint on scale_degree (start with lowest degree)

The example notebooks and the documentation are not up to date.
We should make some changes to ensure that the code runs as advertised.

Extend success condition of calibration model fits utilizing fit_scipy_global().

All this code uses only self.likelihood, so it can be easily extracted into a function.

This makes it a little easier to test, or to re-use in other contexts.

Working on #12 I started changing the broadcasting behavior of CalibrationModel.likelihood and CalibrationModel.loglikelihood, but it's not as simple and deserves it's very own issue & PR.

What needs to be done:

• Come up with a comprehensive list of shape_indep, shape_obs, model_ndim, scan_x combinations and precisely define the shape_expected.
• Keep in mind that shape_indep may be higher-dimensional, for example when broadcasting with a 5-dimensional tensor..
• Critically analyze: Do we even need scan_x if the broadcasting behavior is clearly defined?
• Make the necessary changes to CalibrationModel.loglikelihood and CalibrationModel.likelihood

The function is used by, for example, fit_scipy to check for and mask out NaN or infinity values in the data.

On multivariate independent inputs we should either skip that check or make it compatible.