BCaConfInt can't handle data with same value about bootstrap.jl

[juliangehring]

Thanks for discovering this error, it certainly needs better handling! After digging into this, here are some of my thoughts:

The issue is caused because all t1s are identical, i.e. the variance of t1 is 0. The value of t0 doesn't play a role here, though it will probably equal to t1 in most case. My concern is that if t1 has no variance, reporting a confidence interval might be conceptionally questionable. @nignatiadis Can I pick your brain on that one? Does talking about a confidence interval sound ill-defined here?

On a more practical matter, other tools (such as boot in R) refuse to compute a confidence interval in such a situation. I'd also argue that if a bootstrap always yields the same value, this often indicates some issues with the data/statistic and the results should be treated with a bit of caution.

I'll need to have a closer look, and I'm happy for any input and thoughts. At the moment, I'm leaning towards

• checking in all confidence interval methods (not only BCa) if var(t1) == 0
• reporting an informative warning/error and/or return a confidence interval of (t0, NaN, NaN)

[nignatiadis]

Hi @juliangehring! I think either choice is fine! Below some thoughts:

Say somebody tries to bootstrap the statistic f(x)=0.0, then maybe the interval [0.0,0.0] indeed makes sense. Similarly if the data is identical.

Of course, if such an interval is returned, it is probably questionable if this is really what the user was going after. But then again, bootstrap intervals are not always accurate anyway, there could be many reasons they might not have the right coverage, yet I still think it makes sense to return them (and the user can decide if they trust the result).

So, I would go with returning a interval of width zero, since Julia in general is not a language that tries to hold people's hands (as long as the difference to R's boot is properly documented).

But also a warning and/or NaN, NaN makes sense to me. I feel an error might be too much though.

[juliangehring]

Thanks @nignatiadis for the detailed explanation - this is a big help! In this case, let's stick with @rofinn original suggestion: Return a confidence "interval" with width 0 around t0, and don't raise a warning for now (I might reconsider this at a later point). This only changes the behaviour of the BCa confidence interval (the others implicitly behave this way), and leaves the interpretation up to the user.

[juliangehring]

Closed with #41.