Off-by-one error when binning in powerlaw.pdf about powerlaw

[jeffalstott]

Thanks @mountaindust for the great analysis.

bins = range(int(xmin2), int(xmax2)) is already wacky because xmax is not required to be an integer, and so int()ing it has potential for problems around the edges. Can you think of a more general solution to this problem, which would also address your issue?

There are similarly other issues with edge cases with logarithmic bins, which will leave your single outlier still unincorporated. Consider xmax2=999.1 and xmin=1:

xmax2=999.1
xmin2=1

log_min_size = log10(xmin2+.0001)
log_max_size = log10(xmax2+.0001)
number_of_bins = ceil((log_max_size-log_min_size)*10)
bins=unique(
        floor(
            logspace(
                log_min_size, log_max_size, num=number_of_bins)))
print(bins)
[   1.    2.    3.    4.    5.    6.    8.   10.   13.   17.   22.   28.
   35.   45.   57.   72.   92.  117.  148.  188.  239.  303.  385.  489.
  620.  787.  999.]

Can you think of a more comprehensive solution?

Pull requests are welcome!

[mountaindust]

I wasn't thinking about non-integers before.

The bins = range(int(xmin2), int(xmax2)) solution suggested to me integer bins were desirable for linear binning, which would make sense from a plotting point of view (otherwise, why not use linspace?). If that is indeed the case, it doesn't matter if the last bin overshoots the data a bit (a bit being <1) - I think it's more important to include all data points. A more general solution than I had before is bins = range(int(xmin2), ceil(xmax2)+1), where the ceil comes from math so it returns an int? That is guaranteed to include the last point in all cases.

Certainly this does not add any more error than the current code (consider for example a case where xmax2 = 999.9), and is an improvement in that xmax will be included.

The logarithmic issue is more subtle, but I think can be similarly dealt with by taking the ceil of the last entry rather than the floor. So how about:

log_min_size = log10(xmin2)
log_max_size = log10(xmax2)
number_of_bins = ceil((log_max_size-log_min_size)*10)
bins = logspace(log_min_size, log_max_size, num=number_of_bins)
bins[:-1] = floor(bins[:-1])
bins[-1] = ceil(bins[-1])
bins = unique(bins)

The same ceil can be used; even though it returns an int, it will get converted to float to match the dtype of bins (if that matters). If you agree I can open a pull request, and I'll also throw in an edit for #66.

[jeffalstott]

Sounds like a plan! I'll look for the pull request. Thanks!