[Feature Request] `Frequency` merge operations about fsharp.stats

[bvenn]

For now I decided to go with an unsatisfactory hybrid of (B) and (C). I added a parameter that requests the user to specify if the maps are based on equal binning or if it is categorical data. If its continuous data with unequal binning, the merge fails with a description explaining the issue.

In future a procedure could be implemented that dissect both maps and creates a new one with a new binning. If my understanding is correct the bandwidth must be double the maximal bandwidth that is observed in the input maps.

[bvenn]

A difficulty that I'm not sure how to tackle is the required bandwidth equality on continuous data. If you want to add the following histograms:

let a =
    [(0.1,1);(0.2,1);(0.3,1)] //bandwith = 0.1
    |> Map.ofList

let b =
    [(0.15,1);(0.3,1)] //bandwidth = 0.15 or 0.05, nobody knows..
    |> Map.ofList

merge a b  
// result: [(0.1,1);(0.15,1);(0.2,1);(0.3,2)] is not valid!!

Histograms (regardless if they are Frequencies or EmpiricalDistributions) that should be merged, have to have the same bandwidth. For categorical data this is no issue!

Solution

• (A) introduce a Frequency/EmpiricalDistribution type that contains the frequency map as well as a bandwidth field that can be checked when merged
  • downside: When dealing with categorical data this is totally useless.
• (B) Leave it as it is and properly document this behaviour and trust the user not to merge histograms of differing bandwidth
  • downside: Can you make the user responsible for this issue?
• (C) Check the bandwidth when merged
  • this is not possible in the current form because bin can be missing in the map structure and the bandwidth would be determined wrongly
• (D) Add parameters to merge functions that requests the user to state the used bandwidth of both histograms. Nothing is done with these bandwidth except to check and fail if they do not match. This adds irrelevant ceremony to the function call, but ensures to not merge histograms of differing bandwidth.
  • downside: For categorical data however this parameter would be hard to define