fergm / fattails

Code and Notes for fat-tailed statistics.

Fat Tails

Notes and Code for fat-tailed statistics. Inspired by Nassim Taleb's Technical Incerto.

Table of Contents

• 1. Blog
• 2. Python Notebooks
• 3. External Resources
• Other

Blog

See fattails/blog/

• 2023-10: I am drafting some articles about Entropy as used in information theory.

Notebooks

See the notebooks/README.md for more detail.

My favourite notebooks so far:

• Central Limit Theorem: How the sum of Uniform values is Gaussian
• S&P500: How geometric average return is impossible
• GameStop: January 2021 was not an outlier if you assume Power Law tails.
• Tail Alpha: How to estimate the mean of a power law

External Resources

Technical Incerto Book One:

• Free PDF: researchers.one, arxiv.org
• Errata: fooledbyrandomness.com

More Links:

• Uncertainty Reading Club: Website
• Code and Links collected by Marcos Carreira: StatisticalConsequencesOfFatTails
• Anthony Voto: Diagnosing the SP500 Twitter thread, GitHub Code

Other Notes

Contact ✉

How to reach me:

• @MFergal on Twitter

Contributors

Special thanks to:

• Daniel Reti: github.com/drettt

Python Package

I started this project as a Python Package. Since then I shifted focus to python notebooks instead. Below are some notes about the packaged code and functions.

Quick Access:

• fattails.mad(): Calculates mean absolute deviation.
• fattails.plot_MS_moments(): Plots the cumulative max/sum ratio of moments 1 to 4.

Other:

• fattails.metrics.get_survival_probability(): Calculate survival probabilities for a given dataset.
• fattails.metrics.calculate_moments(): Generate dataframe with the chosen moments for each datapoint
• fattails.metrics.max_over_sum(): Calculate the cumulative max/sum ratio

Example:

$ pip install fattails
$ python

>>> import fattails
>>>
>>>
>>> fattails.mad([1,2,3]) # Calculate Mean Absolute Deviation of [1,2,3]
0.6666666666666666
>>>
>>>
>>> fattails.metrics.get_survival_probability([1,2,3]) # Get survival probability for each value in your data
0    0.75
1    0.50
2    0.25
Name: survival_probability, dtype: float64

Derivations

Roughwork:

• How much data do I need?

fattails's Issues

Geometric mean should say Arithmetic mean

Notebook-11 about Ergodicity mixes up the naming of geometric and arithmetic mean.

Add License

What

• I guess MIT or BSD license
• Django has a brief comments on the matter here

Why

• So others can use the code
• and know what the expectations are

Notebook 37 Kappa Section

Problems:

• The kappa section at the end is incomplete roughwork at the moment.
• The kappa section bootstrapping takes a long time to run

README.md Import Example

Hi. In the example in the readme.md, I think the import is missing the metrics part. I think it should be from fattails import metrics and then you can use metrics.mad(...). Hope this helps.