Roberto maestre - [email protected]
Bojan Mihaljevic - [email protected]
https://controls.engin.umich.edu/wiki/index.php/Correlation_and_Mutual_Information
Mutual information (also referred to as transinformation) is a quantitative measurement of how much one random variable (Y) tells us about another random variable (X). In this case, information is thought of as a reduction in the uncertainty of a variable. Thus, the more mutual information between X and Y, the less uncertainty there is in X knowing Y or Y knowing X. For our purposes, within any given process, several parameters must be selected in order to properly run the process. The relationship between variables is integral to correctly determine working values for the system. For example, adjusting the temperature in a reactor often causes the pressure to change as well. Mutual information is most commonly measured in logarithms of base 2 (bits) but is also found in base e (nats) and base 10 (bans).
data:
[ (0, 0, 1, 1, 0, 1, 1, 2, 2, 2),
(3, 4, 5, 5, 3, 2, 2, 6, 6, 1),
(7, 2, 1, 3, 2, 8, 9, 1, 2, 0),
(7, 7, 7, 7, 7, 7, 7, 7, 7, 7),
(0, 1, 2, 3, 4, 5, 6, 7, 1, 1) ]
./it_tool.py
Entropy(X_1): 0.759176
Elapsed time: 0.000941
Entropy(X_3): 0.000000
Elapsed time: 0.000046
Entropy(X_4): 0.856864
Elapsed time: 0.000247
Entropy(X_0, X_1): 0.759176
Elapsed time: 0.000639
Entropy(X_3, X_3): 0.000000
Elapsed time: 0.000082
MI(X_0, X_1): 0.472903
Elapsed time: 0.001174
MI(X_1, X_2): 0.555834
Elapsed time: 0.002696