tommasobelluzzo / pydtmc

A library for discrete-time Markov chains analysis.

🔧 Technologies

📈 GitHub Stats

🙏 Donation

If you found any of my projects useful to you, please consider making a donation to support its maintenance and development:

pydtmc's Issues

Citation format

Hi @TommasoBelluzzo , since I am using this package for my master thesis, is there any specific format to cite this package? I found some way to format from the web:

1. https://www.wikihow.com/Cite-a-GitHub-Repository
2. https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/archiving-a-github-repository/referencing-and-citing-content
3. https://carpentries.topicbox.com/groups/discuss/T33d60669d32ac1d7/best-way-to-cite-github-repository

Let me know your thoughts.

Incorrect determination of regularity

Expected Behavior

p = [[0.4, 0.6], [1., 0.]] is regular because the square of that matrix is [[0.76, 0.24], [0.4 , 0.6]], which has all positive entries.

Current Behavior

The is_regular property returns False.

Steps to Reproduce

p = [[0.4, 0.6], [1., 0.]]
mc = MarkovChain(p, ['A', 'B'])
print(mc.is_regular)

Possible Solution

The problem seems to arise from the inappropriate use of ** in the source code for is_regular, specifically in the second-to-last line of the definition: result = _np.all(self.__p**k > 0.0). The ** gives element-wise exponentiation. For matrix exponentiation, use _np.linalg.matrix_power():
result = _np.all(_np.linalg.matrix_power(self.__p, k) > 0.0)

States Indices Very Slow for Large Matrices

Description

I'm trying to run the package on a relatively large number of states (~2000) and I have realised that calling self._transient_states_indices runs extremly slow (I waited several minutes and interrupted it then). Is there any possibility to speed it up?

Motivation and Context

I think this is a bottleneck for large MCs as many other functions use this property

Possible Implementation

Better Handling of Fractions and Normalization

Description

Getting validation errors around summing to 1 when I'm using fractions that convert to floats in transition matrices that are outside the acceptable tolerances of the numpy functions being used for validation. For example:

transition_matrix_sum 0.9999999999999999
[0.         0.         0.         0.         0.         0.
 0.         0.         0.         0.         0.11627907 0.
 0.         0.         0.         0.04651163 0.         0.04651163
 0.65116279 0.13953488 0.        ]

transition_matrix_sum 1.0000000000000002
[0.00436535 0.38415044 0.00011193 0.00145512 0.06525632 0.00223864
 0.00044773 0.00570853 0.0003358  0.00111932 0.26393553 0.
 0.         0.00873069 0.02070741 0.09032908 0.00391762 0.14651892
 0.         0.         0.00067159]

Motivation and Context

Being able to use the python fractions library or a helper function that rounds a matrix to one without losing much info would be helpful for larger numbers and states.

Possible Implementation

I attempted to implement this (https://stackoverflow.com/a/34959983) and even roll my own custom function below, but neither are working:

def force_transition_matrix_row_to_one(transition_list: list, markov_state: str, default_index_to_add_for_sum: int):
    sum_of_list = transition_list.sum()
    if sum_of_list == 1:
        return transition_list
    else:
        difference_in_values = 1 - sum_of_list
        if difference_in_values > 0:
            index_to_update = next(
                x[0] for x in enumerate(transition_list) if x[1] > difference_in_values
            )
            transition_list[index_to_update] = transition_list[index_to_update] - difference_in_values
            return transition_list
        else:
            transition_list[default_index_to_add_for_sum] = transition_list[default_index_to_add_for_sum] + difference_in_values
            return transition_list

Output of plot_graph(mc) seems to have an overlapped figure

Whenever I call plot_graph(mc), it does output an image, but it seems like there is an overlapped empty figure as attached.

Incorrect computation of accessibility matrix

Expected Behavior

The Markov chain with transition matrix p = [[0. , 1. , 0. ], [1. , 0. , 0. ], [0. , 0.5, 0.5]] has the accessibility matrix [[1, 1, 0], [1, 1, 0], [1, 1, 1]].

Current Behavior

The accessibility_matrix property returns the incorrect matrix [[1, 1, 0], [1, 1, 0], [0, 1, 1]]. Note the difference in the (2, 0) element.

Steps to Reproduce

p = [[0. , 1. , 0. ], [1. , 0. , 0. ], [0. , 0.5, 0.5]]
mc = MarkovChain(p, ['A', 'B', 'C'])
print(mc.accessibility_matrix)

Possible Solution

The computation of the accessibility matrix in the source code is incorrect. Specifically, the line am = (i + a)**(self.__size - 1) is incorrect. When applied to a NumPy array, ** is element-wise exponentiation, not true matrix exponentiation. To get matrix exponentiation, you should use numpy.linalg.matrix_power():
am = numpy.linalg.matrix_power(i + a, self.__size - 1)

ValidationError: The "p" parameter must contain only values between 0 and 1.

Expected Behavior

For a dense 2d matrix, each row in the matrix is normalized [0,1]. This should be a valid transition matrix p for initializing a MarkovChain

Current Behavior

Throwing a validation error:
in
2 import networkx as nx
3
----> 4 mc = MarkovChain(dense)

~/Library/Python/3.7/lib/python/site-packages/pydtmc/markov_chain.py in init(self, p, states)
131 except Exception as e:
132 argument = ''.join(trace()[0][4]).split('=', 1)[0].strip()
--> 133 raise ValidationError(str(e).replace('@arg@', argument)) from None
134
135 self._digraph: tgraph = nx.DiGraph(p)

ValidationError: The "p" parameter must contain only values between 0 and 1.

Steps to Reproduce

#For a dictionary states = {col:count (int)}
dense = []
for i in range(len(states)):
row = np.zeros(len(states))
for k, v in mat[i].items():
row[k] = v
row = row/row.sum()
dense.append(row.tolist())
Using dense results in the error above.
Even if I bound everything in dense in the following way:
dense[dense >= 1.] = 0.99999
dense[dense <= 0.] = 0.00001
I still get the error above.

Environment

• Application Version:
  Python 3.7.6
  PyDTMC version: 4.9.0

• Operating System:
  OSX

Possible Solution

Incorrect `is_absorbing`

How is this chain absorbing even though the only absorbing state is '3'. (Which is correct) but state '1' is not communicating with state '3'. In an absorbing markov chain, all states should be able to communicate with an absorbing state if I'm not wrong.

Thanks :-)

Hoping for a reply soon!

Graph output style is different compared to the old one in the examples given

Previously, when I called the plot_graph, it will output something like the example as so:

But now it is like this:

Is there a way to revert to the old style?

Version 8.2.0 removed?

Expected Behavior

My Streamlit app, which uses v8.2.0 should work (it was previously)

Current Behavior

I get an error including the following

[pipenv.exceptions.InstallError]: ERROR: Could not find a version that satisfies the requirement pydtmc==8.2.0 (from versions: 0.1.2, 1.0.0, 1.1.0, 1.2.0, 1.3.0, 1.4.0, 1.5.0, 1.6.0, 1.7.0, 1.8.0, 1.9.0, 2.0.0, 2.1.0, 2.2.0, 2.3.0, 2.4.0, 2.5.0, 2.6.0, 2.7.0, 2.8.0, 2.9.0, 3.0.0, 3.1.0, 3.2.0, 3.3.0, 3.4.0, 3.5.0, 3.6.0, 3.7.0, 3.8.0, 3.9.0, 4.0.0, 4.1.0, 4.2.0, 4.3.0, 4.4.0, 4.5.0, 4.6.0, 4.7.0, 4.8.0, 4.9.0, 5.0.0, 5.1.0, 5.2.0, 5.3.0, 5.4.0, 5.5.0, 5.6.0, 5.7.0, 5.8.0, 5.9.0, 6.0.0, 6.1.0, 6.2.0, 6.3.0, 6.4.0, 6.5.0, 6.6.0, 6.8.0, 6.9.0, 6.10.0, 6.11.0, 7.0.0, 8.0.0, 8.7.0)

[pipenv.exceptions.InstallError]: ERROR: No matching distribution found for pydtmc==8.2.0

ERROR: Couldn't install package: {}

Package installation failed...

/usr/local/lib/python3.10/subprocess.py:1072: ResourceWarning: subprocess 17 is still running

_warn("subprocess %s is still running" % self.pid,

ResourceWarning: Enable tracemalloc to get the object allocation traceback

sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name=4 encoding='utf-8'>

ResourceWarning: Enable tracemalloc to get the object allocation traceback

sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name=7 encoding='utf-8'>

ResourceWarning: Enable tracemalloc to get the object allocation traceback

/mount/src/lrisk_calculator

Steps to Reproduce

You can clone my app here, and run it with streamlit run Longtermist_Risk_Calculator.py