How can I use a self-defined state in tianshou? about tianshou

[Trinkle23897]

Btw, the state stored in the buffer may be a shallow copy. To make sure each of your graphs stored in the buffer is distinct, return the deep-copy version of your state in your env.

def reset():
    return copy.deepcopy(self.graph)
def step(a):
    ...
    return copy.deepcopy(self.graph), reward, done, {}

[Trinkle23897]

Do you mean https://github.com/networkx/networkx?

[Trinkle23897]

You could possibly hack the code. If your state is

{'graph': nx.Graph()}

then change this line

tianshou/tianshou/data/buffer.py

Line 156 in 1fce527

if name == 'info':

to

if name in ['info', 'obs', 'obs_next']:

This is to enable the dict storage in np.ndarray. And in your network side,

class NN(nn.Module):
    def __init__(self, ...):
        ...
    def forward(self, state, ...):
        graph = state['graph']
        (other operations)

[mrbeann]

great idea! I'll try it.

[Trinkle23897]

I’ll add a commit tomorrow so that you could use nx state without hacking the codebase.

[Trinkle23897]

@mrbeann You can have a try with the current master codebase. The state could be either a dict {'g': nx.Graph()} or directly a graph nx.Graph().

[mrbeann]

Yeah, it works now. Thanks!

[mrbeann]

Thanks for the reminder.