Multi object search history and tree construction about pomdp-py

[zkytony]

But when I run the code, it looks like the history is not being deleted but being expanded upon.

I think you caught a bug. _history is a field in the Cython Agent class, and can't be overridden on the python side. So clear_history in MOS Agent simply won't do its job. The correct way is to add set_history function in the Cython Agent class, similar to set_belief. I guess I was never really bothered by a growing history.

is it possible the tree construction in PO_UCT also needs to be reset every time a real action is taking?

Tree update should happen upon planner.update. If you have called that function, then you shouldn't experience the issue you're describing. Regarding debugging, check out the tree debugger.

[RajeshDM]

But when I run the code, it looks like the history is not being deleted but being expanded upon.

I think you caught a bug. _history is a field in the Cython Agent class, and can't be overridden on the python side. So clear_history in MOS Agent simply won't do its job. The correct way is to add set_history function in the Cython Agent class, similar to set_belief. I guess I was never really bothered by a growing history.

is it possible the tree construction in PO_UCT also needs to be reset every time a real action is taking?

Tree update should happen upon planner.update. If you have called that function, then you shouldn't experience the issue you're describing. Regarding debugging, check out the tree debugger.

yep, that history needs to be reset in the Cython code. What do you think about the consequence of making this update to the behavior of get all actions function : [pomdp-py/pomdp_problems/multi_object_search/models/policy_model.py](https://github.com/h2r/pomdp-py/blob/85d9a3182e6a883e44315617c9fdfde484d05c01/pomdp_problems/multi_object_search/models/policy_model.py#L31 - in this case the length of history will never be greater than 1, so I believe that should be changed to len(history) > 0

I will check out the update and the debugger thank you.

[zkytony]

The history does get accumulated during planning. get_all_actions is called by self._agent.valid_actions by default. You can try len(history) > 0. I believe that means the agent can do Look, Find. Currently it has to do Move, Look, Find. I think it's fine either way.

Regarding fixing clear_history (I believe this is what you mean by "this update"), I don't think there is noticeable difference before/after the fix.