Hello,
I would like to save logs of some metrics such as the forgetting_metrics() using CSVLogher class, but it can not save the metrics in CSV file! Do you know what is the problem with this issue?
The code used for the logger was according to the below:
from avalanche.evaluation.metrics import (
forgetting_metrics,
)
text_logger = TextLogger(open("log.txt", "a"))
interactive_logger = InteractiveLogger()
csv_logger = CSVLogger()
eval_plugin = EvaluationPlugin(
forgetting_metrics(experience=True, stream=True),
loggers=[interactive_logger, text_logger, csv_logger], or loggers=[csv_logger]
collect_all=True,
)
Also, I get the below error:
AttributeError: 'DQNStrategy' object has no attribute 'mb_y'
Thanks in advance for your help.
avalanche-lib==0.3.1
Hello, thank you for poiting out this problem!
The error comes from the `forgetting_metrics` function that is imported from avalanche, in particular the error comes from this line of code [error line](https://github.com/ContinualAI/avalanche/blob/12b50197f74c186aef38b97e1e58ab76b0697203/avalanche/evaluation/metrics/forgetting_bwt.py#L276).
`DQNStrategy` or in general RL strategies don't have a dataset, so the attribute `mb_y` does not exist. We are aware of some inconsistencies between Avalanche and Avalanche-RL that cause errors like this and we are working to fix them asap.
The fastest way to solve the problem now would be to implement your own `forgetting_metric` functions that works with the current state of the codebase.
Let me know if you have any further questions, I will be happy to help you!
Thank you, also I had another question, I would like to save the trained agent after the agent learns from experiences and restore it for evaluation, how I can do it?
In order to store the model during training you can use the `checkpoint plugin` ([_link_](https://github.com/ContinualAI/avalanche/blob/master/avalanche/training/plugins/checkpoint.py)) that is implemented in Avalanche. You can also find an example on how to use the plugin [_here_](https://github.com/ContinualAI/avalanche/blob/master/examples/checkpointing.py).
⚠️This implementation of checkpoint store the model **ONCE** at the **END** of the training process.
As for the forgetting metrics there is one issue related to the differences in the two repositories. In order to have the plugin correctly working with `RL Strategies`, i.e. `DQNStrategy` that you were mentioning above, you need to modify line [222](https://github.com/ContinualAI/avalanche/blob/a92fe8dde2b121cfd630ca20fb8385dd6882115f/avalanche/training/plugins/checkpoint.py#L222) with the following assignment:
`ended_experience_counter = strategy.timestep`
This is due to the fact that in the current implementation of Avalanche-RL `Clock` is not supported. I have already some partial work done and I hope to fix these issues in the coming weeks.
I modified line 222 in checkpoint.py, but unfortunately I get this error now:
AttributeError: 'CheckpointPlugin' object has no attribute 'before_rollout'
Yeah I found that error too while working on the problem.
To fix the error you should add the two callbacks that are added by the RL strategies which are `before_rollout` and `after_rollout`. Since you don't need to do anything during those callbacks you can just leave them empty.
Try to add to the `CheckpointPlugin` class the folowing methods:
`
def before_rollout(self, strategy: BaseTemplate, *args, **kwargs): pass
`
`
def after_rollout(self, strategy: BaseTemplate, *args, **kwargs): pass
`
In this way everything should work correctly. I apologize for these small but annoying ploblems, we are in a transition period and a lot of things have to be changed in order to simplify the use of plugins across libraries.
Dear Dr. Piccoli, Thank you so much for your guidance and reply. Also, about the forgetting_metric, is there any resource or paper that get some help how to implement this metric in avalanche-rl?
> Yeah I found that error too while working on the problem. To fix the error you should add the two callbacks that are added by the RL strategies which are `before_rollout` and `after_rollout`. Since you don't need to do anything during those callbacks you can just leave them empty. Try to add to the `CheckpointPlugin` class the folowing methods: `def before_rollout(self, strategy: BaseTemplate, *args, **kwargs): pass` `def after_rollout(self, strategy: BaseTemplate, *args, **kwargs): pass` In this way everything should work correctly. I apologize for these small but annoying ploblems, we are in a transition period and a lot of things have to be changed in order to simplify the use of plugins across libraries.
This is actually a bug. Plugins should be supported even if they don't implement `before/after_rollout`. This is how we call plugins in avalanche (`avalanche.training.utils`):
```
def trigger_plugins(strategy, event, **kwargs):
"""Call plugins on a specific callback
:return:
"""
for p in strategy.plugins:
if hasattr(p, event):
getattr(p, event)(strategy, **kwargs)
```
missing callbacks are ignored. RL should use `trigger_plugins`.
> Yeah I found that error too while working on the problem. To fix the error you should add the two callbacks that are added by the RL strategies which are `before_rollout` and `after_rollout`. Since you don't need to do anything during those callbacks you can just leave them empty. Try to add to the `CheckpointPlugin` class the folowing methods: `def before_rollout(self, strategy: BaseTemplate, *args, **kwargs): pass` `def after_rollout(self, strategy: BaseTemplate, *args, **kwargs): pass` In this way everything should work correctly. I apologize for these small but annoying ploblems, we are in a transition period and a lot of things have to be changed in order to simplify the use of plugins across libraries.
Thanks to Mr. Piccoli, I solved this problem. Could I close this issue?