Moving benchmarks to avalanche

#7 · open · 6 comments

View on GitHub ↗

AntonioCarta

I think Avalanche is ready to integrate the CRL benchmarks. @NickLucche do you agree with moving the benchmarks in avalanche? Once they are in the main repository, they would be considered "stable" and possibly receive more attention. There are a lot of people using Avalanche just for the benchmarks that may not know about avalanche-rl.

Comments

NickLucche

Good job with new API, it looks like a much simpler abstraction over the core concept of streams, I'll try to create a PR in avalanche asap integrating the work I did on CRL benchmarks.

NickLucche

Done here https://github.com/ContinualAI/avalanche/pull/984

vlomonaco

Shouldn't we remove them now from Avalanche-RL?

AntonioCarta

I agree, we shouldn't duplicate features. `avalanche-rl` should use `avalanche` benchmarks now.

NickLucche

Yep, in order to do so we need to add this #8 tho, currently we can't just import the latest avalanche version due to the major changes in its structure since I last backported it

NickLucche

Re-opening with a possible solution implemented here https://github.com/ContinualAI/avalanche-rl/tree/new-avalanche-version. It involves a workaround to keep the old avalanche BaseStrategy [around](https://github.com/ContinualAI/avalanche-rl/blob/new-avalanche-version/avalanche_rl/compat.py) before fully switching to the Templates model. It uses the newly implemented benchmarks in avalanche (up to date with the master branch) and BasePlugin/BaseLogger.