How to use custom loss ? about tianshou

[duburcqa]

Ok, I got your point and I agree with you. But what about adding a loss_fn in the abstract base class for policies, that is basically doing nothing by default but can be overridden by the user? Because I really don't like the idea to have to overwrite 'learn' itself since it is a major source of error.

In this case, it is not a custom loss strictly speaking, but rather additional component to the original loss function (regularization), that may depend on the actor. So that it only consists in an extra function call before calling backward. I don't know if doing so is usual or not.

[Trinkle23897]

That's a good question. Currently, you can either inherit the policy class (as you mentioned) or change the original framework's code to meet your expectations.

It can be discussed further. Some existing frameworks (like RLlib) modularized the loss function part. But in my opinion, this could be inconvenient for further development. Since the loss function is highly customizable, making the abstraction of the loss function will double the code complexity.

[duburcqa]

@Trinkle23897 Up !

[Trinkle23897]

@Trinkle23897 Up !

I have no time after #106 before this Friday...Many things to do

[duburcqa]

No problem ! I can do it ! But what do you think about the idea ?

[Trinkle23897]

I think that add loss_fn is okay, but what's its input?

[oldcricket]

@duburcqa It's a great idea to make it easier with a customized loss. I wondered if you have made any progress on that. Thanks!

[MischaPanch]

The loss is an integral part of the algorithm, so maybe inheriting and overriding is better than allowing users to pass custom losses. It's a central design question, I don't see it being necessary for the 1.0.0 release, but would keep the issue open