ReDial Recommender Results about crslab

[IcyFish332]

As previously mentioned by @wangxieric, there is a performance degradation during training due to a lack of pre-training (because it was originally designed for the specific dataset ReDial and can not be applied to all datasets). If you want to reproduce closer performance, you could try batch_size: 128 and lr: !!float 1e-4 in the rec part of the file config/crs/redial/redial.yaml instead of the default setting.

This is the result using the above parameters:
{hit@1: 0.01262 hit@10: 0.06246 hit@50: 0.1947 mrr@1: 0.01262 mrr@10: 0.02364 mrr@50: 0.02948 ndcg@1: 0.01262 ndcg@10: 0.0326 ndcg@50: 0.06119 rec_loss: 20.52}(14 epochs)

[wangxieric]

I had a similar observation of this result, and the ReDial model keeps recommending nearly identical recommendations. However, a possible reason could be the missing preprocess component of ReDial:
#37.

[wangxieric]

The above solution for the ReDial model does not enable stable performance. The results can get close to the given results as above, but also frequently end up with much lower performance.