ConfidNet performs worse than MCP when I reproduce SVHN results about confidnet

[chcorbi]

Hi Paul,

Thank you for your interest in the paper!

Looking back at my configuration files, I've noticed that using no weight decay during ConfidNet training helped to obtain better performances. You should try without it in your config file, the rest seems fine to me.

Please let me know if it helped :)

Charles

[pfjaeger]

Thank you for your quick reply!
Goot hint with the weight decay, this helped indeed. After experiments on all datasets and multiple runs I can see how confidnet is often better than mcp, although with limited consistency: results/rankings seem very volatile and dependent on the current run and train-split etc. Also: mcp of the dropout-based mean softmax seems to be a strong competition for confidnet ;)