Validation on dev set with BLEU scorer about marian-dev

[emjotde]

Works now for cross-entropy and perplexity. Leaving it open, as it does not do BLEU yet. Should probably also add early-stopping based on the first metric listed. Logging to a file might be useful as well.

[hieuhoang]

you mean there's already a framework that works for cross-entropy & perplexity, and we just have to do another 1 for bleu? Or we should still create a framework?

[emjotde]

Same framework can be used. Just needs a working translator and bleu scorer. Will work on the first thing this week.

[snukky]

We already have a validating translator. Next step is to use scorers from https://github.com/marian-nmt/moses-scorers

[emjotde]

We have "bleu" validation metric now, does BLEU on the batch content, so no preprocessing, but probably correlates well with actual BLEU, just overestimated due to oversegmentation with BPE.