Warm-up of CFG weight about maskgit-pytorch

[llvictorll]

Hello,

You are right, I forgot to mention this in the report. Actually, the warm-up on the CFG comes from the MUSE paper (Section 2.7): "This allows the early tokens to be sampled more freely, with little or no guidance, but increases the influence of the conditioning prompt on the later tokens".

I'll try to do some experiments on the influence of this warm-up when I have time, and update the report accordingly. But I cannot give you an exact release date.

Thanks for the correction of the weight calculation, I will also update the code :)

Best,

Victor

[yukara-ikemiya]

Thank you for the reply! I've also read the MUSE paper, but I might miss that part. I'll check it out!
I'll also investigate how each technique affects sampling results by myself.
Let me close this thread.

Best regard,
Yukara