how to apply transformer decoder to vae?in vae text folder, the implementation of transformer decoder is not consistent with decoder/transfor_decoder.py about texar

[ZhitingHu]

Thanks for the issue. The VAE example is a little staled. @jxhe will fix this soon.

The memory argument of TransformerDecoder is of shape [batch_size, max_memory_length, hidden_dim] (see the doc) so the decoder can attend to the whole sequence of encoder.

[jxhe]

Sorry the VAE example is a bit out of date, I'm fixing it now. To quickly answer, the output from encoder should be expanded to shape [batch_size, 1, hidden_dim] as the memory argument of TransformerDecoder. A clean way to do this is to set decoder_initial_state_size argument as [1, hidden_dim] when defining ReparameterizedStochasticConnector (see vae_train.py). We will update the vae example soon and show the specific usage.

[ZhitingHu]

Code is now fixed.

So the essential idea is to set max_memory_length=1 in the VAE case.