about batch training about lm-lstm-crf

[LiyuanLucasLiu]

It would be great if you can give more details about the question. i'm not sure whether i get it or not

[ZhixiuYe]

oh, sorry,
For example, in https://github.com/spro/practical-pytorch/blob/master/seq2seq-translation/seq2seq-translation-batched.ipynb , the function 'forward' in class 'EncoderRNN'
packed = torch.nn.utils.rnn.pack_padded_sequence(embedded, input_lengths)
outputs, hidden = self.gru(packed, hidden)
outputs, output_lengths = torch.nn.utils.rnn.pad_packed_sequence(outputs) # unpack (back to padded)
it packs a batch of sentences firstly, and then uses gru.
And you do it in this way:
outputs, hidden = self.gru(embedded, hidden)
And I just want to know why you do not pack them, and what is the difference between using pack_padded_sequence or not.
thank you!

[LiyuanLucasLiu]

Get it!
it's essentially due to a advanced feature of PyTorch, which is padded sequences of variable length.

In our code, we manually conducted the padding, and do not need to use this function (since i've heard that the padded sequence would cause inefficiency, but i'm not sure). And it seems that the example you refers use this technique. This is why i do not pack them, and they need to.