Segment-Level Recurrence with State Reuse about fast-transformers

[angeloskath]

Hmm, I am not sure I follow. If you are using the causal-linear model then why do you need the segment level recurrence?

Regardless of the above, if you do want to add a segment level recurrence, I would do it as a module that contains the segment transformer and not edit the CUDA code. This should be implementable at a higher level.

Cheers,
Angelos

[burcehan]

I use the causal-linear model, I try to calculate this matrix,that the shape of Q is (L,D), the shape of K is (S,D),the shape of V is (S,D),D is dimension ,L and S are the sequence length ,they are not equal ,I try this in the causal-linear model,but an error was returned.It does not support such calculations

[angeloskath]

Hi,

f165966 should be fixing this issue (namely L and S can be different now). If you have more questions or if the issue should not be closed feel free to reopen it or open another issue.

Cheers,
Angelos