class MHAPyTorchScaledDotProduct about llms-from-scratch

[rasbt]

These are good questions!

I am not sure which one is better: .reshape() or .view()?

I wouldn't say one is better than the other but yeah, they are subtly different

• The .view method returns a new tensor with the same data as the original tensor.

• The .reshape method and can handle non-contiguous tensors and will automatically make a contiguous copy of the tensor if necessary.

I've been using .view in the main book for consistency (and because it's been the original method in PyTorch for reshaping before .reshape was added later).

But yeah, it requires the .contiguous here. (Maybe not a bad thing though for educational purposes.)

I will update it here in bonus material as well, changing .reshape to .view, to make it more consistent.

.unbind(0) is not necessary (the shape of queries, keys, values does not change without it), is it a speed concern?

I think you might be right. I will investigate ...

According to the equivalent implementation in F.scaled_dot_product_attention(), it seems like self.proj() is missing at the end:

Good catch, I may have missed that one. Will update it!

[rasbt]

Looks like adding the missing projection layer had only minimal impact on the runtime. You can find the updated results in #131 . Thanks again for all your thoughtful question. Please feel free to reopen this or post new issues if you have any follow-up questions.