This project is for learning purpose in my ai journey . I learned about KV cache which is very nice to be honest in the context of the LLMs. And also about rotary positional encodings who are crazy crazy, it is amazing how the math was applied here.
Based on this video : https://www.youtube.com/watch?v=Mn_9W1nCFLo