sj0605-datasci / ai4bharat_project

Extensive study about Gemma and Bloom models with quantisation for cross lingual task

Ai4Bharat_Project

Extensive study about Gemma and Bloom models with quantisation for cross-lingual task

Task 1

Read about Bloom and Gemma models and write code for the same calling them via huggingface and do code completion with different sorts of searches like that of Beam, topk, topP, nucleus sampling and more..

Task 2

Read about different quantisations and implemented some of them via bitsandbytes and huggingface and called big models of Bloom and Gemma

Task 3

Implemented Google Deepmind paper regarding SVCCA (Singular Value Canonical Correlation Analysis) and took out intermediate layer (1,5,8,....) embeddings of Gemma and Bloom Apply PCA and TSNE and calculate their similarity through SVCAA