thnkang / w266_sp2022_final_project

W266_Sp2022_Final_Project

DATASCI w266 Spring 2022 final project

Basumitra Chaki, Srinivas Jakka, Taehoon Kang

Automatic caption generation is gaining attention in the field of Artificial Intelligence (AI) with rapidly developing machine learning technologies. In this paper, we followed prior works that utilized the encoder-decoder framework in which we used pre trained CNN models, VGG16 and Xception, as image parts. The LSTM model is utilized for the language decoder part. The baseline model performance measured by the BLEU score was limited. To improve performance, we introduced the attention mechanism and which outperformed the previous CNN + LSTM models. The hyperparameter tuning boosted the transformer model performance and achieved 32.9% BLEU score.