catherineyellow / sta303-assignment02

STA303-Assignment02: Applications of CLIP model

The experiment has explored the capabilities of the CLIP model in diverse scenarios, focusing on image classification and recognition tasks. Through rigorous evaluation on datasets like MNIST, Fashion-MNIST, and Food101, the model’s performance in zero-shot and linear-probe settings was analyzed. Additionally, its application in recognizing small cars in a video was investigated.

The results revealed notable strengths in cross-modal understanding, zero-shot learning, and robustness when leveraging CLIP for feature extraction. However, computational intensity, data sensitivity, and high memory usage emerged as potential weaknesses, particularly evident in tasks with substantial processing requirements.

In practical applications, CLIP demonstrated versatility, showing promise in tasks like food and fashion classification as well as character recognition. However, limitations were observed in vehicle recognition tasks, indicating the need for further exploration and parameter tuning.

In conclusion, the findings underscore the importance of thorough validation of CLIP’s performance on diverse datasets and scenarios. Recommendations for future investigations include refining computational efficiency, addressing data sensitivity issues, and exploring the potential of CLIP in new application domains.