cloud-cv / gsoc-ideas Goto Github PK
View Code? Open in Web Editor NEWCloudCV GSoC Ideas
Home Page: https://gsoc.cloudcv.org
CloudCV GSoC Ideas
Home Page: https://gsoc.cloudcv.org
Project Title: Writing tutorials and implementing popular deep learning architectures in CloudCV
Description: This project involves implementing popular deep learning algorithms and integrating them with the code base of CloudCV. Having said that, this project also aims at developing a
Web-application where users can share their code, discuss about their research work and write
tutorials .
Deliverable: By the end of this project one can expect a web-application which would function as a learning platform to all the members of the computer vision community with the following features :
It will lay the foundations of a complete repository for prominent deep learning algorithms discussed and enable the computer vision community to invest their time in learning something new rather than implementing the existing algorithms.
The web application would host source codes for popular deep learning algorithms, provide an interactive platform to the users to view the results of each algorithm along with in-depth tutorial.
To sum up the end-product of this project ie.. a web-application will essentially be a one stop resource where users can find relevant papers, open source code based on those papers, an online demo of that algorithm and a descriptive tutorial on the same.
Mentor: Deshraj Yadav (@deshraj), is a suitable mentor as he had been involved in the project when it was proposed last year as well. Other mentors too can be added ?
Skills:
Familiarity with containers, javascript, HTML, CSS and PHP would be a big plus.
Expertise in using python based web-servers like Flask and Django.
Familiarity with deep learning frameworks like Caffe / Theano / Keras / TensorFlow etc. Students are expected to have played around with these tools and should be familiar with the input / output pipelines.
Familiarity with building multi-threading, multi-processing architectures, and asynchronous operations.
Expertise in Lua (especially Torch framework) to build a similar tool for Torch. This may require building an interface for python-torch communication and the student will have to experiment with various ways since Lua is not as mature as Python in terms of open source web-frameworks.
Skill Level: Medium
We need to remove the extra content You can do this project as part of the Google Summer of Code program. Click here for more information.
Project Title: Share Deep Learning Models online
Description: One of the problems facing young researchers who want to learn more about deep learning models is the amount of effort it takes to learn these new frameworks. Here is a small list of 64 deep learning frameworks that are available. Each framework is good at different purposes. The goal of this project is to provide an online platform for trying deep learning algorithms / models that will reduce the barrier of entry to the world of deep learning and applications in computer vision.
In GSoC '16, @gauravgupta22 built the first version of Fabrik, a platform to build deep-learning models via a simple drag-and-drop interface. As of today, we support export to model definition files of widely popular deep learning frameworks like TensorFlow, Caffe and Keras, as well as the ability to import and visualize models developed in these frameworks.
Our next goal is become a collaborative platform, where students and researchers can have model discussions, and collaboratively build and edit models in real time
Deliverable: the issues mentioned above should be implemented by the end of the GSoC period. Moreover, we expect students to also have implemented other improvements that they have proposed.
As an additional goal, we want to add support for PyTorch which is another widely popular deep learning framework.
Mentor: Viraj Prabhu @virajprabhu, Deshraj Yadav @deshraj, Harsh Agrawal @dexter1691
Co-Mentor: Shiv Baran Singh @spyshiv
Skills: ReactJS, Python, Django, familiarity with deep learning a plus (but not essential)
Skill Level: Medium
Get started: Take a look at our issues on Github, the ones marked as starter-project
are good places to start. Feel free to reach out to us on our Gitter channel if you have questions.
Project Title: Static code upload challenge evaluation and enhancements in GitHub based challenge creation
Description:
EvalAI is a platform to host and participate in AI challenges around the globe. To a challenge host, reproducibility of submission results and privacy of the test data are the main concerns. Towards this, the idea is to allow users to submit a docker image for their models and evaluate them on static datasets. In order to achieve this we want to build a pipeline which will use the dockerized models and run it on kubernetes based infrastructure with stored test annotations and report the results on the EvalAI leaderboard. Another part of the project is to streamline our challenge creation pipeline. Last year we added support for github based challenge creation which allows challenge hosts to use a private github repository to create and manage updates in a challenge. The goal for this year is to support bi-directional updates for challenges created using github. This feature will allow hosts to sync changes from EvalAI UI to their challenge github repository. The goal is to enhance the challenge creation experience for challenge hosts involving minimal support from the EvalAI team.
Deliverable:
Mentor: Khalid Riyaz (@KhalidRmb), Ram Ramrakhya (@Ram81), Rishabh Jain (@RishabhJain2018)
Skills: Python, Django, Kubernetes, Docker
Skill Level: Hard
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSOC-2020)
Important Links:
Project Title: Improvements in EvalAI frontend
Description:
After last year’s GSOC, we’ve reached feature parity on EvalAI-ngx with the existing UI, this project will involve fixing the last remaining kinks in the UI. The goal of this project would be to improve the new UI as we replace the existing UI with the new UI before GSOC. We will be improving on the new UI and incorporating the feedback we will receive from the challenge hosts and participants for the AI challenges organized this year.
Deliverable:
Mentor: Kajol Kumari (@Kajol-Kumari), Mayank Lunayach (@lunayach), Rishabh Jain (@RishabhJain2018), Ram Ramrakhya (@Ram81)
Skills: Angular 7, HTML, CSS, Typescript
Skill Level: Medium
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSOC-2021)
Important Links:
Project Title: Implementing python package and new features for EvalAI
Description:
a) Currently, if someone wants to make a submission to a challenge on EvalAI then he has to login to EvalAI and upload a file. We want to make this process even easier for researchers who really like their terminal screen. The first main goal of this project is to create a python package for EvalAI which lets the participants to import evalai
as a python package and then make submissions through their python script instead of logging in to the website and then doing it.
b) EvalAI uses Django REST Framework (DRF) at the backend. We want to add a lot of new features this summer where we want the student to implement RESTful web services using DRF. We expect the student to write simple and clean REST APIs, unit tests and an exhaustive documentation for each feature that he is building. Please see the deliverables section to get more details about the project.
Deliverables:
This package should include the following functionalities (but students are strongly encouraged to come up with their ideas/features and should definitely mention in their proposal):
For Participants:
For Hosts:
The above mentioned features should be implemented by the end of the GSoC period. Moreover, we expect students to also have implemented improvements that they have proposed.
Mentors: Rishabh Jain @RishabhJain2018, Shiv Baran Singh @spyshiv
Co-Mentor(s): Deshraj Yadav @deshraj
Skills: AngularJS, Python, Django, Django REST Framework, D3.JS, AWS
Skill Level: Difficult/Ambitious
Getting started:
We expect the students to solve at least one (the more the better) from the following list of issues which are super relevant to the project. Here is the list of getting started issues solve (some of these would already be open issues on Github):
Recommended: Try to fix some issues (note that there are some issues labeled with GSOC).
Important Links:
Project Title: Improvements in EvalAI User Interface
Description: The goal of this project is to improve the overall user experience for challenge hosts and participants on EvalAI by allowing them to tag and filter challenges and creating an intuitive and informative leaderboard. This project will involve creating - a comprehensive search feature to find challenges, a tagging system for different challenge types (ex: Computer Vision, NLP, etc) for categorization. This will help participants and challenge hosts find challenges and search for related ones based on tags. An improved leaderboard, along with the search feature, will help in streamlining the process of organizing challenges, participating in challenges, and ranking participants. In addition, we will also work on adding support for relevant metadata for each challenge such as prize money, sponsors, etc.
Deliverable:
Search, Filter and Tagging
Leaderboard
Mentor: Gunjan Chhablani, Ram Ramrakhya
Skills: Python, Django, AngularJS, AWS
Skill Level: Medium
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSOC-2023)
Important Links:
Project Title: Improve demo creation in Origami
Description:
Origami (previously called CloudCV-fy your code) is a AI-as-a-service solution that allows researchers to easily convert their deep learning models into an online service that is widely accessible to everyone without the need to setup the infrastructure, resolve the dependencies, and build a web service around the deep learning model.
Deliverable:
Mentor: Avais Pagarkar @AvaisP, Utkarsh Gupta @uttu357
Skills:
Skill Level: Medium
Get started: Take a look at our issues on Github, the ones marked as starter-project
are good places to start. Feel free to reach out to us on our Gitter channel if you have questions.
Important links:
Project Title: Adversarial Data using Gradio and EvalAI
Description: The aim of this project is to develop an infrastructure that enables the collection of adversarial data for models submitted to EvalAI. This will be achieved by integrating Gradio with EvalAI's code upload challenge pipeline and deploying the models as web services. The web services will record all user interactions, providing a dataset for each submission that can be used to evaluate the robustness of the model.
Deliverable:
Mentor: Rishabh Jain, Ram Ramrakhya
Skills: Python, Django, AngularJS, AWS
Skill Level: Hard
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSOC-2023)
Important Links:
Project Title: Improvements in EvalAI frontend
Description: Add meta tags for each challenge page. If a user shares some challenge page link on twitter, then it should show the details of challenge and challenge cover picture instead of EvalAI image.
Deliverable:
Mentor:
Skills:
Skill Level:
Get started:
Project Title: Improvement in EvalAI frontend
Description:
As a part of last year’s GSOC, we took the first step towards modernizing our UI which involved shifting the current codebase from Angular 1 to Angular 7. As we’ve reached the feature parity with the existing UI, this project will involve fixing the last remaining kinks in the UI and incorporating latest UI feedback which we have received from the challenge hosts and participants for the AI challenges organized this year. The goal of this project would be to successfully replace the existing UI with the new UI after GSOC.
Deliverable:
Mentor: Sanjeev Singh @Sanji515 , Mayank @lunayach , Shekhar Prasad Rajak @Shekharrajak, Rishabh Jain @RishabhJain2018
Skills: Angular 7, HTML, CSS, Typescript
Skill Level: Medium
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSOC-2020)
Tutorials:
Important Links:
Project Title: Evaluating submission code in Docker containers on EvalAI
Description:
The rise of reinforcement learning based problems or any problem which requires that an agent must interact with an environment introduces additional challenges for benchmarking. In contrast to the supervised learning setting where performance is measured by evaluating on a static test set, it is less straightforward to measure generalization performance of these agents in the context of the interactions with the environment. Evaluating these agents involves running the associated code on a collection of unseen environments that constitutes a hidden test set for such a scenario. The goal of this project is to set up a robust pipeline for uploading prediction code in the form of Docker containers (as opposed to test prediction file) that will be evaluated on remote machines and the results will be displayed on the leaderboard.
Deliverable:
The aforementioned features should be implemented by the end of the GSoC period. Moreover, we expect students to also have implemented improvements that they have proposed.
Mentors: Deepesh Pathak @fristonio , Rishabh Jain @RishabhJain2018 , Deshraj @deshraj
Skills Required: Docker, AWS-CLI, Django, DRF [Knowledge of Reinforcement Learning is not necessary.]
Skill Level: Difficult/Ambitious
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSOC-2019)
Tutorials:
a) Docker
b) AWS-CLI
c) Buildpack
d) jupyter-repo2docker
Important Links:
Which database is used? Is it postgreSQL or MySQL?
Project Title: Monitoring setup for EvalAI admins
Description:
As the number of challenges on EvalAI are increasing, we want to focus on improving the performance of our services. As a first step, we will focus on monitoring and measuring all the key metrics of our services. Insights from these will allow us to efficiently utilize our infrastructure, improve uptime and reduce costs. The project will concentrate on setting up metric reporting and alerts infrastructure, writing REST API’s, plotting relevant graphs and building analytics dashboards to help EvalAI admins maintain and monitor the services.
Deliverable:
Mentor: Deshraj Yadav (@deshraj), Rishabh Jain (@RishabhJain2018), Ram Ramrakhya (@Ram81)
Skills: Python, Django, Django rest framework, Docker
Skill Level: Medium
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSOC-2021)
Important Links:
Project Title: Enhanced Exception Handling Testing Documentation
Description: This project is dedicated to elevating the overall user experience on EvalAI by implementing robust exception handling mechanisms, strengthening the suite of test cases, and enhancing the comprehensiveness of documentation.
This project aims to significantly contribute to EvalAI's reliability, user-friendliness, and developer-friendliness. The deliverables below collectively contribute to a more stable and user-centric EvalAI experience.
Deliverable:
Mentor: @gautamjajoo , @RishabhJain2018
Skills: Django, Markdown, Python
Skill Level: Easy
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSoC-2024).
Important Links:
Project Title: Port EvalAI from Angular 1 to Angular 5
Description: The current frontend code of EvalAI uses Angular 1. We want to migrate it to Angular 5 since it brings a lot of new features which brings easy maintainability, better SEO etc. Some of the initial tasks include:
Getting started with Angular 5
Setting up EvalAI-ngx (https://github.com/Cloud-CV/EvalAI-ngx)
Setup Docker for EvalAI-ngx
Setup CI/CD pipeline using Travis CI
Implement the features in Angular 5 that are already present in current version
Implement responsive web application
Write the robust test suite
Properly document the codebase
Setup sphinx documentation
Extended goals:
Integrate server side rendering for Angular 5
Besides this, students should also provide ideas of their own.
Deliverables: The issues mentioned above should be implemented by the end of the GSoC period. Moreover, we expect students to also have implemented improvements that they have proposed.
Mentors: Akash Jain @aka-jain, Shiv Baran Singh @spyshiv, Rishabh Jain
@RishabhJain2018
Co-Mentor(s): Deshraj Yadav @deshraj
Skills: AngularJS, Python, Django, Django rest framework
Skill Level: Medium
Get started: Take a look at our issues on Github.
Recommended: Try to fix some issues (note that there are some issues labeled with GSOC on the project repository). For example, look at this issue: Cloud-CV/EvalAI-ngx#46
Important Links:
Project Title: Analytics Dashboards for EvalAI Users
Description: The goal of this project is to provide challenge hosts and participants with insightful analytics to track their progress on the platform. This project will involve writing REST APIs, plotting relevant graphs, and building analytics dashboards for both challenge hosts and participants. The analytics will help challenge hosts view the progress of participants in their challenge (changes in performance and ranking over time), and participants will be able to visualize the performance of all their submissions over time and their corresponding rank on the leaderboard.
Deliverable: Expectations from the student at the end of the project
Mentor: Gunjan Chhablani, Ram Ramrakhya
Skills: Angular 7, Django, Django Rest Framework, D3.js
Skill Level: Medium
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSOC-2023)
Important Links:
Project Title: Project title, short enough to catch attention
Description: General information about the project, avoid one Liners, the description should be as detailed as possible.
Deliverable: Expectations from the student at the end of the project
Mentor: Who is the mentor? Who is the Co-Mentor? Also please assign the issue to the mentor!
Skills: Which skills are needed? Programming languages, frameworks, concepts etc.
Skill Level: Easy, Medium, Hard
Get started: Tasks that mentors may want to suggest students so that they can start contributing to the code base (e.g. junior jobs, low hanging fruits, discussion on the mailing list)
Project Title: Easy challenge management on EvalAI
Description:
This project will focus on streamlining the newly adopted GitHub challenge creation pipeline, building API’s for fully automating challenge creation on EvalAI, adding new capabilities in EvalAI’s latest frontend for a seamless user experience, and making our backend robust and less error-prone by adding test cases for different frontend and backend components. As of now, EvalAI admin has to be in the loop for the challenge creation process with respect to scaling worker resources for prediction-based AI challenges, setting up remote evaluation for AI challenges, and most importantly setting up code-upload AI challenges on EvalAI, the goal of this project is to remove EvalAI admin out of the loop by fully automating the process.
Deliverables:
Add feature to approve participants for a challenge by challenge hosts for unauthorized signups in the challenge.
Give challenge hosts/participants the control to remove participants from a challenge/team.
Minor: Fix the issue of entering email ID in caps while signing up on EvalAI.Add git bi-directional sync support on EvalAI. See this PR for reference.
Add support to create new challenge phases/dataset splits, etc. from the Github-based challenge creation pipeline after the challenge has been created.
Add feature in GitHub-based challenge to manage multiple challenge configs over the years for the same challenge.
Add feature to create GitHub repositories for existing challenges on EvalAI to allow users to migrate from config challenge creation to github based challenge creation.
Add APIs and celery tasks support to re-run bulk submissions. Also, add complete UI changes to allow hosts to re-run all existing submissions to the challenge.
Increase submission message time limit in the SQS queue to prevent the submission messages from expiring when there are large number of pending submissions.
Add APIs and UI changes to allow challenge hosts to rename the metrics on the leaderboard without needing to re-run submissions.
Add challenge configuration examples with documentation for code-upload challenges.
Add examples and documentation for remote challenge setup on EvalAI.
Mentors: - Ram Ramrakhya, Rishabh Jain
Skills Required: - Python, Django, AngularJS, AWS
Project size - 175 hours
Difficulty - Medium
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSOC-2022)
Important Links:
Hey @RishabhJain2018
I see #37 was a GSoC 2021 Idea but hasn't been implemented yet, are there any plans of Cloud-CV to participate in GSoC 2022 if yes then which all projects will be taking part in that??
EvalAI??
Project Title: Analytics dashboards for challenge hosts and participants
Description:
This project will involve writing REST API’s, plotting relevant graphs and building analytics dashboards for challenge hosts and participants. The analytics will help challenge hosts view the progress of participants in their challenge -- for instance, comparing the trends of the accuracy from participant submissions over the period of time. Participants will be able to visualize the performance of all of their submissions with time and their corresponding rank on the leaderboard. The final goal is to provide users with several analytics to track their progress on the platform.
Deliverable:
For challenge host -
For participants -
Mentor: Gautam Jajoo (@gautamjajoo), Rishabh Jain (@RishabhJain2018), Ram Ramrakhya (@Ram81)
Skills: Angular 7, Django, Django Rest Framework, D3.js
Skill Level: Medium
Project size - 175 hours
Difficulty - Medium
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSOC-2022)
Important Links:
Project Title: Seamless User Experience & Leaderboard Porting
Description: This project aims to enhance the overall user experience on EvalAI by introducing user-centric features, improving existing functionalities, and optimizing integrations with external platforms.
Two key features that we aim to implement in this project will be:
These features, coupled with additional deliverables below, will improve the overall experience of EvalAI users and add several utilities for improving challenge participation experience.
Deliverable:
Mentor: Rahul Singh, @gautamjajoo , @gchhablani @RishabhJain2018
Skills: SQL, Django, AngularJS, AWS
Skill Level: Medium
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSoC-2024).
Important Links:
Project Title: Add search option on all challenges page
Description: General information about the project, avoid one Liners, the description should be as detailed as possible.
Deliverable: Expectations from the student at the end of the project
Mentor: Who is the mentor? Who is the Co-Mentor? Also please assign the issue to the mentor!
Skills: Which skills are needed? Programming languages, frameworks, concepts etc.
Skill Level: Easy, Medium, Hard
Get started: Tasks that mentors may want to suggest students so that they can start contributing to the code base (e.g. junior jobs, low hanging fruits, discussion on the mailing list)
Project Title: Streamlining challenge creation on EvalAI
Description:
EvalAI is a platform to host and participate in AI challenges around the globe. To host a challenge, challenge creation is one of the core features which is utilized by challenge hosts to create AI challenges. The idea is to use private GitHub repositories to host the challenge files instead of the zip file. The next step is to build and integrate a continuous deployment pipeline with GitHub so that for every new commit in the challenge repository, the changes are automatically reflected on the UI. We will also build support for tests so that new commits are fully tested before they are pushed to the live challenge hosted on EvalAI. The goal is to enhance the challenge creation experience for challenge hosts and set up a challenge on EvalAI by the challenge hosts involving minimal human effort from the EvalAI team.
Deliverable:
Mentor: Ram @Ram81 , Rishabh Jain @RishabhJain2018
Skills: Python, Django, Django Rest Framework, Knowledge of any CI/CD tool.
Skill Level: Ambitious
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSOC-2020)
Tutorials:
Important Links:
Project Title: Adversarial data collection with Gradio
Description:
This project will focus on building an infrastructure to allow exposing models submitted to EvalAI as demos in order to collect adversarial data for the model. As part of the project, we will integrate Gradio with our code upload challenge pipeline to allow deploying the models as web services. Additionally, this web service will record all interactions to curate a "in-the-wild" dataset for each submission.
Deliverables:
Mentors: - Ram Ramrakhya, Rishabh Jain
Skills Required: - Python, Django, AngularJS, AWS
Project size - 175 hours
Difficulty - Hard
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSOC-2022)
Important Links:
Project Title: Origami
https://github.com/Cloud-CV/cvfy-lib
https://github.com/Cloud-CV/cvfy-frontend
Description: Deep learning and its application in AI-subfields (computer vision, natural language processing) has seen a tremendous growth in the recent years. Driven in part by code, data, and manuscript sharing on github and arxiv, we are witnessing increasing public access to state-of-the-art deep learning models for object detection, classification, image captioning, and visual question answering.
However, running someone’s released implementation and making sense of the results often involves painstaking preparation and execution involving steps like setting up the environment and dependencies (installing torch / caffe / tensorflow / keras / theano), setting up the I/O pipeline, keeping track of inter-package consistencies, etc.
Origami (previously CloudCV-fy your code) is a platform that can automatically create an online demo and a corresponding API that other researchers / developers can use without understanding fine-grained details about how the algorithm works. Testing or experimenting the model should be as simple as going to a web-page and uploading images to look at the results.
Examples of such manually curated demos can be found at:
http://cloudcv.org/vqa/
http://cloudcv.org/classify/
http://cloudcv.org/vip/
Mentor: Deshraj Yadav @deshraj , Harsh Agrawal @dexter1691
Pre-requisites:
Deliverables:
CMS system that gives user enough flexibility to modify the page according to his/her need. Different tasks need different input/output setup. For example, object classification will take an image as input and output a class label, while visual question answering will take image + question as input and return an answer. Therefore CMS should provide enough flexibility to support such use-cases.
Pre-defined templates for the most popular Artificial Intelligence tasks.
Support for third-party integrations like upload images from Dropbox, or save results to Dropbox.
Support Default sample files to be uploaded by the user.
Discover page to search different demos.
Improve the cvfy-lib and release it as pypy package
Setup continuous integrations
REST APIs for easy third party integrations like FB / Slack bot system
Deep integration with EvalAI. Allow "to compare" two algorithms on real user provided data.
Anonymous link to the demo! This is to complement a paper in review where anonymity is important. It would be super cool if there can be central reliable place for users to anonymously make their demo available so that reviewers can see it without being able to see who the authors are for that demo!
By the end of GSOC, students are expected to finish the above mentioned deliverables.
Skill Level: Medium
Get started: Take a look at our issues on Github, the ones marked as GSOC
are good places to start. Feel free to reach out to us on our Gitter channel if you have questions.
Project Title: Implement robust evaluation pipeline in EvalAI
Description:
Currently, the submission worker that evaluates the challenge requires manual scaling. Moreover, logging & metrics-monitoring isn’t available to the challenge hosts for the submission worker in real-time. Also, an often requested feature by the challenge organizers has been the ability to test their competition package (evaluation scripts, etc) locally before uploading it to EvalAI. This capability will also reduce assistance required by the platform maintainers. The goal of this project is to write a robust test suite for submission worker, port it to AWS Fargate to setup auto-scaling and logging. The tasks will also include giving control to challenge hosts over the submission worker from the UI in terms of starting, stopping and restarting it.
Deliverable:
Extended Goals:
Mentor: Ram Ramrakhya @Ram81 , Rishabh Jain @RishabhJain2018 , Deshraj @deshraj
Skills: Python, Django, Django Rest Framework, AWS, Docker
Skill Level: Hard
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSOC-2019)
Tutorials:
a) Docker
b) AWS-Fargate
Important Links:
Project Title: Issues for GSOC 2019
Description: We need new issues for GSOC 2019, so potential students (like me) can know what projects to focus on and which issues to tackle for GSOC application.
Deliverable: List of issues
Description: Fabrik is an online collaborative web application for building and visualizing neural networks in the browser, aimed at lowering the barrier to entry to getting started with deep learning. We wish to create a model-agnostic platform which works with most popular deep learning frameworks and provides a seamless experience when trying to visualize existing models, create new ones, or to export them to the framework of your choice.
Over the last couple of years, we have built a version of this platform which provides a drag-and-drop interface for creating neural networks, extensively supports two popular deep learning frameworks (Caffe and Keras), and has experimental support for Tensorflow, another widely-popular framework.
Our goal moving forward is to complete support for our existing frameworks and extend support to new and upcoming frameworks like PyTorch. Initially, we would like to accomplish the following tasks:
Deliverables: The issues mentioned above should be implemented by the end of the GSoC period. Moreover, we expect students to also have implemented improvements that they have proposed.
Extended Goals:
Mentors: Utsav Garg
Co-Mentor(s): Deshraj Yadav
Skills:
Skill Level: Medium
Getting started: Take a look at our issues on Github, the ones marked as starter-project
are good places to start. Feel free to reach out to us on our Gitter channel if you have questions.
Important links:
Project Title: Enhancements in code upload pipeline
Description:
As EvalAI hosts more code-upload challenges and researchers utilizes our modular kubernetes based infrastructure for hosting these challenges, we would like to automate this pipeline as much as possible to enhance user experience. During GSoC 2019, we built this pipeline for evaluating AI model’s code by running it against unseen test environments in real time and this year the plan is to add features like start, stop, restart, delete cluster, etc. so as to give challenge hosts more control over their challenge evaluation cluster. This will not only involve control over the nodes running evaluation but also viewing logs which are being updated in real-time. Finally, the plan is to give challenge hosts the capability to run the evaluation cluster in their cloud by simply plugging in their keys and rest all will be taken care by EvalAI.
Deliverable:
Mentor: Kartik Verma @vkartik97 , Rishabh Jain @RishabhJain2018
Skills: Docker, Kubernetes, AWS, Django, DRF
Skill Level: Difficult
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSOC-2020)
Tutorials:
Important Links:
Project Title: Showing a progress bar while uploading a submission
Description: This is one of the expected task in the process of improving the existing UI before GSOC 2021. The expected tasks are from the feedback received from the challenge hosts and participants for the AI challenges organised that year.
Deliverable: Show progress bar apt to the style and theme of the application when a participant is uploading a submission from UI.
Mentor: @RishabhJain2018
Skill Level: Medium
Project Title: Implementing RESTful web services for EvalAI
Description: is an evaluation server that will host AI challenges like Visual Question Answering, Image Captioning. Our next goal of this project is to implement REST APIs. Some of the initial tasks include:
Besides this, students should also provide ideas of their own.
Deliverable: the issues mentioned above should be implemented by the end of the GSoC period. Moreover, we expect students to also have implemented improvements that they have proposed.
Mentor: Taranjeet Singh @trojan, Akash Jain @aka-jain, Shiv Baran Singh @spyshiv
Co-Mentor: Deshraj Yadav @deshraj
Skills: AngularJS, Python, Django
Skill Level: Medium
Get started: Take a look at our issues on Github. Try to fix some (note that there are some issues labeled with GSOC).
Project Title: Robust test suite and infra optimization setup
Description:
This project will focus on building a robust test suite for EvalAI's functionalities. As part of the project we will focus on making EvalAI robust and less error-prone by adding test cases for different frontend and backend component. It will involve adding unit tests for the API suite, prediction upload evaluation workers, code upload evaluation workers (on EKS) and integration tests for the end to end testing of all the components.
Deliverables:
Mentors: - Ram Ramrakhya, Rishabh Jain
Skills Required: - Python, Django, AngularJS, AWS
Project size - 175 hours
Difficulty - Medium
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSOC-2022)
Important Links:
Project Title: Enhance UI/UX of EvalAI
Description:
This will focus on improving the existing UI of EvalAI to improve the experience of both challenge organizers and participants. We also want to improve the discoverability of all the features that are supported on EvalAI. With the increase in the number of users of on EvalAI, it is critical to have a frictionless and intuitive user experience. The goal of this project is to ease the pipeline for challenge creation, enhancing the user experience of the platform, adding plots for displaying the progress of state-of-the-art algorithms, for displaying the progress of participant team in a challenge over the years and several other features.
Deliverable:
view all submissions
based upon participant team nameExtended Goals:
Mentor: Gali Prem Sagar @galipremsagar, Shivani Prakash @shivaniprakash95 (Design Mentor) , Rishabh Jain @RishabhJain2018
Skills Required: AngularJS, HTML, CSS, Javascript
Skill Level: Medium
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSOC-2019)
Tutorials:
a) AngularJS
b) Javascript
Important Links:
Project Title: Challenge Synchronization with GitHub Repositories
Description: This project aims to facilitate the migration of legacy challenges from challenge zip files to GitHub repositories, providing hosts with access to their old challenges through a familiar interface. Additionally, it will establish a bidirectional sync between EvalAI and GitHub repositories, ensuring that changes made on either platform are reflected seamlessly. With enhanced compatibility and synchronization capabilities, this project will contribute to a smoother experience for hosts managing challenges on EvalAI.
Deliverable:
Mentor: @gchhablani, @RishabhJain2018
Skills: Django, Markdown, Python
Skill Level: Medium-Hard
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSoC-2024).
Important Links:
Project Title: Evaluation Infrastructure Optimization
Description: This project aims to enhance EvalAI's functionalities through automating large worker deployments in AWS, adding relevant features for efficient challenge management and also writing a robust and efficient test suite. The focus of the project is two-fold:
Deliverable:
Mentor: Gunjan Chhablani, Ram Ramrakhya
Skills: Python, Django, AngularJS, AWS
Skill Level: Medium
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSOC-2023)
Important Links:
Project Title: Analytics Dashboard for EvalAI Admin, Challenge hosts & Participants
Description:
As the number of compute-intensive challenges on EvalAI are increasing, we want to focus on improving the performance of our services. As a first step, we will focus on monitoring and measuring all key metrics of our services. Insights from these will allow us to efficiently utilize our infrastructure, improve uptime and reduce costs. The project will concentrate on writing REST API’s, plotting pretty graphs and building analytics dashboards to cater for all three types of users on EvalAI i.e. admin, challenge hosts and participants. The analytics will help challenge hosts to view the progress of participants in their challenge for instance, comparing the trends of the accuracy from participant submissions over the period of time, etc. The final goal is to provide users with several analytics so as to display their progress on the platform and utilize the resources efficiently to reduce costs.
Deliverable:
EvalAI Admin -
Challenge Host -
Participants -
Mentor: Shekhar Prasad Rajak @Shekharrajak, Rishabh Jain @RishabhJain2018
Skills: Angular 7, Django, Django Rest Framework, D3.js
Skill Level: Medium
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSOC-2020)
Tutorials:
Important Links:
Project Title: Redesigning CloudCV using React + Django.
https://github.com/Cloud-CV/CloudCV
Description: CloudCV main website has a lot of legacy code that needs to be cleaned and needs a complete revamp. The goal of this project is to use best practices for deployment + operations and support all the other infrastructural needs for other projects like EvalAI, IDE, CV-fy etc.
Deliverable:
Mentor: Deshraj Yadav @deshraj / Harsh Agrawal @dexter1691
Skills: React, Python, Django
Skill Level: Medium
Get started: Take a look at our issues on other Projects. Try to fix some (note that there are some issues labeled with GSOC).
Project Title: Analytics dashboards for challenge hosts and participants
Description:
This project will involve writing REST API’s, plotting relevant graphs and building analytics dashboards for challenge hosts and participants. The analytics will help challenge hosts view the progress of participants in their challenge -- for instance, comparing the trends of the accuracy from participant submissions over the period of time. Participants will be able to visualize the performance of all of their submissions with time and their corresponding rank on the leaderboard. The final goal is to provide users with several analytics to track their progress on the platform.
Deliverable:
For challenge host -
For participants -
Mentor: Rishabh Jain (@RishabhJain2018), Ram Ramrakhya (@Ram81)
Skills: Angular 7, Django, Django Rest Framework, D3.js
Skill Level: Medium
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSOC-2020)
Important Links:
Project Title: Admin Tools Enhancement and Cost Optimization
Description: The goal of this project is to focus on improving admin experience on EvalAI, as well as target efficient cost-reduction for maintaining EvalAI.
One of the primary focuses will be on enhancing the existing automation of the cancellation of submissions which have expired messages on the SQS queues. The second focus will be identifying underutilized/overutilized ECS instances using AWS health metrics to automatically determine the required compute. Other improvements will include admin actions on Django administration for starting/stopping/restarting the EC2 instance workers, and providing automated deletion of code-upload infrastructure on challenge un-approval.
These features, along with others mentioned in the deliverables, will make EvalAI administrative experience seamless and will also save costs in the longer run.
Deliverable:
Mentor: @gchhablani, Rahul Singh, @gautamjajoo, @RishabhJain2018
Skills: Python, Django, AngularJS, AWS
Skill Level: Medium
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSoC-2024).
Important Links:
Project Title: New frontend for EvalAI based on Angular 5
Description:
EvalAI’s current frontend is setup using Angular 1 which is not maintained by the community actively. Angular in the later versions support really nice features like better SEO, client-side rendering, etc. We want to migrate the current codebase in Angular 5 with a new design and achieve feature-parity. The first half of the summer will focus on adding the existing features from the older version with a new UI, while the latter half will focus on building an exhaustive analytics platform for challenge host and participants. The tasks will also include adding the UI for hosts and participants for reinforcement learning based challenges.
Deliverable:
Mentor: Mayank Lunayach @lunayach, Shekhar Prasad Rajak @Shekharrajak Shekharrajak,
Shivani Prakash @shivaniprakash95 (Design Mentor) , Rishabh Jain @RishabhJain2018
Skills: Angular 5, HTML, CSS, Typescript
Skill Level: Medium
Get started: Try to fix some issues in EvalAI (note that there are some issues labeled with GSOC-2019)
Tutorials:
a) Angular Tutorial
b) Angular Basic Application
Important Links:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.