cloud-cv / evalai-ngx

Revamped codebase of EvalAI Frontend

EvalAI-ngx

Note: Development in this repository has been stopped and the code is moved to the main repository EvalAI to support mono-repo structure.

Revamped codebase of EvalAI Frontend

---

EvalAI is an open source web application that helps researchers, students and data-scientists to create, collaborate and participate in various AI challenges organized round the globe.

In recent years, it has become increasingly difficult to compare an algorithm solving a given task with other existing approaches. These comparisons suffer from minor differences in algorithm implementation, use of non-standard dataset splits and different evaluation metrics. By providing a central leaderboard and submission interface, we make it easier for researchers to reproduce the results mentioned in the paper and perform reliable & accurate quantitative analysis. By providing swift and robust backends based on map-reduce frameworks that speed up evaluation on the fly, EvalAI aims to make it easier for researchers to reproduce results from technical papers and perform reliable and accurate analyses.

A question we’re often asked is: Doesn’t Kaggle already do this? The central differences are:

• Custom Evaluation Protocols and Phases: We have designed versatile backend framework that can support user-defined evaluation metrics, various evaluation phases, private and public leaderboard.

• Faster Evaluation: The backend evaluation pipeline is engineered so that submissions can be evaluated parallelly using multiple cores on multiple machines via mapreduce frameworks offering a significant performance boost over similar web AI-challenge platforms.

• Portability: Since the platform is open-source, users have the freedom to host challenges on their own private servers rather than having to explicitly depend on Cloud Services such as AWS, Azure, etc.

• Easy Hosting: Hosting a challenge is streamlined. One can create the challenge on EvalAI using the intuitive UI (work-in-progress) or using zip configuration file.

• Centralized Leaderboard: Challenge Organizers whether host their challenge on EvalAI or forked version of EvalAI, they can send the results to main EvalAI server. This helps to build a centralized platform to keep track of different challenges.

Goal

Our ultimate goal is to build a centralized platform to host, participate and collaborate in AI challenges organized around the globe and we hope to help in benchmarking progress in AI.

Performance comparison

Some background: Last year, the Visual Question Answering Challenge (VQA) 2016 was hosted on some other platform, and on average evaluation would take ~10 minutes. EvalAI hosted this year's VQA Challenge 2017. This year, the dataset for the VQA Challenge 2017 is twice as large. Despite this, we’ve found that our parallelized backend only takes ~130 seconds to evaluate on the whole test set VQA 2.0 dataset.

Development setup

Use Docker Compose to run all the components of EvalAI-ngx together. The steps are:

1. Get the source code on to your machine via git.

   git clone https://github.com/Cloud-CV/EvalAI-ngx.git evalai-ngx && cd evalai-ngx

2. Build and run the Docker containers. This might take a while. You should be able to access EvalAI at localhost:8888.

   docker-compose up
   

3. That's it. Open web browser and hit the url http://127.0.0.1:8888. Three users will be created by default which are listed below -

   SUPERUSER- username: admin password: password
   HOST USER- username: host password: password
   PARTICIPANT USER- username: participant password: password

For deploying with Surge:

Surge will automatically generate deployment link whenever a pull request passes Travis CI.

Suppose pull request number is 123 and it passes Travis CI. The deployment link can be found here: https://pr-123-evalai.surge.sh

Code scaffolding

Run ng generate component component-name to generate a new component. You can also use ng generate directive|pipe|service|class|guard|interface|enum|module.

Code Documentation

We are using compodoc for documentation. The goal of this tool is to generate a documentation for all the common APIs of the application like modules, components, injectables, routes, directives, pipes and classical classes.

Compodoc supports these JSDoc tags.

Contributing

Please go through our Contribution Guidelines. Also go through our detailed Code Structure Guide to make the most of existing re-usable features. Finally, go through the Pull Request Template when creating your pull request.

Building and Serving the documentation

Run the following command to build and serve the docs:

npm run doc:buildandserve

Open http://localhost:8080 in the browser to have a look at the generated docs.

Build

Run ng build to build the project. The build artifacts will be stored in the dist/ directory. Use the -prod flag for a production build.

Running unit tests

Run ng test to execute the unit tests via Karma.

Running end-to-end tests

Run ng e2e to execute the end-to-end tests via Protractor.

The Team

EvalAI-ngx is currently maintained by Shekhar Rajak, Mayank Lunayach, Shivani Prakash Gupta, Rishabh Jain and Deshraj Yadav.

evalai-ngx's Issues

Improve Test Coverage

More Unit Tests need to be added to take coverage to 90%. 💻

UI Design for Get Involved page

Here is the final design. Please refer to screenshot below.

You can refer to the following image for further details on creating the page :

You can refer to the following image for further details on creating the page :
https://drive.google.com/drive/folders/1pk5qIzxfIfYJriUQ0JkiXxDWLdVQJ3cB

You can get the relevant text from the present website itself : https://evalai.cloudcv.org/get-involved

Please comment below if you want to work on this or in case of any question/suggestion.

Mentors for EvalAI-ngx Project (@aka-jain @spyshiv @shivaniprakash95 @RishabhJain2018 @deshraj) will try to respond to this thread ASAP so that you are not blocked.

Add Installation instructions using Docker

We have setup the docker for ngx but the docs are still missing. The deliverable of this issue is to add the installation instructions using docker. Please feel free to comment if you have concerns.

Command to run docker: docker-compose -f docker-compose.dev.yml up -d --build

Ahead-of-Time Compilation for production builds

More here : https://help.syncfusion.com/angular/gettingstarted/getting-started-with-webpack-aot

Why do Ahead-of-Time Compilation

• Faster Rendering
• Fewer asynchronous requests
• Smaller Angular framework download size
• Detect template errors earlier
• Better security

@aka-jain @spyshiv

Make up-arrow visible on privacy-policy page and smooth scroll

Observered Behaviour

The up-arrow is not visible on privacy-page when scrolled at the bottom of the page.

Expected Behaviour

The up-arrow should be visible and a smooth scroll effect to it.

@RishabhJain2018

Git clone statement error

I'm submitting a

(Check one with "x" from given options)

• Bug report
• Feature request

Current behavior

On using git clone [email protected]:Cloud-CV/EvalAI-ngx.git it gives an error regarding access rights

Expected behavior

Change the statement to git clone https://github.com/Cloud-CV/EvalAI-ngx

I would like to work on this

Create a separate file for all API paths

I'm submitting a

(Check one with "x" from given options)

• Bug report
• Feature request

Separate file for all the API paths.

Setup Docs for EvalAI-ngx using Github Pages

UI Design for All Challenges page

Here is the final design. Please refer to the screenshots below.

(There are two screenshots here, of the same page :
-The first one shows how the page should look if the filter tabs Ongoing and Upcoming challenges are selected, and the Side upper tag showing which one is what(either upcoming or ongoing.
-The second screenshot shows a scenario if all the filter tabs are selected then the side upper tag in each box changes, you can notice the one in the last box changes to the brown color)

**You can refer to the following image for further details on creating the page : **

You can get all the related assets for creating the page from the link below :
https://drive.google.com/drive/folders/1fhWkj6Ir7HLZzwBZoPCKyyoS-ulFdmg_

The screenshot shows the dummy text, it's just for illustration purpose. You can refer to the current page of the website to find the exact data: https://evalai.cloudcv.org/web/challenges/list

Please comment below if you want to work on this or in case of any question/suggestion.

Mentors for EvalAI-ngx Project (@aka-jain @spyshiv @shivaniprakash95 @RishabhJain2018 @deshraj) will try to respond to this thread ASAP so that you are not blocked.

Use compressed images and assets

I'm submitting a

• Bug report
• Feature request

The png resources are heavy as of now. Replace all by compressed png files before taking ngx to production 💻

Creating this issue now so we don't lose track of it.

Add a carousel for adding more organizations

I'm submitting a

(Check one with "x" from given options)

• Bug report
• Feature request

Current behavior

Expected behavior

"Who Uses EvalAI" organizations must be in a carousel format to add more organizations easily

UI Design for Login page

Here is the final design. Please refer to screenshot below.

You can refer to the following image for further details on creating the page :

You can get all the related assets for creating the page from the link below :
https://drive.google.com/drive/folders/1JsK6l1B5pSKJY2MCSMQVZYq1EwGuOx4t

Please comment below if you want to work on this or in case of any question/suggestion.

Mentors for EvalAI-ngx Project (@aka-jain @spyshiv @shivaniprakash95 @RishabhJain2018 @deshraj) will try to respond to this thread ASAP so that you are not blocked.

Docker setup is not working

I'm submitting a

(Check one with "x" from given options)

• Bug report

It's happening After this commit in evalai repo.
@RishabhJain2018 @deshraj

Remove the scroll bar after “Who uses EvalAI”

Observed Behaviour

Scroll bar is visible under Who uses EvalAI on normal screen. The scroll bar is used for medium and small screen. Find attached screenshot.

Expected Behaviour

The scroll bar should not be visible on normal screen.

@RishabhJain2018

Add Codecov support for code coverage

I'm submitting a

(Check one with "x" from given options)

• Bug report
• Feature request

Current behavior

At present, there is no support for testing code coverage.

Expected behavior

There should be a support for testing code coverage as code coverage is a measurement used to express the degree to which the source code is executed when a test suite runs.

@deshraj @aka-jain @RishabhJain2018 @spyshiv Please give suggestions.

I want to work on this issue.

Create UI for EvalAI Pages

How does the new UI look?

EvalAI team has been working on the new UI for last one month. We have come up with this mock-up of the home page (yes it takes a lot of time in planning and discussion. Special thanks to @shivaniprakash95 for taking lead on this):

Note that the second last section is still empty and we want to show twitter feed there in that section.

Deliverables

Now that we have shared the mockup design, we expect the student to have the following things ready by 27th March 2018 (GSoC Proposal submission deadline):

1. Create an HTML page with the exact same design (it's quite easy, right?). We also expect modular code structure for this particular task since the code will get merged to the master branch at the end of the day if you do a really good job at this. (Additional points if you can make the pages responsive)
2. Extended Goal: We also expect the students to think more and create UI with similar design pattern for other pages such as Challenge list page, challenge details, submission, leaderboard, login/signup etc.
3. Create a video for the same showing how the page looks.
4. Share the link to the code in

How to get started?

All the important information has been shared on this issue. We look forward to seeing some really cool pages before the GSoC proposal submission deadline. Please let us know if you have questions. Mentors for EvalAI-ngx Project (@aka-jain @spyshiv @shivaniprakash95 @RishabhJain2018 @deshraj) will try to respond to this thread ASAP so that you are not blocked.

Form input component

Need a reusable form input component that can be reused across the website for all forms.
It should be able to:

• Material design
• Check for required input elements
• Check for validations
• Display required/ validation message
• Should check for validations before the form can be submitted

import global css easily from components

UI Design for About Us page

Here is the final design. Please refer to screenshot below.

You can refer to the following image for further details on creating the page :

You can get all the related assets for creating the page from the link below :
https://drive.google.com/drive/folders/1VIEmGtKwBEOs_kzFu6lGFfnCzXm9GyB9

You can get the relevant text from the present website itself : https://evalai.cloudcv.org/about

Please comment below if you want to work on this or in case of any question/suggestion.

Mentors for EvalAI-ngx Project (@aka-jain @spyshiv @shivaniprakash95 @RishabhJain2018 @deshraj) will try to respond to this thread ASAP so that you are not blocked.

Create header component for EvalAI home page

Attaching design for EvalAI home page intro section.

Create header according to the attached design.
Follow variables.scss for styling (add new variables wherever needed).

Note: Try to make responsive header.

Let us know in case of any query.

Documentation and Refactoring

I'm submitting a

• Bug report

• Feature request

• All the UI components need a detail markdown documentation for future contributors.

• Most of the UI components currently reside in the app folder. They need to be refactored into subfolders.

This needs to be done before we take ngx to production.
(Created this now so we don't lose track of this.) 🍕

404 Page

I'm submitting a

(Check one with "x" from given options)

• Bug report
• Feature request

Need a 404 page for paths that are not found by app-router.

Update Issue and PR template

Expected behavior

Presently, Contributor has to do a lot of changes in Issue and PR template. One has to remove lots of text present already in templates and then has to edit according to issue/PR.

Observed behavior
There should be simple templates so that contributor has to just mention the bug/feature instead of editing and removing lots of text.

Custom Toasts

I'm submitting a

• Bug report
• Feature request

Expected behavior

Need custom toasts/messages for success or failure of API requests.

Please post the designs here @shivaniprakash95 🍡

cc: @deshraj @RishabhJain2018 @spyshiv @aka-jain

Fix Typo

Close bracket is missing in performance comparison.

Challenge Page

I'm submitting a

(Check one with "x" from given options)

• Bug report
• Feature request

Need UI for the Challenge Page.

@shivaniprakash95 Please share the documentation for the final challenge-page design here 🍕

cc: @RishabhJain2018 @spyshiv @aka-jain @deshraj

Add information about EvalAI in README.

Information to add in README file:

• EvalAI Logo
• About EvalAI
• Goal
• Performance comparison

@deshraj @aka-jain @RishabhJain2018 Can I work on this?

Create common button styles for EvalAI

Attaching design of EvalAI home page for the reference.

Create buttons as used in this design (white and transparent). basically, you have to add common styles for these buttons (modular scss).

Let us know in case of any query.

Add scroll bar for navigating through questions on privacy-policy page

Observed Behaviour

There is a scroll bar for navigating through questions on Privacy-Policy page which is not functional.

Expected Behaviour

Add a functional scroll bar for navigating through questions.

@RishabhJain2018

Fix Docker setup

The docker setup mentioned in the setup instructions isn't working and it fails with the error log:

"cp: cannot stat '/code/settings/dev.sample.py': No such file or directory
ERROR: Service 'django' failed to build: The command '/bin/sh -c cp /code/settings/dev.sample.py /code/settings/dev.py' returned a non-zero code: 1

Add footer component for EvalAI

@aka-jain @spyshiv

Fix failing travis build

The master branch Travis build is failing due to this reason: https://travis-ci.org/Cloud-CV/EvalAI-ngx/jobs/501229397#L549

Setup Travis CI

UI Design for Challenge Detail page

Here is the final design. Please refer to screenshots below.

For Overview:

For Evaluation :

For Phases :

You can refer to the following image for further details on creating the page:
Documentation for Overview :

Documentation for Evaluation :

Documentation for Phases :

** NOTE:** Please note that other tabs such as Participate and Leaderboard are going to have a similar structure as in the current website, except for the color theme, etc (This will be in sync with the current design)
You can refer to the following link for further details on creating the page :
https://drive.google.com/drive/folders/11z6ME50N4dQ0O5qa3ue9TYMzIVqEe-GO

You can get the relevant text from the present website itself: https://evalai.cloudcv.org/contact

Please comment below if you want to work on this or in case of any question/suggestion.

Mentors for EvalAI-ngx Project (@aka-jain @spyshiv @shivaniprakash95 @RishabhJain2018 @deshraj) will try to respond to this thread ASAP so that you are not blocked.

UI Design for Contact Us page

Here is the final design. Please refer to screenshot below.

You can refer to the following image for further details on creating the page :

You can refer to the following image for further details on creating the page :
https://drive.google.com/drive/folders/12WqC1rHA2rakJQLr-FTjZnn4lj0uPXTv

You can get the relevant text from the present website itself: https://evalai.cloudcv.org/contact

Please comment below if you want to work on this or in case of any question/suggestion.

Mentors for EvalAI-ngx Project (@aka-jain @spyshiv @shivaniprakash95 @RishabhJain2018 @deshraj) will try to respond to this thread ASAP so that you are not blocked.

Auto deployment of every PR using Surge

I'm submitting a

(Check one with "x" from given options)

• Bug report
• Feature request

Current behavior

At present, there is no auto deployment of Pull request.

Expected behavior

There should be auto deployment of PR which will ease the review process.

Mentors, Please give your views on this.

I want to work on this.

Signup Url not found on signing up

I'm submitting a

(Check one with "x" from given options)

• Bug report
• Feature request

Current behavior

When a user fills the required fields on the signup form on auth/signup and clicks the submit button, it doesn't proceed forward, checking the console shows that the endpointsService is not present on the self object.

Expected behavior

Upon clicking the signup button after filling the required fields, it should send request to backend without any errors.

Steps to reproduce

1. Launch the application
2. Click on the Signup button to navigate to signup page
3. Fill the required fields
4. Click the signup button
5. It won't proceed further as it should

Related screenshots

Other information

System information

Ubuntu 18.04 LTS
Node.JS: v8.12.0

Numerous build errors prevents the project from compiling

• [x ] Bug report
• Feature request

Current behavior

Numerous build errors prevent the project from compiling. Some include;

• @angular/[email protected] requires typescript@'>=2.4.2 <2.7.0' but 2.9.2 was found instead.
  Using this version can result in undefined behaviour and difficult to debug problems.
• ERROR in src/app/app.module.ts(116,30): error TS2339: Property 'forRoot' does not exist on type 'typeof NgxTwitterTimelineModule'.

Expected behavior

After running ng serve project should compile with without errors

Steps to reproduce

Run ng serve

Related screenshots

System information

OS: Kali linux
Npm: 6.7.0
Node: v11.8.0

Addition of COC & Contributing Guidelines

I'm submitting a

(Check one with "x" from given options)

• Bug report
• Feature request

Expected behavior

Addition of Code of Conduct & Contributing Guidelines

@deshraj @RishabhJain2018 @live-wire

JS Doc integration

Feature request

• Feature request

Need JS Doc integration for the technical documentation of the functions and the project.

TravisCI Build fails while waiting for an input to surge.sh

I'm submitting a

• CI enhancement

Current behavior

Travis CI builds fail after 10mins as Surge expects an email input which it doesn't get.

Steps to reproduce

Create any Pull Request (or a new commit in any of the existing Pull Requests)

Useful link:
https://surge.sh/help/integrating-with-travis-ci

Failing e2e tests due to chrome version error on travis

Research and add FAQ page for end users

Deliverables

In continuation with Cloud-CV/EvalAI#1359

• Add a new page for FAQ (design same as Privacy-policy page).
• Research some FAQ for end users (Maybe discussing the questions would be better before adding). Some of them could be :

1. What is EvalAI?
2. What kind of challenges does EvalAI support?
3. Can I run EvalAI on my private server?
4. How to host a challenge on EvalAI?
5. How to participate in a challenge in EvalAI?
6. What is the best way to contact EvalAI team?
7. I am interested in contributing. Where should I start from?

@RishabhJain2018

Footer component for all pages

Feature Request

• Feature request

Need a Footer Component that can be reused on all the pages.
(See the footer on home page in #46 for reference)

Add tool to fetch pull request in order to test it on local machine.

Deliverable: It should be a script which fetches a pull request and then switches the user to the new branch.

Create UI for privacy page

UI Design for privacy page

Here is the final design. Please refer to screenshot below.

---

Please comment below if you want to work on this or in case of any question/suggestion.

@shivaniprakash95 please share the details about assets and content that will be needed for this.

Implement the authservice

Implement the isLoggedin function

• check the presence of token in localstorage

Need a Loader animation for all API calls.

I'm submitting a

(Check one with "x" from given options)

• Bug report
• Feature request

Need a loader animation for all API calls.

Add font family.

Add Open Sans font family for global styling.
Steps for creation:

• Create a new file fonts.scss for this addition
• Import it in base.scss.

Here is the reference link https://fonts.google.com/specimen/Open+Sans
Please let us know in case of any query.

Update Readme

The current readme should be updated by including badges. Following list should be added:

1. Codecov
2. Travis-CI

Also modify the last line of readme with
EvalAI-ngx is currently maintained by [Akash Jain](http://www.jainakash.in/), [Shiv Baran Singh](http://www.shivbaran.in/), [Shivani Prakash Gupta](https://www.behance.net/shivaniprakash19), [Rishabh Jain](https://rishabhjain2018.github.io/) and [Deshraj Yadav](https://deshraj.github.io)