dsgt-dlp / deep-learning-playground Goto Github PK

Given a Pytorch architecture based on what the user defines in the GUI, visualize it (ie: display visual to the user)!

Write circle CI or Github Actions build script. Identify appropriate docker image and run the appropriate commands to activate conda env and run pytest tests. setup github actions for this

Make it so that users can login in and make their own account and save the visuals and models that they created. After creating a model, a pop up will appear asking the user if they would like to share it on their Facebook or social media page or story. This will help with STEM outreach and advertising

When we train a deep learning model, train time can be very long. It would be useful to store the model + weights in what's called a .pt file. We want to give that file as an output to the user that's downloadable along with a "code snippet" they should copy and paste in their local editor to load in the .pt file.

After Iteration 1, we are able to support basic neural networks. However, that might not be interesting. For example, if we have Image Data, CNN (convolutional neural networks) generally are a good start. We should have our drag and drop playground support CNNs and we should allow users to upload image data (and have them mention that the dataset is image data) so that we build image loader and kick off the training. Shouldn't be too much elbow grease to integrate into the backend.

Layers/Activations/transforms to add: Dropout, Conv2d, BatchNorm2d

Currently, for classification, we show accuracy plots + loss plots. It would be nice to show the confusion matrix in order to see what classes the deep learning model is discerning well vs. which classes the model is not classifying very well on the frontend. We want to see if we can make our visualizations interactive. I would also like the existing plots (accuracy + loss plots to be interactive using libraries like plotly or bokeh). We also would want to show AUC/ROC plots on our frontend

Plots do not change after using the same csv file but changing number of epochs

Include description of AWS Deployment Pipeline in the form of a visual diagram. Might be helpful for the whole team to know about it

It would be neat to have a progress bar or some spinner icon saying some catchy message like "CPUs and GPUs firing up!" while the model is training. Spinner icon should not show when the visualizations, onnx file and result table are populated.

Need to generate documentation for frontend in the README.md. There are good VSCode extensions to generate documentation like I did for the file structure for the backend directory. Need to interface with @farisdurrani to understand what each component does in the frontend so that it's easier to work with (from a efficiency and maintainability standpoint)

We have email notification working with twilio sendgrid API. However, we want to use Amazon SES to handle email notification. Krishi is able to give us a way to send up to 50000 emails/day. That's great from a scalability standpoint. Incorporating this into our service/software would be very very good.

In torch, you can group layers together into one sequential module and chain them together (eg: bottleneck architecture used in resnet). a stretch goal could be to support such a thing from the user's end and render such type of architecture in torch. eg: linear, relu, linear, sigmoid could be bundled into a sequential and you can drag a bunch of those "sequentials" together to build bigger and deeper models in principle

We have SSL Certificate for our domain via Amazon Certificate Manager (ACM). Our domain can be found on AWS Route 53 and we need to set our EC2 instance/load balancer to accept traffic from HTTPS request.

For Image Data like MNIST, CIFAR-10, there has been relatively recent advancements in CNNs and deep learning for these datasets. In particular, some famous architectures include VGG16, Resnet, AlexNet, GoogleNet. These models have already been implemented in Pytorch. What would be nice is if our deep learning playground can allow for users to train these "pretrained" models on their image datasets without having to write any code (low code, no code kinda thing)

We have added support for Image Data. What would be nice is to support NLP related problems. For example, BERT, Transformers, Attention, etc are very common things within NLP. Supporting that would be great as NLP is being "deepified" nowadays.

Example type of problem where BERT is used: Question Answering (SquAd dataset is very well known within this problem domain)

sklearn has a ton of built in datasets. If the user sets default=True, we want them to be able to choose a toy dataset from sklearn to play around with. This might involve the user entering a URL to the built in dataset from sklearn to use and loading that said dataset in before the get_default_dataset() call in driver.py

We have onnx file output locally, but website renders png vis from matplotlib and result table. allow for the onnx file thats is produced locally returned to the user on the frontend. also, make sure all outputs from thr Dl mldel (png plots, result csv table, onnx file output) are downloadable.

Models such as AlexNet, GoogleNet, VGG16, ResNet etc have become very popular. To make our Deep learning playground more useful, it would be great if we can add support for users being able to select that they want to use AlexNet, GoogleNet, ResNet etc with pretrained weights.

I'm trying to run the App that was merged into main branch. I tried uploading a CSV file. However, that csv file wasn't being ingested by the backend to train the model. Instead, the iris dataset was used at default. Can someone please look into this issue ASAP? This is a high priority, must fix issue

@farisdurrani has created a custom JavaScript object which would allow us to add more "layers" that the user can select from in the Drag and Drop (right now as I'm writing this issue, the "available layers" and/or activations are Linear, Non-Linear, ReLU, Sigmoid. We want our playground to support things like Conv2D, LSTM, Pretrained models (eg: ResNet, AlexNet, ...). However, users might not fully understand the meaning behind these layers and/or activation functions. We would want to add an extra attribute to the JavaScript object that provides a description for the layer and/or activation function, link to a good resource and explain what the parameters mean so that the user can play around accordingly.

Ideal Behavior: When the user hovers over the "?" icon for each draggable layer, the information regarding the use of the draggable layer will pop up

This page will describe on the frontend what Deep Learning Playground, how to use it.

MOST IMPORTANT: Make the About Page welcoming and nicely styled and interactive for the user

Currently, our backend is a pure Python Flask service. @Dhruvpargai suggested that we can have our backend on Node and have requests from the frontend route to Node which would involve a slight migration/refactoring.

Make it so that the email notification in the backend is more general so that it can be used for functionality other than just sending images and the ONNX file.

Can someone fix this issue:

I came across this tool from this Linkedin post: https://www.linkedin.com/posts/huggingface_evaluation-machinelearning-ml-activity-6937445348592300032-jzdV?utm_source=linkedin_share&utm_medium=android_app

You all are free to read up on it for your intellectual edification

adding support for user to be able to drag various preprocessing steps into a pallette to process image data prior to model training. will require both backend and frontend enhancements

Nick did a good job with getting the wiki page up on the site. Might be good to make the styling a bit better and welcoming for the audience to read. I think showing relevant visualizations + TLDR descriptions of things like Softmax, ReLU, etc with appropriate diagrams would be good (along with links to further reading)

We have an issue with dataloaders not causing basic neural network model to not run on iris dataset.

Files:

/backend/driver.py is the driver file

This issue needs to be fixed because we just need to get a basic torch model to train properly in order for backend to work properly

Currently, our playground supports training and prototyping of deep learning models. However, to make our playground even more generalizable, we should add support for classical ML. The backend should be working and ready to go for classical ML. We just need a frontend page for "classical ML playground" on top of the "deep learning playground page" and we should be able to test our system!

On the Navbar, put github icon and when user clicks on it, they should be taken to our open source repo!

Add functionality to /backend directory (maybe create a file called datasets.py where there is a function that takes in URL to github gist and uses pandas to read that file into a csv. return that file and train/test split kicks in at /backend/driver.py and data loaders hopefully work properly to train the model)

Thinking about adding sound effect features to this tool to create a more enjoyable user experience. We could maybe have a setting icon in which users can turn this feature on and off.

We can also use the statistics collected from the users of the website to gamify the website. We can include rankings and awards. For example a reward for most models make, highest accuracy model made on this website etc. We can also include contests and a ranking board.

React App frontend where user can drag and drop model architecture (pieces that are draggable for now are Softmax, Conv2d, Linear (ie: hidden layer of NN), ReLU, Sigmoid, BatchNorm). For each of these draggable pieces, the users should be able to toggle relevant parameters (if any). For example, Conv2d parameters users can wiggle around with are in_channels, out_channels, kernel_size, stride. Given the pieces user drags and drop into the pallette/plate, drag and drop should render a string of the list of dragged pieces and that gets passed to the backend. Pay attention to the backend schema of how each layer should be read as a string (in particular, don't forget the nn before each layer). For example, nn.Linear(in_features=50, out_features=30) GOOD, but Linear(in_features=50, out_features=30) BAD

Currently, I'm not able to visualize a pytorch architecture using pytorchviz library. I was able to generate an ".onnx" file containing the pytorch model. The user needs to be notified via email or text about the .onnx file so that they can download it and open it in netron.app. We need this written in the backend (user will provide his/her email address and our backend should handle the email notification endpoint)

For training say image classification architectures with architectures like VGG16, ResNet, AlexNet, GoogleNet, these architectures can get pretty big. We need to introduce the option to use cuda if available (ie: torch.cuda().isavailable = true, then torch.to("cuda")) when we are running the dl_drive function

when we deploy version 1.0 and alpow for beta testing, it would be convenient if we could have a page for people to submit feedback or feature request and route those emails to [email protected] (talk to @avayedawadi ). adding that as a page that exists on the navbar would be convenient

It would be nice to incorporate analytics where we as the software engineers can view analytics on what pages users go on, what is being viewed, active time, etc? (pretty much google analytics). A good task would be to configure google analytics to keep track of this data especially when the deployment happens

Configure a Github Action script that will deploy our React App with backend spun up on AWS. This Github Action should happen only in the main branch and after the linux-build happens

In the Drag and Drop frontend, we support dragging layers and entering the relevant parameters by the user. However, in order to increase the user friendliness, it might be a good idea to provide some kind of "mini-documentation" when the user hovers over the "?" icon on a draggable layer. See layer_docs directory in the frontend/playground-frontend for details. We should be able to render those markdown files in our site when the user hovers over the "?" icon for each draggable layer.

@farisdurrani

Create footer component similar to nav bar that is on each page link containing contact information, links to other resources etc

Suppose that the user drags in a deep learning model architecture. If the training is successful, along with the outputs we currently have (ie: existing visualizations), it would be good to include a "code snippet" that the user can copy/paste into their editor that represents their deep learning architecture from our website. This task is a full stack task. Work off the frontend branch for this.

Have the website keep tracks of the habits of users. As well as create a timeline of this with features added. This will allow us to see how effective a new feature was in terms of increasing user engagement.

Have an eye for design? Well, DLP needs a logo to represent what it's about. I need a visually appealing logo that makes sense with our project

Dhruv wants to deploy this app using AWS Fargate + Load Balancer + ECS + ECR and string these components together. More details will be fleshed out in the design doc

Make brief Youtube video explaining our AWS deployment pipeline