microsoft / hack-workshop-lobe

Workshop for student hackathons focused on Lobe.ai

Scenario: Getting started with Lobe

Ever wish you could be alerted when your package has arrived at your door without relying on less-than-accurate delivery-tracking apps? What about needing someone else count your reps during your workout?

Lobe is here to the rescue! Lobe is a free, private desktop application that has everything you need to take your machine learning ideas from prototype to production.

In this lab, you'll build an image classification model to solve a problem that you and your team come up with together.

Prerequisites

Skills

Your team should be familiar with the following:

• Git and GitHub
  • Forking and cloning repositories
  • Creating and managing branches
• Using a terminal (e.g., Bash, PowerShell, etc.)

Hardware

⛔️ Lobe is not currently supported on Apple computers with Apple Silicon chips, such as the M1, M1 Pro and M1 Max

• A computer capable of running arbitrary code and on which you have administrative rights
• A stable internet connection (for setup and data download only)

Software

Each member of your team will also need the following software installed:

• Lobe
• Git
  • Install git on macOS
  • Install git on Windows
  • Install git on Linux
• Visual Studio Code or any another text editor or IDE (e.g., WebStorm, Notepad++, Atom, Brackets, etc.)
• Node.js
  • Install Node.js on Windows
  • Install Node.js on Linux or MacOS
• Yarn

Resources

A series of resources will be provided to help your team determine the appropriate steps for completion. The resources provided should provide your team with enough information to achieve each goal. If you get stuck, you can always ask a mentor for additional help.

Exploring the source code

The key folder for the application, apps, contains a starter web application in which to drop an exported model from Lobe. The flow of the application is as follows:

1. A user navigates to the page and is presented with the option to take a photo or upload a photo
2. User takes a photo using their device's camera or uploads a photo from their device.
3. After submitting the photo, the model will attempt to classify and tag the photo.

💡 No updates to the application code will be made during this workshop. Your team will be able to successfully complete the workshop without any experience with React. The only files your team will add are generated by the Lobe app exporting function.

Goals

Your team will obtain the starter, train the model, and use the model in locally-run web application.

1. Obtain the source code: The first step when working with any codebase is to download it. Your team's first goal will be to obtain the code from GitHub.
2. Train the model: Because we need a model to classify photos, your team will first need to train a model. For this workshop, your team will use Lobe, which will train the model for you based on photos and tags that your team selects.
3. Test the model: A key aspect of training a model is testing and improving it. Here you will take or upload photos to improve the model by fine-tuning it's training.
4. Export the model: Our webapp will only work if it has a model to run! You will use Lobe to export your model as a TensorFlow.js script and model files.
5. Run the app: Run your app! For this goal, you will run your web application on your local computer and try to classify images with it.

Where do we go from here?

This project is designed as a potential seed for future development. If you were to continue with this idea, your team could potentially:

• Deploy and run your model on a Raspberry Pi and use a camera module to take photos!
• Build a Rock, Paper, Scissors game and play against your model!
• Build a Package Detector and get notifications when a package is left at your door!