rishit-dagli / greenathon-plant-ai

Identify Diseases in Plants☘️ with Machine Learning on the web using TFJS

Greenathon-Plant-AI🌱

This is a web application🌐 which could help easily diagnose diseases in plants 🌱 using Machine Learning all on the web, powered by TensorFlow JS.

You can use and test the latest web app from below 👇

About 🔮

• Creates a TensorFlow Model for identifying plant diseases
• Creates optimized TensorFlow JS Models
• Uses TensorFlow JS to perform inferences on-device
• Creates a fully functional web app using React
• Exposes a hosted API built with TensorFlow Serving for inferences from the TensorFlow Model

UI Design 🎨

Click to View Plant AI web app Design from below

About the Notebooks

plant-diseases-training.ipynb

This notebook contains the code to train a model on the PlantVillage dataset to identify diseases from plant images. Here we provide a subset of our experiments on working with this data. Finally we export our model as a TensorFlow SavedModel.

plant_diseases_TFJS.ipynb

This notebook shows the the process of converting the TensorFlow SavedModel we built in the prequel notebook to the TFJS format for the Plant AI model. It also shows performing optimizations on this.

plant_diseases_TFLite.ipynb

This notebook shows the the process of converting the TensorFlow SavedModel we built in the prequel notebook to the TF Lite format for the Plant AI model.

Run Locally

To get up and running with this web-app, run the following commands, make sure you have Node.js installed. This runs the app in development mode:

git clone https://github.com/Rishit-dagli/Greenathon-Plant-AI # or clone your own fork
cd Greenathon-Plant-AI
npm install
npm start

Your app should now be running on localhost:3000 🚀.

Lint ✅

This project uses GitHub Super Linter which is Combination of multiple linters to install as a GitHub Action.

Following Linters are used internally by super linter (enabled for this project):

• JavaScript: eslint
• CSS: stylelint
• HTML: HTMLHint
• JSON: jsonlint
• YAML: YamlLint

Want to Contribute 🙋‍♂️?

Awesome! If you want to contribute to this project, you're always welcome! See Contributing Guidelines. You can also take a look at Greenathon-Plant-AI's Project Status Tracker for getting more information about current or upcoming tasks.

Want to discuss? 💬

Have any questions, doubts or want to present your opinions, views? You're always welcome. You can start discussions.

Contributors

• Rishit Dagli
• Rishabh Singh
• Rucha Yagnik

Citations

@misc{hughes2016open,
      title={An open access repository of images on plant health to enable the development of mobile disease diagnostics}, 
      author={David. P. Hughes and Marcel Salathe},
      year={2016},
      eprint={1511.08060},
      archivePrefix={arXiv},
      primaryClass={cs.CY}
}

License

Copyright 2020 Rishit Dagli

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

greenathon-plant-ai's Issues

Add TF 2 Saved Model and TFJS Model for Disease Identification

After working on adding the training scripts for this (#3 ) it would make sense to upload the TF 2 SavedModel and the TFJS models so others from the community could easily make use fo them.

Unhandled Rejection (TypeError): Failed to fetch

Describe the bug
Error when doing

let response = await fetch('https://coverimages.blob.core.windows.net/plantai-tfjs-model/class_indices.json');

To Reproduce
Steps to reproduce the behavior:

1. Run npm start
2. Click on "Try now"
3. See the error

Expected behavior
Idk, this is the first time I'm using this

Screenshots

Desktop (please complete the following information):

• OS: [CHrome OS]
• Browser [chrome]
• Version 108.0.5359.172 (Official Build) (64-bit)

Additional context
Add any other context about the problem here.

Static Web App Development

Create a static web application in React, that covers the designs for the Plant-AI project.

Demo link not working

Describe the bug
Demo link doesn't work

To Reproduce
Steps to reproduce the behavior:

1. Go to www.plant-ai.tech/
2. See error

Expected behavior
I haven't seen the website ever, but the expected behavior is a website.

Screenshots

I don't think it matters which device.

Training scripts to identify plant diseases from images

One of the major aspects of this project is to be able to identify plant diseases from images and we want to create a model for doing so using the data collected and made public in the paper An open access repository of images on plant health to enable the development of mobile disease diagnostics by Hughes et al.

Minor UI Bugs

After #17 , minor UI bugs were found:

• Fix "Learn More" Card text vertical alignment
• Adjust "Learn More" Card image width
• Result Page Overflow issue in mobile devices
• Irregular image width issue in the "Learn More" section

Model doesn't predict the correct label even when the input is from the dataset through Phone's Camera

Describe the bug
I am running the model on React Native. I input the image taken from the phone's camera into the model, and almost always get an incorrect label. The Input itself is pictures taken from the dataset via my Phone's Camera.

Steps to Reproduce

1. I open the picture of the leaf on my laptop screen (from Kaggle)
2. I take its picture from the phone's camera.
3. Output is incorrect class index.

Screenshots
I am attaching the inputs to the model as it is.

Input 1

Input 2

Output of Input 2

This should be "32": "Tomato___Septoria_leaf_spot", not "36": "Tomato___Tomato_mosaic_virus".

Am I doing something wrong? Is the input to the model wrong? I am very new to ML, so I have very little idea as to how I can improve the classification. Would other models yield the same incorrect labels?

Smartphone (please complete the following information):

• Device: Samsung Galaxy A52
• OS: Android 12
• Browser: Nope, Bare React Native

Code
`
import React, { useEffect, useState } from 'react';
import { Image, StyleSheet, Alert } from 'react-native';
import * as tf from '@tensorflow/tfjs';
import { fetch, bundleResourceIO, decodeJpeg } from '@tensorflow/tfjs-react-native';
import * as jpeg from 'jpeg-js';
import * as FileSystem from "expo-file-system";

export default function CaptureResult({ navigation, route }) {
const [tfReady, setTfReady] = useState(false);
const [diseaseDetector, setDiseaseDetector] = useState("");

async function loadModel() {

    await tf.setBackend('cpu');
    await tf.ready();
    setTfReady(true);
    console.log("[+] TF Ready!");


    console.log("[+] Loading custom Disease detection model")
    const modelJson = await require("../../assets/model/model.json");
    const modelWeight = await require("../../assets/model/group1-shard.bin");
    // const diseaseDetector = await tf.loadLayersModel(bundleResourceIO(modelJson, modelWeight));
    const diseaseDetector = await tf.loadLayersModel('https://coverimages.blob.core.windows.net/plantai-tfjs-model/model.json');
    console.log("[+] Model Loaded!");

    setDiseaseDetector(diseaseDetector);
}




const getDisease = async () => {


    const fileUri = `file://${route.params.photo}`;
    console.log("[+] Retrieving image  :" + fileUri)
    const imgB64 = await FileSystem.readAsStringAsync(fileUri, {
        encoding: FileSystem.EncodingType.Base64,
    });


    const imgBuffer = tf.util.encodeString(imgB64, "base64").buffer;
    const newData = new Uint8Array(imgBuffer);


    let offset = tf.scalar(255)
    let imageTensor = decodeJpeg(newData).resizeNearestNeighbor([224, 224]).toFloat().expandDims(); // transforms byte array into 3d tensor
    let imageTensor_scaled = imageTensor.div(offset)
    let prediction = await diseaseDetector.predict(imageTensor_scaled).data();

    console.info(prediction);
    let predicted_class = tf.argMax(prediction)
    let class_idxs = Array.from(predicted_class.dataSync());
    console.log(class_idxs)
    let class_idx = class_idxs[0]
    console.log(class_idx) // <--- THIS USUALLY GIVES '7', which I match from the class_indices.json file, which comes out to be corn.

    console.log("[+] Prediction Completed")

}

getDisease()



useEffect(() => {
    loadModel();
}, []);

return (
    <Image source={{ uri: `file://${route.params.photo}` }} style={StyleSheet.absoluteFill} />


);

}`

Write an explanatory blog

I think it is a great idea to write a small explanatory blog about how we built this project.

---

cc: @mindninjaX @RuchaYagnik

Set up Linting

For our next code changes, it would be great to set up linting as GitHub Action to:

• help prevent broken code from being uploaded to the default branch
• follow best practices
• streamline code reviews

Website Wireframe

I have created an initial wireframe for our project.

Please have a look & share feedback along with required changes so we can start with the actual fronted build part.

Plant AI Wireframe

The wireframe is interactive and comes with basic navigation & transitions.

Add content to the "Learn More" section

Allow labels to be imported from JSON

We plan to use label convention from https://opensource.creativecommons.org/contributing-code/repo-labels/