jk0n / tensorflow-image-recognition-chrome-extension

Chrome browser extension for using TensorFlow image recognition on web pages

tensorflow-image-recognition-chrome-extension

Chrome browser extension for using TensorFlow image recognition on web pages

This is a simple test on how to use TensorFlow.js image recognition in Google Chrome extension. This extension is intercepting all image fetch requests made by the browser and pushing them to TensorFlow pretrained ImageNet model (mobilenet_v1_0.25_244) to recognize items in images. The model is downloaded when the extension is started. After that it will start automatically modifying IMG element title (mouse hover text) html attribute to display image URL, original title and prediction results.

It will only run the recognition if width or height of the image is larger than 128px. It fails to update the title sometimes when there is some fancy lazyloading module (or some other js manipulation) used on page or the images are embedded (data:image/png;base64, ...). You can inspect the background page view (on chrome extensions page) to see more information about what is happening behind the scenes.

How to try it?

git clone https://github.com/JK0N/tensorflow-image-recognition-chrome-extension.git

cd tensorflow-image-recognition-chrome-extension/

npm i

npm run build

• Open Google Chrome extensions page: chrome://extensions/

• Enable developer mode

• Click [LOAD UNPACKED]

• Select tensorflow-image-recognition-chrome-extension/dist/ -folder!

• Hover over images on web pages to display image recognition details.

Examples

Lion fish

Hot dog

tensorflow-image-recognition-chrome-extension's Issues

npm build problem

Hi!

On windows when running npm run build it will display an error stating that cp is not an internal or external command, it can be fixed by replacing cp with copy in the package.json however it still outputs this error.

The syntax of the command is incorrect.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] copy: "copy src/content.js dist/src/ && copy src/imagenet_classes.js dist/src/"
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] copy script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\07088\AppData\Roaming\npm-cache\_logs\2018-09-25T10_31_01_339Z-debug.log
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] build: "parcel build src/background.js -d dist/src/ -o background && npm run copy"
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\07088\AppData\Roaming\npm-cache\_logs\2018-09-25T10_31_01_432Z-debug.log

GYP error during build using npm i

gyp ERR! configure error
gyp ERR! stack Error: Command failed: C:\Anacoda3\python.exe -c import sys; print "%s.%s.%s" % sys.version_info[:3];
gyp ERR! stack   File "<string>", line 1
gyp ERR! stack     import sys; print "%s.%s.%s" % sys.version_info[:3];
gyp ERR! stack                                ^
gyp ERR! stack SyntaxError: invalid syntax
gyp ERR! stack
gyp ERR! stack     at ChildProcess.exithandler (child_process.js:294:12)
gyp ERR! stack     at ChildProcess.emit (events.js:189:13)
gyp ERR! stack     at maybeClose (internal/child_process.js:970:16)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)
gyp ERR! System Windows_NT 10.0.17134
gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
gyp ERR! cwd D:\GitHub test\tensorflow-image-recognition-chrome-extension-master\node_modules\deasync
gyp ERR! node -v v10.15.1
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok
Build failed
npm WARN [email protected] No repository field.
npm WARN [email protected] license should be a valid SPDX license expression
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules\fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"win32","arch":"x64"})

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `node ./build.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\Shafay Haseeb\AppData\Roaming\npm-cache\_logs\2019-02-23T22_18_51_957Z-debug.log

Cannot use import statement outside a module error when loading chrome extension

Hi, I copied the manifest.json file out of dist into the root folder and load the whole folder unpacked into chrome. (I cant load just the manifest in the dist folder, its giving me other errors).

But now, I am getting this error below regarding not able to import statement outside of module. Is this a new thing with chrome extensions?

Uncaught SyntaxError: Cannot use import statement outside a module
Context
_generated_background_page.html
Stack Trace
src/background.js:1 (anonymous function)

import 'babel-polyfill'; <--- this line is highlighted indicating its the cause of the error
import * as tf from '@tensorflow/tfjs';
import { IMAGENET_CLASSES } from './imagenet_classes';

const MOBILENET_MODEL_PATH = 'https://storage.googleapis.com/tfjs-models/tfjs/mobilenet_v1_0.25_224/model.json';
const IMAGE_SIZE = 224;
const TOPK_PREDICTIONS = 10;

class BackgroundProcessing {

Add license

Hi, I'd like to use your work however it's not clear under which license this work is open sourced. Having permissive license such as Apache V2 would be great. You can just add from Github's templates: https://help.github.com/articles/adding-a-license-to-a-repository/