cogentlabs / cloud-vision-explorer

Google Vision API demo

Google Cloud Vision API Explorer

Demo Site (Chrome v50 or later only)

About Cloud Vision Explorer

Quick start

Copy src/javascripts/config.js.sample to src/javascripts/config.js and setup the variables.

Run npm install.

Run npm start and open http://localhost:3000/webpack-dev-server/index.html.

If you can't stand the banner, open http://localhost:3000/ instead.

Deploying

Run npm run build, then upload the contents in build/prod folder.

Custom

See Wiki

cloud-vision-explorer's Issues

Generate sample JSON result like Cloud Vision API's one

• just 1000 labels+scores is enough ( by Remi)

UI: add simple sidebar (for the next alpha version)

Download images from Wikimedia

put them on GCS

Giving name to each cluster

T/O

Word 2 vec processor

Generate image batch analyzer

Sending to Vision API and store its result as JSON for each image on GCS
(this is the one you're working on now)

Provide an easy way to find/zoom into a cluster

It's a request from Kaz. My ideas

• Labels for clusters, and zoom into the cluster by clicking on the labels
• Floating text-labels in the galaxy

@tjorta @nickjanssen Please consider.

UI Design - Modify and cleaning

Setup a Cloud SQL instnace

To store results from Vision API

Get 100k results from Vision API and merge them into one big JSON file

Define image-metadata format

It contains result from Vision API.

3D: add click events for nodes

So that when you click a node, an event can be sent to be caught by @urelx

Create a simple sidebar on UI for users

Script to load metadata/thumbnail onto Cloud SQL

As a cache

Modify SQL schema

Eliminate x, y and z as they are populated in output of the clustering program.

Open GCS HTTP endpoint for image

https://github.com/reactivekk/cloud-vision-explorer/wiki/Image-URL

Create websocket API for image-metadata

Based on https://github.com/reactivekk/cloud-vision-explorer/wiki/Backend-Server-API

3D: stress test 100 000 different images

Resize Images

Please resize images along the requirements, such as size < 1MB.
For unsupported formats of VIsion API, convert them to JPEG if it's easy one.

Let me know when you're ready. Let's discuss in detail

Move the set of tiny scripts to github

T/O

Sanity check models

Write a Dockerfile to build an image

https://github.com/reactivekk/cloud-vision-explorer/wiki/Building-Docker-Image

Create an entrance page

Create websocket API for thumbnail

Based on https://github.com/reactivekk/cloud-vision-explorer/wiki/Backend-Server-API

app: setup client socket connection

3D: render 100000 particles

Find a way to render 100000 particles at 60FPS

Build an nginx docker

with

• Websocket upgrade support
• gzip compression
• connectivity to the app

Install gcsfuse and mount to have access to the images locally

Convert image to JPEG

T/O

Clustering 3D space. Tensor Flow?

Maybe using Tensor Flow

https://codesachin.wordpress.com/2015/11/28/self-organizing-maps-with-googles-tensorflow/
https://codesachin.wordpress.com/2015/11/14/k-means-clustering-with-tensorflow/

3D: show images up when zooming in

Define the data format for datapoint.json

It containts

• X, Y, Z
• Color
• Image ID
• Label (?)

Dimensionality Reduction using t-SNE

Request pulled to lvdmaaten/bhtsne#9

Create DataProc environment for classification

API format for image-metadata

on server

Filter possibly-adult/violence containing image

Those images should be filtered before giving it to front-end.
Let's discuss how we proceed

Create an instance or Pod for running frontend application

Make thumbnails

A script to generate thumbnail of the images from wikimedia. The script

• takes argument for size of thumbnails (like x=50px, y=50px)
• output format should be JPEG
• (optional) output to GCS using gsutil

Create an environment to run TensorFlow docker container

Draw sketch of the UI

@nickjanssen please collaborate with @tjorta
This has to be done by end of 7th

Spin up a GCE instance as a Docker base

Update "Systems Architecture" in the doc

Please update the doc based on today's discussion.

https://docs.google.com/document/d/1D4DRVvSpQ2Fv4uWS_HKA-3jvbBHTEy_0VyzSBYIcQfc/edit#heading=h.uqzadtf94274

Spearmint - Hyper Parameters

Create a GCS bucket for everything

3D: further improve visuals

Some ideas:

instead of circle images, actually draw circles using a pixel shader
better background instead of just black
[x] try different blending modes of lines/points
glow effects
[x] improve line thickness/opacity when going nearer
[x] slow rotating effect
[x] use random seed to be in control of color
fix z-fighting (not sure why this happens)
[x] mousehover effect over images
loading animation when starting webpage

Define schema on Cloud SQL

T/O

Remove small images and thumbnails

Maybe <5KB. Need to check with eyes.

Setup 3D UI scaffold

load all the thumbnail / image-metadata onto RDB

As soon as all the files are populated under vision

Rename images

Rename them to sequential number or ID-like string. There are so many non-alphabet, so want to avoid unexpected behaviour.

This should be done prior to #10