mwater / mwater-android-app

An Android App that snaps a picture of a petrifilm, counts the bacteria and posts the counts on a server for the mWater project

THIS VERSION IS NOW SUPERCEDED BY VERSION 3

See https://github.com/mWater/app-v3

---

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.

mwater-android-app's Issues

Decide whether we need a database or not for this iteration

PetriGell Button: GridView to select water source to take the incubated sample picture

we will make a grid view of pictures with their sample code underneath so that the field worker can simply touch the image of the source that they want to link to the picture of the sample after it has incubated.

Reasoning:
if we geocode only the petri dish, the field worker may be 3 villages over from the contaminated water. having two images, one take at the source, and then coded, and the correlated with the incubated sample's image we will get accurate geocoding of the water source, and a potential measurement of other interesting factors for the researcher such as data quality or degradation if we find that samples which are incubated far from the source are less accurate etc.

Remove hard-coded paths from Eclipse project

Replace with environment variables.

Port the python image processing to c++

Probably the simplest and most robust way to get the image processing to run on the android is to port our python prototype to C++, and embed it in the app like the sample OpenCV android apps

Set up simple server to receive POST

To test the POST produced by the Android client we can either setup a simple WebPy or Node.js server

Sketch out application UI

Hi John (@rocketboy76)

I made a very rough sketch of the application workflow and possible UI as I understand it. I'd appreciate you having a look at it and giving some feedback when you have a minute.

https://github.com/downloads/AndroidImageProcessing/AndroidBacteriaImageProcessing/Android%20App.png

Thanks, Clayton

P.S. @adrianao, you might want to have a look too!

Android Client Iteration 2 - add infrastructure for NDK+OpenCV

Take an existing working OpenCV in Android code (probably one of the tutorials) to set up the infrastructure needed for @pakdel's C++ port of the Image Processing script.

Normalizing the ackground

Narrow down the histogram of the background.

def determinebackground(img, mask):
    # Get histogram
    histr=gethist(img, mask, 2)
    histg=gethist(img, mask, 1)
    histb=gethist(img, mask, 0)
    return (bestval(histb)[0], bestval(histg)[0], bestval(histr)[0])

https://github.com/AndroidImageProcessing/AndroidBacteriaImageProcessing/blob/master/data/stage3.png

Link up the image processing .cpp to a java class

We need to integrate the .cpp into the jni area of the Android app. @pakdel maybe we can get together tuesday after work? (plus anyone else who wants a tutorial on setting up the NDK)

Android Content Provider to hold sample data until upload

We need a simple db (wrapped in a content provider) with one table to store pending image metadata with a flag (uploaded/notuploaded) that the Queue activity will use to determine which samples need to be uploaded.

The rows will be metadata per image capture.

timestamp, latitude,longitude, filename, sampleuniqueidentifier,userwhotooktheimage,userwhotookthesample(?)

This info will be written to the DB by the TakePicture Activity, and read+written by the ServerSync activity.

icon

Obtain official 3M training samples

We are using the 3M E.coli/Coliform Count Plate. The interpretation guide is available at:

http://multimedia.3m.com/mws/mediawebserver?mwsId=66666UuZjcFSLXTt4XMa4xTaEVuQEcuZgVs6EVs6E666666--&fn=70-2008-4574-4.pdf

It has a few low-resolution samples, but we really need more. 3M must have a collection of reference image samples if they developed a plate reader. Could we contact 3M and ask for some images to make a training set to tune and evaluate the image processing algorithm?

Draw slider in OpenCV showing user if they are in or out of focus

I work on an app that uses barcode scanning and what we found is that really cheap phones, like the ones you want this to run on, won't have camera's with autofocus capability.

For our app, we just disable the barcode scanning capability in our app when we detect the lack of autofocus. But it sounds like it might be critical to your project. Anyway, it's something to look into and i would definitely try to get a hold of one of the prospective devices.

Locate sample image processing code files which have already be integrated in the Android environment

We have a few projects which use the OpenCV in the Android NDK enviroment, lets pull out those code files so we know what we will be tweeking once we know enough OpenCV to code to its API

Find colonies

Find and count bacteria colonies.

def filtercolonies(img):
    red=numpy.compress([False, False, True], img, 2)
    red=192-red.clip(0,192)

    # Remove green and blue
    green=numpy.compress([False, True, False], img, 2)
    blue=numpy.compress([True, False, False], img, 2)
    greenblue=numpy.minimum(192-green.clip(0,192), 192-blue.clip(0,192))+64
    col=(greenblue-red).clip(64,255)-64
    col=255-((col*3).clip(0,255))
    return col

def findcolonies(col):
    thrs=cv2.adaptiveThreshold(col, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY_INV, 5, 30)
    cv2.findContours(thrs, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE)
    cv2.imshow('img', thrs)
    cv2.waitKey()

https://github.com/AndroidImageProcessing/AndroidBacteriaImageProcessing/blob/master/data/stage5colonies.png

Sample pictures and their official counts

@grassick
We need some pictures with their official counts to adjust our image processing.

Currently, we have some pictures that are accompanied with a document: "Comalapa Water Sampling Results.docx". Therefore, I tried to find the relation between pictures in hand and that document.
Please take a look at AndroidBacteriaImageProcessing/docs/README.md [https://github.com/AndroidImageProcessing/AndroidBacteriaImageProcessing/tree/master/docs]

Contribute Image Processing on Android Client stack to ODK

Get metadata for POST upload to accompany the petri gel image

The information posted:

• the user authentication
• the geo-location
• the image of the water source.
• The description of the water source

Other information once the post is done

• the date and time of image capture

connect with the DC/nairobi working on water flow pattern recogntion

@grassick Dan left us a message on RHOK back in Jan
Jan 31
Hi there,

I just came accross the update on the water test pattern recognition prototype. Very exciting to see this problem's getting attention. Would love to hear what the status of things are with the team today and what help is needed.

I am part of the team that organized the water hackathon from the World Bank and I think there's two ways we might be able to help. First, if you still need help on the ushahidi/reporting side, there's a killer team out of London Water Hackathon that has the skills/enthusiasm. Second, if there's any interest, we can help get the word out // introduce you to people who might like to deploy when that's feasible.

And I have to ask.. how hard is the VC part? There's a team here in DC/nairobi working on water flow pattern recogntion problem that is still struggling. Would be great make the link.

Cheers

Dan

Sync Button: query the un-uploaded images

The ServerSync activity is launched by a button by the user when they have wifi (or 3g) connectivity. It cycles through the db, getting rows that haven't been uploaded and launches an intent for each to the UploadImageService.

This activity should be pretty straight forward to code. If the connectivity failes then the UploadImageService simply wont tag the images as uploaded and they will be retried later.

Focus distance requirements

The petrifilm center is only 5cm in diameter. If the camera is 2MP, the focus distance to pixels wide are as follows:

5cm = 1200 pixels
10cm = 600 pixels
15cm = 400 pixels
20cm = 300 pixels.

We need to check what the minimum number of pixels is for accurate colony counting and that the camera has a macro mode capable of doing such close focusing.

Remove the yellow grid

Find and remove the yellow grid on the petri-film.

def removeyellow(img):
    h, w = img.shape[:2]

    # Get max histogram value
    for x in xrange(0,w):
        for y in xrange(0,h):
            img[y][x][0]=max(min(img[y][x][1],img[y][x][2]),img[y][x][0])

https://github.com/AndroidImageProcessing/AndroidBacteriaImageProcessing/blob/master/data/stage4.png

Find out what DB Ushahidi is using

Probably MySQL? If so we can create a table for our data..

YouTube demo the user stories

Create a detail user story for the client for:

• Create a water source entry
• Create a water sample entry
• Sync with the server

Find a non-hacky way for thresholding and filtering out background

For the moment, adaptiveThreshold is quite hacky.

Code Review for GPS detection in takepicture

Make transparency copies of petrifilm

@rocketboy76 We'd like to print a transparency copy of a petrifilm for testing. To do that, we need to know the standard size that they are.

1. What is the diameter of the center circle?
2. What is the overall size of the sheet?

Android UploadImageService

We have the existing multipart form upload via POST, just need to create the extras and pass them around to open the pictures and upon upload succes, flag that row as uploaded

Jan 25th RHOK meeting

@DrDub @adrianao @peristeri @pakdel @sleepokay @baharsateli @cesine

Shall we meet up on January 25th at Notman House from 6-8pm? There is a rhok followup then...

I've been productive today... We have a new application flow in the app, a new circle detection algorithm that actually works and in-app camera capture. Give it a try! I also altered the structure of the eclipse project to make it easier to code and debug.

Cheers,

Clayton

Accept a request to create a new user

The user is a staff that will have access to:

• Posting a location of a water sample
• Posting the image of petri dish

The user authentication is required.
Meta-data can be collected such as list of water samples, level of reliability, communication info

Use SharedPreferences to store state between app launches

set up shared preferences to hold state like the output dir, the server url the experimenter id of the person who is taking the water samples etc.

Faucet/Tap Button: Take picture activity

We need an activity which snaps a picture and saves a row in the db

Connect up the image processing script and the webpy

Depending on how much work we could connect up the image processing script to fire on recieving a POST... Or alternatively focus on bringing the image processing down to the andriod client.

Auto-take picture when petri-film detected?

Could make it easier to use instead of a capture button.

Potential problem: user must hold petrifilm steady. If it takes photo prematurely, it's a problem.

Simplify architecture

We want to simplify the app to its core components, no menus, no settings, no upload, no GPS. Just image processing and saving the input and output of the image.

Find the circle of the petri-film

Filter out what is outside the inner circular transparent film.

def findcircle(img):
    # Gray scale and blur
    gray=cv2.cvtColor(img, cv.CV_BGR2GRAY)
    blur=cv2.GaussianBlur(gray, (5, 5), 0)

    # Get min max radius
    h, w = img.shape[:2]
    minrad=min(h,w)/8
    maxrad=min(h,w)/2
    circles=cv2.HoughCircles(blur, cv.CV_HOUGH_GRADIENT, 2, 100, None, 100, 100, minrad, maxrad)

    return (circles[0][0][0],circles[0][0][1],circles[0][0][2])

https://github.com/AndroidImageProcessing/AndroidBacteriaImageProcessing/blob/master/data/stage1.png
https://github.com/AndroidImageProcessing/AndroidBacteriaImageProcessing/blob/master/data/stage2.png

GPS data in the TakePicture activity

Need to get the latitude and longitude in the updateImageMetadata method in the takepicture activity

How to debug OpenCV C++ code in an NDK project

one option is to build the C++ in another project

another is to do image print to a path on the sdcard

and the default is to use __android_log_print(

Give developers feedback about the milestones

Take a look at our milestones,

https://github.com/AndroidImageProcessing/AndroidBacteriaImageProcessing/issues/milestones

And add your comments or questions about why or what we should be doing. We put the milestones in this order because of dependancies and complexity (which we couldn't solve in 2 days). The project isn't really complex. But the integration with existing systems is always more complex than rolling-our-own.

Happy Holidays meeting?

@DrDub @adrianao @grassick @peristeri @pakdel @sleepokay

Now that the holidays have hit, do we have time to get together and hack for a few hours, or alternatively just meet up and welcome our new team members!

How about Wednesday Dec 28?

Vote for options by replying to this email

Option A:
Hack together at my house (Metro Jolicoeur = good Wifi, good cider)
Wednesday 10am - 5pm

Option B:
Get together for coffee (Metro Guy-Concordia = good enough coffee)
Wednesday 3pm-5pm

Make server db in sqlite

Email the apk to rocketboy76, or publish apk on Market

we will have to decide on a delivery mechanism..

Find ecoli

Find blue areas that represent ecoli.

def filterecoli(img):
    red=numpy.compress([False, False, True], img, 2)
    green=numpy.compress([False, True, False], img, 2)
    blue=numpy.compress([True, False, False], img, 2)

    red=192-red.clip(0,192)+64

    # Remove green and blue
    greenblue=numpy.minimum(192-green.clip(0,192), 192-blue.clip(0,192))
    col=(red-greenblue).clip(64,255)-64
    col=255-((col*1).clip(0,255))
    return col

https://github.com/AndroidImageProcessing/AndroidBacteriaImageProcessing/blob/master/data/stage6ecoli.png

Questions about using the app in the field:

@rocketboy76 @cesine

I have a few questions that might help the usability of the client application:

1. How often are samples taken from a water source?
2. Can we label the petri gells with a bar code of sorts?
3. What are the distances between water sources? The GPS have an accuracy of +- 30 meters
4. Can the many devices capture images of multiple water source dishes? This link to the idea of id(ing) the dishes.

Filter out the background

Remove the normalized background

Need a package name for the app

Hi @rocketboy76,

We need a unique identifier for the app before we can go too far in the development. The identifier is used by the Android to install the app and make sure it's not being replaced by malicious app etc. It's also used for other things. It usually looks like a domain name, but backwards. Right now we are using a domain name that one of the team members owns but ideally we want an identifier that reflects your project.

Do you have a domain name that you own (or a blog)?

Alternatively we could use something based on your current affiliation... all that really maters is that it is unique.

example:
edu.columbia.eee.feighery.pml.android

Check presense of GPS

Check if the Android device has GPS before trying to turn it on.

Post the petri film image

The post will send:
The authenticated user
The image of the petri dish

Add a hook for image processing on the TakePicture pipeline

Once we have some OpenCV running in the client, lets make it even easier to integrate (and test) our custom C++ image processing by running the OPenCV somewhere in the takepicture pipeline. Details to come once #28 is done..

Find out where Ushahidi can accept POST transactions

We are uploading images via POST