A two bit image processor app for whiteboard capture images
Home Page: http://tigue.com/whiteboarder_a_two_bit_photo_processor/
License: MIT License
Whiteboarder is a two-bit image processor for whiteboard capture photos. It is Python code released under an MIT license.
Although the utility is based on stock Python and Jupyter, the code has been optimized to run on Colab. Click the following button to try it out:
brightfield_cuboid_packager.jpg:
red_kneecap.jpg:
git_and_jupyter_book.jpg:
See the demo directory for more before and after demonstrations.
https://developers.google.com/drive/api/v3/push
Obviously, this would be for a non-Jupyter packaging of whiteboarder. Could have an inbox folder, when a whiteboard image is uploaded, something (say an AWS Lambda) would pull it down, give it the treatment, and upload a cleaned up image.
import requests
import PIL
from IPython.display import Image
import PIL.ImageDraw
# Get background image
input_url = 'https://johntigue.github.io/whiteboarder/images/headers/four_pale_markers.png'
a_filename = '/content/four_pale_markers.png'
r = requests.get(input_url, allow_redirects=True)
open(a_filename, 'wb').write(r.content)
# Get font
!wget -O chunkfive.zip https://www.fontsquirrel.com/fonts/download/chunkfive
a_font = ImageFont.truetype('ChunkFive-Regular.otf', 50)
in_image = PIL.Image.open(a_filename)
img_draw = PIL.ImageDraw.Draw(in_image)
img_draw.text((652, 61), 'Whiteboarder', fill='#d01020', font=a_font)
img_draw.text((654, 175), 'A Two Bit Image Processor', fill='black', font=a_font)
img_draw.text((651, 288), 'Free Jupyter Based App', fill='#1010c0', font=a_font)
img_draw.text((653, 405), 'MIT Licensed Python', fill='green', font=a_font)
in_image.save('drawn_image.png')
pil_img = Image(filename='drawn_image.png')
display(pil_img)This is just like with brightfield images but there things are crapping out around 20MB, not < 2MB as is the case here.
Perhaps different things happen with camera-in-macOS-Photobooth compared to camera-in-Colab notebook take picture.
Some microscopist was doing similar, without learning.
Add a rando string on to a analytics tracking URL.
Hub issue for Bokeh
Until March 2020, Bokeh was perfectly working on Google Colab. Some months ago with the new Colab version, it was not working anymore.
[2020-06] I think Google Colab finally upgraded to Bokeh 2.1.1. But now the export PNG that used to work with above code throws this error
push_notebook does not function in Colab because Google actively blocks usage of notebook comms. Supposedly they are working to change this but we have no control over their schedule,
Bounding "box" could be whiteboard edges. But later could have clipping glyphs recongizers that could capture a specified region (auto-clipping).
Python OpenCV. Find rectangular object and warp it to bird view perspective
skimage.filters.threshold_li():
initial_guessfloat or Callable[[array[float]], float], optional.
Li’s iterative method uses gradient descent to find the optimal threshold. If the image intensity histogram contains more than two modes (peaks), the gradient descent could get stuck in a local optimum. An initial guess for the iteration can help the algorithm find the globally-optimal threshold.
Is this "gradient descent" stocastic?
30 is SLOW. Is that 30 squared. Or more like pi R squared i.e. 900pi. Whatever pi is just the constant
In the repo, in the main notebook's UI.
MIT?
Maybe google stuff is conditionally imported? Can that even be done in python like:
if some_condition:
import fooPython APIs to Instagram requires auth key on URL requests.
The primary quality issue looks to be cause by poorly cleaned boards. That might respond well to rolling ball background removal.
Using a camera with decent resolution (say, 2.5K by 2.5K) seems to be causing the runtime to disconnect.
2k x 4k is probably low for the average cell phone camera these days, let alone dedicated cameras.
PNG and GIF at least. TIFF I guess.
See #30
It sounds like there is a way to capture higher res camera image than currently being done.
https://colab.research.google.com/drive/1i2wiH26PtOhGPC3Sm10AJwYUB_koABBK#scrollTo=v1PIvTC8vYCs
# TODO: Want to try png for bigger file, no lossy compression.
#
# TODO: Higher res not: canvas.getContext('2d').drawImage(video, 0, 0);
# https://developers.google.com/web/updates/2016/12/imagecapture
# Previously, it was only possible to 'take a photo' by calling
# drawImage() on a canvas element, using a video as the source (as per the example here).
See also #3
Each Colab session is essentially just injecting images into markdown blog posts.
input() to get a title and description.Features:
tags: is just another frontmatter lineManual and algorithmic (should be able to find the bounding box)
Late in the pipeline someone doesn't like *.png files. Same file as *.jpeg works though
There's too much of the image right next to the pen lines left. Binary opening/closing, dilation, or even contrast enhancement seem to be tricks to try: 2 Easy Tricks to Remove White Edges or Fringes from Cut-Outs in Photoshop.
There have to be techniques for this. And there will probably be a slider to control some parameter.
Convert to grayscale, then rolling ball background to remove reverse images on page.
First contract the mask, then anti-alias.
White (or colored) chalk on a blackboard. Same thing but inverted. So, black&white-boarder? Aww.
Super lame.
Need to not error, but also need to figure out how to force scroll/focus on the output of the cell with the file select UI.
So:
See slider.py should be part of eight_bit_gauges.
Why couldn't controls be part of gauges? Lots of gauges have human input feedback indictors.
This might be a way to get around the large images causing disconnects, assuming the problem is that the images are being transfered as base 64 data in jupyter internal comms:
https://colab.research.google.com/notebooks/snippets/advanced_outputs.ipynb#scrollTo=_7dYIo63EdgL
So, same algorithms to generate image files but just park them on the the Colab VM's FS. Then client directly loads image as if from a %%HTML magic or some JS.
Since all plot/show image code is in Whiteboarder component, rather than raw calls to UI libraries, this can be globally controlled.
More elegant would to have UI controls for that. I.e. user specifies how high quality to download. Starts at medium for decent performance by user can turn it up for inspection.
See also #34.
A good solution might be bokeh server. But it doesn't seem to work out of the box yet: https://discourse.bokeh.org/t/colabs-bokeh-server/5286
Consider specific shorter pipelines. E.g., one that does nothing except rolling ball background removal.
Colab's docs for capturing pics:
Using a webcam to capture images for processing on the runtime.
# Paraphrasing
from IPython.display import display, Javascript
from google.colab.output import eval_js
from base64 import b64decode
def take_photo(filename='photo.jpg', quality=0.8):
js = Javascript('''
async function takePhoto(quality) {
...
const video = document.createElement('video');
const stream = await navigator.mediaDevices.getUserMedia({video: true});
await video.play();
// Wait for Capture to be clicked.
await new Promise((resolve) => capture.onclick = resolve);
const canvas = document.createElement('canvas');
...
return canvas.toDataURL('image/jpeg', quality);
}
''')
display(js)
data = eval_js('takePhoto({})'.format(quality))
binary = b64decode(data.split(',')[1])
with open(filename, 'wb') as f:
f.write(binary)
return filenameThere may be value to enabling custom color-to-grayscale mapping as explained in the article, Photoshop Black and White Conversions – Hue/Saturation Adjustment. Left as an exercise to the reader :) Or is that more like gamma correction? Custom LUT?
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.