benehiko / ocr-core Goto Github PK

Monash University Industrial Experience Project: Team DreamTeam

Java 100.00%

number-plates ocr tesseract tess4j opencv-java

ocr-core's Introduction

OCR-core

Repository Purpose

The DreamTeam is a Monash University undergrad team building a software solution for Aztomix (our "employers") under the guidance of our Monash University mentor. This counts as credits spanning accross 2 semesters and is called the Industrial Experience Project (FIT3047 - FIT3048).

What is it?

A Java based Automatic NumberPlate Recognition (ANPR) build on OpenCv and Tesseract. This software presents a Java websocket server running on port 10000 and accepts a byte[ ] which returns a string of extracted text.

How to run it?

java -jar ocr.jar

Running it in the background use the screen package:

sudo apt install screen
screen -d -m java -jar ocr.jar

How to build it

git clone [email protected]:Benehiko/ocr-core.git
cd ocr-core/
mvn clean install
cd target/
java -jar ocr-core.jar

A full guide is available on this repository's Wiki

Setting up the Server

Licensing

ocr-core's People

Contributors

Watchers

ocr-core's Issues

Reduce package size

Dependencies are taking way too much space.
Reduce the dependencies by selecting only needed.

Create an easy endpoint

Endpoint design

Send in a byte array of images or URL (?) and receive back extract and all other metadata concerning image.
Ask for all processed images and receive a JSON encoded string.

Current System

Connect to server with [ip]:[port] and send byte[ ]. Primitive socket listener port stays open until image processed.

Integrate old platform code to new dev build

Current development build has been changed.
Usage of IplImage is deprecated, since converting between IplImages and BufferedImage caused crashes. Mat is now the base supported image. Also javacv libraries are used only in some areas, moved over to opencv build libraries.

Tilted Rectangles

Tesseract ignores rectangles that are skewed

Learning Model

The OCR system needs to learn from success stories.
An idea is to write to an SQLite database created by the OCR system.
This system will add only the license plate slice to the database.

Will be made threadable for quick access on multiple occasions. Since we might have to process multiple images at the same time.

Compress Image

Compress image before processing OCR

OcrShapes class -> getRectArray Improvement

Log the approxCurve value used for most success on OCR.
Save it to the Cache for future self-learning.

public Rectangle[] getRectArray(List<MatOfPoint> contours) {
 ArrayList<Rectangle> r = new ArrayList<>();

    int minArea = 800;
    contours.forEach((contour) -> {
        for (int approxCurve = 2; approxCurve < 6; approxCurve++) {
            if (isContourSquare(contour, approxCurve)) {
                
                if (Imgproc.contourArea(contour) > minArea) {
                    Rect rectTmp = Imgproc.boundingRect(contour);

                    r.add(new Rectangle(new java.awt.Point(rectTmp.x, rectTmp.y), new Dimension(rectTmp.width, rectTmp.height)));
                    System.out.println("Found a rectangle");
                }

            }
        }
    });
    return r.toArray(new Rectangle[r.size()]);
}

Random text output from large images

When processing large images (normal mobile device camera quality) the software gives out garbage text.

Slicing Images

Images aren't slicing correctly. Sometimes they have incorrect width and height.
Redo slicing of images...
Exact location:
Class OcrShapes ->
Method extract_images

Some solutions would be to inspect CvSlice object

Accept multiple images with ocr server

Accept multiple files on the server
Process them Asynchronously

Crash on to BufferedImage convesion

`
A fatal error has been detected by the Java Runtime Environment:

SIGSEGV (0xb) at pc=0x00007f2900098d7a, pid=4079, tid=0x00007f29013b8700

JRE version: OpenJDK Runtime Environment (8.0_162-b12) (build 1.8.0_162-8u162-b12-0ubuntu0.16.04.2-b12)
Java VM: OpenJDK 64-Bit Server VM (25.162-b12 mixed mode linux-amd64 compressed oops)
Problematic frame:
V [libjvm.so+0xa32d7a]

Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again

An error report file with more information is saved as:
/home/benehiko/NetBeansProjects/OCR/hs_err_pid4079.log

If you would like to submit a bug report, please visit:
http://bugreport.java.com/bugreport/crash.jsp`

License plates that are captured from an angle not detected

Need to find a way other than template matching (or self-improvement algorithm) for license plate captured from the side

Update shape detection with new algorithm

Created algorithm using python, please use this as an indicator of where to work off of.
Python code

Memory Usage exceeds 1GB with large pictures

Large pictures causes the application to consume large amounts of memory

Images taken in portrait mode was imported skew

Any image taken in portrait mode was read by java as skew.

Software crash on some PNG usage

Currently experiencing some software crashes using a PNG image in our processing.

`run:
OpenCV Error: Assertion failed (_filter == CV_GAUSSIAN_5x5 && src.type() == dst.type()) in cvPyrDown, file /projects/bytedeco/javacpp-presets/opencv/cppbuild/linux-x86_64/opencv-3.4.0/modules/imgproc/src/pyramids.cpp, line 1645
Exception in thread "main" java.lang.RuntimeException: /projects/bytedeco/javacpp-presets/opencv/cppbuild/linux-x86_64/opencv-3.4.0/modules/imgproc/src/pyramids.cpp:1645: error: (-215) _filter == CV_GAUSSIAN_5x5 && src.type() == dst.type() in function cvPyrDown

at org.bytedeco.javacpp.opencv_imgproc.cvPyrDown(Native Method)
at ocr.core.OcrShapes.find_squares(OcrShapes.java:109)
at ocr.core.OcrShapes.recognise_shapes(OcrShapes.java:73)
at ocr.core.OcrCore.openCv(OcrCore.java:82)
at ocr.core.OcrCore.process_image(OcrCore.java:55)
at ocr.core.TestDriver.main(TestDriver.java:19)

/home/benehiko/.cache/netbeans/8.2/executor-snippets/run.xml:53: Java returned: 1
BUILD FAILED (total time: 0 seconds)`