louisbrulenaudet / apple-ocr

Easy-to-Use Apple Vision wrapper for text extraction, scalar representation and clustering using K-means.

Easy-to-Use Apple Vision wrapper for text extraction and clustering

apple_ocr is a utility for Optical Character Recognition (OCR) that facilitates the extraction of text from images. This Python-based tool is designed to help developers, researchers, and enthusiasts in the field of text extraction and clustering. It leverages a combination of various technologies to achieve this, including the Vision framework provided by Apple.

Features

• Text Recognition: apple_ocr uses the Vision framework to recognize text within an image. It extracts recognized text and provides information about its confidence levels.

• Clustering: The tool can perform K-Means clustering on the extracted data. It groups similar text elements together based on their coordinates.

• Interactive 3D Visualization: apple_ocr offers an interactive 3D scatter plot using Plotly, displaying the clustered text elements. This visualization helps users gain insights into the distribution of text and text density.

Dependencies

The script relies on the following Python libraries:

• Torch
• NumPy
• Pandas
• Pillow
• Scikit-learn
• Plotly
• Pyobjc

Usage

Here's how you can use apple_ocr:

1. Installation: Install the required libraries, including Torch, NumPy, Pandas, Pillow, scikit-learn, and Plotly.

2. Initialization: Create an instance of the OCR class, providing an image to be processed.

from apple_ocr.ocr import OCR
from PIL import Image

image = Image.open("your_image.png")
ocr_instance = OCR(image=image)

3. Text Recognition: Use the recognize method to perform text recognition. It will return a structured DataFrame containing recognized text, bounding box dimensions, text density, and centroid coordinates.

dataframe = ocr_instance.recognize()

4. Clustering: Use the cluster method to perform K-Means clustering on the recognized text data. This method assigns cluster labels to each data point based on their coordinates.

cluster_labels = ocr_instance.cluster(dataframe, num_clusters=3)

5. Visualization: Finally, use the scatter method to create an interactive 3D scatter plot. This plot visualizes the clustered text elements, including centroids, text density, and more.

ocr_instance.scatter()

Example

Here's an example of the entire process:

from apple_ocr.ocr import OCR
from PIL import Image

image = Image.open("your_image.png")
ocr_instance = OCR(image=image)
dataframe = ocr_instance.recognize()
cluster_labels = ocr_instance.cluster(dataframe, num_clusters=3)
ocr_instance.scatter()

Citing this project

If you use this code in your research, please use the following BibTeX entry.

@misc{louisbrulenaudet2023,
	author = {Louis Brulé Naudet},
	title = {Easy-to-Use Apple Vision wrapper for text extraction and clustering},
	howpublished = {\url{https://github.com/louisbrulenaudet/apple-ocr}},
	year = {2023}
}

Feedback

If you have any feedback, please reach out at [email protected].

apple-ocr's Issues

ModuleNotFoundError: 'Vision' when using apple_ocr.ocr

I'm trying to use the apple_ocr.ocr module for Optical Character Recognition (OCR). However, I'm encountering a ModuleNotFoundError: No module named 'Vision'.

Steps to Reproduce:

Install apple_ocr (if not already installed).
Import OCR from apple_ocr.ocr and Image from PIL.
Open an image using Image.open("image_filename.jpeg").
Try creating an OCR instance: ocr_instance = OCR(image=image).

Expected Behavior:

The OCR class from apple_ocr.ocr should be created successfully.

Actual Behavior:

A ModuleNotFoundError: No module named 'Vision' is raised.

Additional Information:

Running on Google Colab

If you have any insights into the missing Vision module, mention them here.

Request:

Could you please investigate why the Vision module is not found?
Are there specific installation instructions for apple_ocr that include this dependency?