alpayariyak / transcriptassist

TranscriptAssist

TranscriptAssist is a powerful system that combines Google Cloud Vision API, Mathpix OCR API, and OpenAI's GPT-4 to transcribe images of handwritten math into LaTeX.

The main purpose of the creation of TranscriptAssist was to accelerate internal tagging and labeling of handwritten math data at EdLight, PBC.

Installation

pip install -r requirements.txt

Setup

Add credentials to .env file in root directory of project

# MathPix
MATHPIX_APP_KEY=""
MATHPIX_APP_ID=""
# Google Cloud Vision
GOOGLE_CLOUD_API_KEY=""
GOOGLE_CLOUD_APP_ID=""
# OpenAI
OPENAI_API_KEY=""

Usage

1. Import TranscriptAssist

from transcriptassist.transcriptassist import TranscriptAssist

2. Initialize a TranscriptAssist object

• temperature (float):
  • The temperature to use for the GPT model.
  • default: 0.3

ta = TranscriptAssist(temperature)

3. Transcribe an image

• link_or_path (str):
  • The link or path of the image to transcribe.
• crop_coordinates (tuple):
  • The coordinates to crop the image to.
  • format: (x1, y1, x2, y2)
  • default: None
• use_gcv (bool):
  • Whether to use Google Cloud Vision API.
  • default: True
• use_mathpix (bool):
  • Whether to use Mathpix API.
  • default: True

ta.transcribe(link_or_path, crop_coordinates, use_gcv, use_mathpix)