swhl / convertlatexocrtoonnx

Convert LaTeX-OCR To ONNX

Convert LaTeX-OCR To ONNX

1. Clone the source code.

git clone https://github.com/SWHL/ConvertLaTeXOCRToONNX.git

2. Install env.

Anaconda

conda env create -f environment.yml

Pip

$ conda create -n cvt_latex python=3.10.13

$ conda activate cvt_latex

$ pip install -r requirements.txt

3. Run the convert.py, and the converted model is located in the models directory.

$ python convert.py

# /Users/xxx/miniconda3/envs/convert_latex_ocr/lib/python3.10/site-packages/torch/onnx/symbolic_helper.py:1513: UserWarning: ONNX export mode is set to TrainingMode.EVAL, but operator 'batch_norm' is set to train=True. Exporting with train=True.
#   warnings.warn(
# Exported model has been tested with ONNXRuntime, and the result looks good!
# ONNX Model has been saved /Users/xxx/projects/_self/ConvertLaTeXOCRToONNX/models/image_resizer.onnx
# Exported model has been tested with ONNXRuntime, and the result looks good!
# ONNX Model has been saved /Users/xxx/projects/_self/ConvertLaTeXOCRToONNX/models/encoder.onnx
# Exported model has been tested with ONNXRuntime, and the result looks good!
# ONNX Model has been saved /Users/xxx/projects/_self/ConvertLaTeXOCRToONNX/models/decoder.onnx
# \exp\left[\int d^{4}x g\phi\bar{\psi}\psi\right]=\sum_{n=0}^{\infty}\frac{g^{n}}{n!}\left(\int d^{4}x\phi\bar{\psi}\psi\right)^{n}.

4. Used in RapidLaTeXOCR.

from rapid_latex_ocr import LatexOCR

image_resizer_path = 'models/image_resizer.onnx'
encoder_path = 'models/encoder.onnx'
decoder_path = 'models/decoder.onnx'
tokenizer_json = 'models/tokenizer.json'
model = LatexOCR(image_resizer_path=image_resizer_path,
                 encoder_path=encoder_path,
                 decoder_path=decoder_path,
                 tokenizer_json=tokenizer_json)

img_path = "tests/test_files/6.png"
with open(img_path, "rb") as f:
    data = f. read()

result, elapse = model(data)

print(result)
# {\frac{x^{2}}{a^{2}}}-{\frac{y^{2}}{b^{2}}}=1

print(elapse)
# 0.4131628000000003

convertlatexocrtoonnx's Issues

encoder不能动态尺寸推理

请问encoder只能固定尺寸推理吗，我使用动态尺寸推理会出现如下错误：
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running Add node. Name:'/Add_2' Status Message: /onnxruntime_src/onnxruntime/core/providers/cpu/math/element_wise_ops.h:560 void onnxruntime::BroadcastIterator::Append(ptrdiff_t, ptrdiff_t) axis == 1 || axis == largest was false. Attempting to broadcast an axis by a dimension other than 1. 49 by 57

基于Deformable-detr的端到端文字识别模型，也是在encoder做注意力的节点报错不支持多尺度

          > @Zalways 问题比较综合，有很多地方。建议去参考我的commit来看看吧，我也说不太清。

感谢回复！ 我的模型是一个基于Deformable-detr的端到端文字识别模型，也是在encoder做注意力的节点报错不支持多尺度，就像请问一下大体的修改方向是哪一块

Originally posted by @Zalways in #1 (comment)