how does text extraction work ? about windowtextextractor

[AlexanderPro]

No, only Windows built-in OCR

[AlexanderPro]

Hi @flippythemonster,

WindowTextExtractor uses Microsoft UI Automation. But when you click "Grab" button, it uses Windows OCR.

[flippythemonster]

thanks for the prompt response ! if I don't want to use OCR, how can I extract text?
it seems that whenever I try to hover over an app, I get property related information instead of the textual content.
Sorry if I misunderstand the product!

[AlexanderPro]

You should download the latest version 2.0.0. Open "Text" tab and try to hover over an app for textual content. On "Image" tab you can see graphical window content and can grab it by "Grab" button.

[flippythemonster]

Please look at the attached picture:

This is the output when I try to hover over my chrome window, when I try on other apps, it does not output anything. And I am using the latest 2.0.0 version. I am not trying to use OCR as I need accurate textual representation of the content of my desktop.

[AlexanderPro]

Chrome does not support UI Automation by default. On Chromium-based browsers, try to access chrome://accessibility (it will redirect to edge://accessibility on Edge, doesn't matter), then check all under Accessibility modes.

You will need to do this every time you want to extract text from Chromium-based browsers, per (re-)launch of browser.

[flippythemonster]

Interesting, this works and fixes it. So it depends on whether the app supports UI automation or not ? I ve tried on other apps such as VScode, discord etc and run into similar trouble where these apps are recognised as chrome legacy windows. Are there other means to retrieve textual content from desktop without using OCR and is more global than UI automation ?

I apologise if I am asking for redundant or stupid questions. I am trying to research for ways to do so but so far even state of the art OCR models have failed on my test cases.

[AlexanderPro]

Yes, some windows do not support UI automation. I could not find a universal solution for obtaining window text. Therefore, WindowTextExtractor contains several tabs. "Text" tab is for Microsoft UI Automation extracting and "Image" tab is for Microsoft Windows OCR extracting.

[flippythemonster]

oh thank you !! I am happy that you kept maintaining such project. Are you using tesseract for your OCR support ?