dynobo / normcap

OCR powered screen-capture tool to capture information instead of images

normcap's Issues

Change to "real" addon system for Magics

Instead linking the magics' py file by hard imports, dynamically load all py files from magics folder.
This way, it's easier to develop new or customize existing magics.

• Adjust import logic
• Adjust pyinstaller config to copy magics folder instead of packing it.

Tests

• Linux Arch i3
• Ubuntu Gnome
• Windows 10
• Mac

Check Licence

• Licences of Deps?
• Rules for publishing Bins?
• Is MIT still fine?

Improve speed

The startup speed and also text recognition is okay, but faster would be better.
If you have any ideas on how to improve this, please leave a note here! :-)

[Feature] Update download notification

Check for updates on the Application launch and inform users to download new updates when available.
This will help users enjoy the latest features and enhancements in the application.

As of now, there is no way for users to know if there is a new update for the app, until or unless they visit the GitHub repository.

💡 The idea is to pop up the notification if there is an update and ask the user to Download it or Remind me next time.
if the user chooses to download it, then it will open a browser window with the download link.

[Win 10] Create Installer for Windows Binary Release

Use NSIS (already installed on build agent) along with the appropriate github action.

This is also one step to mitigate #87

Create reference images for unit-testing

• Contain one or two "real" screenshots for current:
  • Line
  • Paragraph
  • Email
  • URLs
  • Mixed content
• Small size (640x480?) to make it usable, embedded in an email, for the screencasts

Implement Magic "Source Code"

Build Magic "Paragraphs"

• Return Line Break separated results
• Strip whitespace per Line
• Scoring:
  • Fewer lines
  • Similar distances between lines (no paragraphs)

Ideas for Heuristics:

• Try to tell if the first word on a line could have easily fit on the previous line, indicating an intentional newline.
• Pay attention to punctuation (paragraphs should end with it.)

Tried launching normcap from terminal on MacOS and was able to see dummy window

Remove actions

Automated action triggering makes code much more complicated (because of platform independence) and is probably not so important. Remove them.

Normcap not shown in fullscreen when using gnome-shell and PopOS tiling

The windows with the red border showing the screenshots to select a region are not covering the full screen, but are getting tiled by the window manager. This is at least the case on Ubuntu 20.04 with the gnome-shell extension of PopOS.

Tasks:

• Test some different window managers to see where else this issue occurs
• Try setting self.tk.attributes("-zoomed", True) for linux
• Think about if it's possible to write a unit test for this issue

[Windows] Failed to execute script main.

I did recommended way but this no works, also is there way to add shortcut keys to execute this app.

Add screencast to README

How to print shortcuts to Screen?
Which recorder to use?

Create Magic "www"

• Return char separated urls incl hash and parameters
• Scoring: length all / length urls
• Action: Open in (multiple) Tabs
• Chrome, Firefox, Safari

Keybindings not working in MacOS

Probably due to the window is not getting focus.

Decrease number of dependencies

I tried to keep the dependencies / third party libs at minimum, but still have the feeling, they could be less.

If you have ideas on how to get rid of another dependency, please comment.

Create Magic "email"

• Return char separated Email (check outlook, thunderbird, gmail)
• Scoring:
  • length of all text / length of mail-adresses
• Action: Create new email

Add automatic language detection

Idea: Do OCR ones in e.g. "EN", run language detection, run OCR again in detected language (if it's different than EN and available in the OCR).

Libraries:

• https://github.com/saffsd/langid.py (seem to be a little less reliable on short texts, but very view deps)
• https://github.com/Mimino666/langdetect (seem no working with pyinstaller)

Add pytest coverage banner

See: https://coveralls.io/

Implement detection of tables/tabular text

• Return tables as CSV
• Score: Detected Columns and Rows
• Action: Clipboard? Excel?

Links that might be helpful:

• tesseract-ocr-utils seems to implement table detection

I shall open an Issue - Error tk Mac OSX

It said so so I did.
Mac OS Sierra (10.12)
installed with pip (Python3.9)
launched from Terminal normcap
useful Application Window never appeared.

If you need anymore information ask for it I am happy to help.

Identify library for fullscreen display

Requirements:

• [R1] Cross Platform
• [R2] Multi monitor
• [R3] Select active Monitor
• [R4] Licence (LGPL or above)

Further Criteria:

• Common used
• Low Dependencies

Prioritized Options:

• tkinter
• wxpython
• pygame
• pyglet
• QT

Build Binaries

• Linux
• Windows 10

Add modifier key for raw / parse / magic

• Visual indicators
• Legend, hint
• Ctrl / Shift
• Default plain function

Add build pipeline for github

See: https://travis-ci.com/

Crash on Wayland, XGetImage failed

NormCap crashes when trying to run it under Wayland:

20:29:41 - INFO - normcap.normcap - Starting NormCap v0.1.2 ...
Traceback (most recent call last):
  File "/home/ige/.local/bin/normcap", line 10, in <module>
    sys.exit(run())
  File "/home/ige/.local/lib/python3.7/site-packages/normcap/__main__.py", line 5, in run
    _ = main()
  File "/home/ige/.local/lib/python3.7/site-packages/normcap/normcap.py", line 153, in main
    normcap_data = client_code(capture, normcap_data)
  File "/home/ige/.local/lib/python3.7/site-packages/normcap/normcap.py", line 106, in client_code
    result = handler.handle(normcap_data)
  File "/home/ige/.local/lib/python3.7/site-packages/normcap/handlers/capture_handler.py", line 26, in handle
    request = self._take_screeshot(request)
  File "/home/ige/.local/lib/python3.7/site-packages/normcap/handlers/capture_handler.py", line 40, in _take_screeshot
    raw = sct.grab(position)
  File "/home/ige/.local/lib/python3.7/site-packages/mss/linux.py", line 403, in grab
    ZPIXMAP,
  File "/home/ige/.local/lib/python3.7/site-packages/mss/linux.py", line 175, in validate
    raise ScreenShotError(err, details=details)
mss.exception.ScreenShotError: XGetImage() failed

[Win 10] Normcap windows notifications can't turned back on

Before my i would like to thank you for application, it is just what i wanted. I am not a programmer of anykind i just wanted app like this for everyday use so here is my problem. I turned of the windows notifications form notifications area but it was a bad decision and i wanted to turn it back on but i can't find any option to turn the apps notifications on. I guess because app is working from a file and not installed the pc windows does not sees the app and i can't be found in the notifications settings. Because of that maybe there can be a another download option like a setup to improve notification settings etc. Or do you know i way i can handle this issue ?

Separate screen capture from selection gui class

Add application menu shortcuts for normcap

1. On Windows: Solved by PR #93.
2. On Linux: investigate, how to create normcap.desktop is a freedesktop compliant way. See also:

• https://github.com/AppImage/AppImageKit/wiki/Bundling-Python-apps
• https://pypi.org/project/install-freedesktop/

Improve scoring

• "paragraph" is way to high
• "email" slightly to low

Refactor PoC Code

• Use Chain of Responsibility Pattern
• Classes
• Naming
• Comments

Improve logging

• Remove duplicate logging
• Log-Level to front (after timestamp)

Remove debug output

Somethings creates an output after the capturing. Seek and destroy.

{'single_line': <normcap.magics.single_line_magic.SingleLineMagic object at 0x7f5db1b7a650>, 'paragraph': <normcap.magics.paragraph_magic.ParagraphMagic
object at 0x7f5db1b7e150>, 'email': <normcap.magics.email_magic.EmailMagic object at 0x7f5db1b9b610>,
'url': <normcap.magics.url_magic.UrlMagic object at 0x7f5db1b9b0d0>}

Fatal error detected

Use best traineddata for windows

Instead the compromising traineddata sets for tesseract, try using the best. I think more accuracy is worth potential slower execution.

Test pyocr

At least with Tesseract, on Linux & Windows

Identify library for taking screenshot

Requirements:

• [R1] Cross Platform
• [R2] Multi monitor
• [R3] Select active Monitor
• [R4] Licence (LGPL or above)

Further Criteria:

• Common used
• Low Dependencies

Prioritized Options:

• Custom (use OS-standards like scrot)
• pyscreenshot
• pyscreeze

Add notifcation/splash screen to show whats happenig after screenshot selection

E.g.:

• "Detecting text..." -> "Parsing email.." -> "Trigger action..."
• Preferably icons only, no text

[Linux] Provide AppImage

Package Normcap as AppImage for easier install/execution without installing too much dependencies.

See also: https://github.com/AppImage/AppImageKit/wiki/Bundling-Python-apps

Bundle as PyPi Packag

To make installable via pip

Enable tests on Windows & Mac for PRs

Extend test workflow to all platforms

Addon system for Magics

Create an addon system for essy extensible Magic Commands.

Should include a scoring mechanism to decide, which magic to run, an transformer and an executor.

[Feature] Tray icon support

Support for the Tray icon can be added so that the user is not always required to relaunch the application.

Enhance README

• Installation on Linux and Windows
• Design principles
  • No Text (multilang)
  • Cross Platform (Lin, Win, Mac)
  • No Service
  • No config (args instead)
• Development setup
• Disclaimer (no warrenty, alpha, works on my machine)
• Screencast / gif

Bundle with Tesseract?

What do you think, should I bundle NormCap together with tesseract and a set of languages? Some thoughts:

Pros:

• More simple installation of NormCap
• Self contained releases
• Easier to debug(?)

Cons:

• Larger release files
• Possibly duplicated tesseract packages, when other programs use tesseract, too
• Every user has to use the same version of tesseract (the bundled one)
• Tesseract bundling also has to be maintained

Improve detection accuracy

Most of the detection quality depends on Tesseract OCR and the input image.

But there are also options/ideas in scope of this project:

• Automatically preprocess screenshot before feeding into OCR
• Tune Tesseract OCR parameters
• Apply heuristics to correct common OCR mistakes
• Improve "Magics" in detecting & transforming the OCR results

I'm happy about any ideas or pull request on this topic!

Add command line args

• Color
• Default mode/addon
• Store snippet path
• Debug level

Write unittests

For automated regression tests

[Windows] Failed to execute script __main__

In windows, you will get the error Failed to execute script __main__ when instead of click & drag the area user simply clicks on the screen.
The error should be handled appropriately.