Feature request: Highlighter/markup height same as Textheight about remarks

[einalex]

https://www.reddit.com/r/RemarkableTablet/comments/n667u7/highlighting_on_27_some_specs/

so 2.7 is out now and it seems, getting marked areas is now easy as pie. Check the linked reddit post.

[lucasrla]

Hey all, remarks now supports reMarkable's "smart" highlighting. See: da6433f

[einalex]

example file illustrating narrow markup
Goodrich, Schultz - 2007 - Human-Robot Interaction A Survey_remarks.pdf

[lucasrla]

Hey @einalex, thanks for reaching out.

I agree that the way highlights look could be improved.

If you (or anyone else reading this in the future) is willing to contribute with PRs towards better highlighting, I am happy to help / review.

---

Right now, in remarks, we infer highlighting coordinates starting from what has been drawn on the reMarkable device.

If we somehow checked our inferred coordinates with the actual PDF textboxes, we should be able to achieve better looking highlights.

Off the top of my head, a path to that could be:

1. Refactor the code to, while parsing highlighted text, get the "actual rects/quads" using PyMuPDF's search_for()

2. If necessary, use some kind of method to get an "enveloping" rectangle per highlighted line (maybe via Shapely or from PyMuPDF itself)

3. Use those "enveloping" rectangles (instead of the seg_rect we currently use) to draw the annotations via addHighlightAnnot() as in drawing.py#L104

This path could work well for "well-behaved"/textual PDFs. Again, those "actual rects/quads" should get us better-looking highlights because they are the actual PDF textbox dimensions.

Great-looking highlights in image-based/OCRed PDFs (e.g. the Turing paper from the demo) should be trickier because their PDF textboxes likely are much less reliable.

Thanks again

[einalex]

My intuition was to get the required geometry from the position of the highlighted text we already extract. Essentially mark that as highlighed..if such a thing is possible.

[einalex]

"This is a significant change to the underlying file format: highlighter strokes have been removed from the .rm files and into JSON files in a separate .highlighter directory. These files contain additional data including the actual text that was highlighted and some offset information that helps when grouping strokes together."
https://www.reddit.com/r/RemarkableTablet/comments/mvewrr/version_27030_available_in_beta_program/

This is likely going to change things.

[lucasrla]

Re your intuition, not sure I completely got what you meant by "mark that as highlighted" but yes, there should be different paths to solve this issue.

Re the new highlighting capabilities. That's very interesting! Thanks for sharing. I was not drawn for this beta program. My device is currently on 2.6.2.75 and, as of today, it says there is no update available.

It seems we should wait until this new update is officially released to tackle this issue.

Thanks

[lucasrla]

Good! I will read the reddit post.

My device is still on 2.6.2.75, though. I have just checked for new updates and it says "Your device is up to date." I guess their release process might take a few hours / days to reach all users.

[stefanheyder]

Do you think that remarks will in the future be able to use the marked areas and highlighted texts (for conversion to .md) from 2.7 by directly parsing the json files created?

If you give me some pointers to the code where this parsing occurs, I might be able to submit a PR for this.