rehackable / scripts Goto Github PK

Hi,
I have installed your scripts on my laptop running under Linux Ubuntu 20.04.
I enabled the USB Web UI on the remarkable 2 tablet.
Typing the following command:
reMarkable/scripts/host/repush.sh -r xxx.xxx.x.xx wcs.pdf
[email protected]'s password:
with good IP address and password, nothing happens, no error message and no file copied on the tablet. It stays there forever.
After killing the command with CTRL c, I get an error message if I try to rerun the command w/o restarting the reMarkable 2 tablet:
repush: Port 9000 is already used by a different process!

Any hints on how to solve this problem is very welcome.
Thanks,
Gilles

First, I want to say I am so grateful for these scripts! Thank you for creating them and sharing them with the world.

Second, the question: Is it possible to save my rM's password to the script? I have to re-enter it for every file I push, which can get a bit annoying when pushing more than three PDFs at a time.

Thanks for your time.

repush works fine when I provide a single file.

However, when I give it a list of files

bash repush.sh -o /Papers test.pdf test2.pdf test3.pdf

something goes wrong. It creates one pdf on the remarkable in the correct folder with the contents of the first file and the name of the last file. The middle files sometimes appear as tmp_repush_test2_tmp_repush.pdf files in the top directory.

I'm on MacOs and also had this grep problem (#8)

Hi
It would be great to enhance the repush.sh script in the following way:
Transferring files to a specific folder on the reMarkable works like this:
repush.sh -o ExampleFolder *.pdf
Unfortunately the folder has to already exist on the reMarkable. If not an error message is thrown ("Unable to find output directory: ExampleFolder"). It would be very nice, if the script could be enhanced in a way, that if the folder does not exist, then a new folder with the specified name is created on the remarkable.

Thanks a lot for considering my request for improvement.

Hi 👋

I have problem with transferring files to my Remarkable 2 through ssh on my wireless network.

When running repush -r x.x.x.x pdffile.pdf, it hangs for about a minute or so then gives the following message:

repush: pdffile.pdf: Failed
repush: Is the WebUI enabled in the settings? (See: https://github.com/reHackable/scripts/wiki/repush.sh)
Failed to push file! Retry? [Y/n]:

I have enabled WebUI when I tested transferring with USB (which works fine), but the option is not available to toggle in the menu when USB is not connected.

• Remarkable software version: 2.4.1.30
• repush-script updated today from raw file

Hi,
first of all: thanks to Patrik ans the other contributors for this collection of scrips. I am working only on Linux and without this the remarkable would be useless to me.

Sleep time too short to generate meta data file
Description:
Transferring a bundle of pdf-files with repush.sh , for some files the error message appears:
"Failed to access remote metdata for 'myfile.pdf'"

Reproducible: yes, always

Analysis:
I extended the sleep time in line 248 from 0.1, to 1.0 and then to 2.0 and the error disappears. Obviously the sleep time is too short. Interesting is that the time to generate the metadata does not depend on the file size. Larger files are pushed without problems and small files may have problems.

Workaround:
Extend the sleep time in line 248 from "sleep 0.1" to:
sleep 2.0

Ideas for Bugfix/Enhancement:
Either generally extend the sleep-time or adapt the sleep time dynamically

If you have further questions, please let me know.

I said this elsewhere, but thank you for this script.

I have it working to send pdf files from my mac to my tablet, and I'm very happy about that.
I can't send epub files, per #14 (which is fine with me)

Also, I can only send files that are .pdf, not .PDF.
if I send a .PDF file, a blank pdf is created in the root directory on reMarkable with the lower-case filename, and the script seems caught in an infinite loop.

Likely this should be caught by the error-catching code that I removed on my machine since my grep does not have -P

#  elif [ -z $is_directory ] && ! echo "$1" | grep -qP "\.pdf$" && ! echo "$1" | grep -qP "\.epub$" ; then
#    echo "repush: File extension invalid or missing: $1"
#    return 0

The first thing I tried was naively rewriting the above to only check for "pdf" (since I disabled the epub issue to get up and running on a mac with no -P option in grep)

    # added by krobinso
  elif [ -z $is_directory ] && ! echo "$1" | grep -q ".pdf" && ! echo "$1"; then
    echo "repush: File extension invalid or missing: $1"
    return 0
    # end addition
#  elif [ -z $is_directory ] && ! echo "$1" | grep -qP "\.pdf$" && ! echo "$1" | grep -qP "\.epub$" ; then
#    echo "repush: File extension invalid or missing: $1"
#    return 0
#******

The result of this is somewhat interesting: when I send a file like filename.PDF,

• the script echos the filename
• a blank file filename.pdf (all lower case) is created on the reMarkable
• the reMarkable wifi disconnects. It needs restarting, and then I have to go enable the webui again.

I am running a virtual (Parallels) Ubuntu linux installation on a MacBook and
bash ./repush.sh Zh.pdf
produces the following error message:
"Failed to establish connection with the device!"

ping 10.11.99.1 (USB connection)
works fine and also I can also ssh into the rM, that is,
ssh [email protected]
works.

Any ideas how to resolve this issue?

Hi @Ameb,

thanks for the scripts. I like repush as it uses the web interface and does not require a reboot (as pdf2remarkable.sh). It works fine, however, I did'n have success to push to a directory.

Successfully established connection, please do not lock your device until the script has completed!

===================================================================================================================================
Shipping documents to output directory. It is highly recommended to refrain from using your device until this script has completed!
===================================================================================================================================

Temporarily disabling Wi-Fi to prevent conflicts with the cloud

Shipping 'test.pdf'
Failed to access remote metdata for '_tmp_repush_test_tmp_repush_'
Applying changes...
Re-enabling Wi-Fi in 5 seconds...

Exit request sent.
Successfully transferred  out of 1 documents

The command was repush -d Privat test.pdf, where Privat is the visible name of an existing folder on the rM. Any clues?

Firmware version 2.0.2.0 has now rolled out to my device, and unfortunately I didn't rollback the webui_invincibility patch before updating, assuming it would just overwrite everything.

Now the UI for the web interface is stuck on Off and changing WebInterfaceEnabled to true gets overridden at process start.

Not really sure how this happened. Maybe the update is a binary diff, and the patch stuck around, still breaking the UI but not working fully?

This is my current xochitl, if anyone could confirm it's not the stock one.

75b0c179e9b3facf7cacef2cd9ae748a08a2695306bb2b8a978c98160602df3c  /usr/bin/xochitl

if so, it will be great to mention it on the readme. Thank you!

I use repull.sh unter Linux in Windows (wsl). Works.
The script does not exit.
I just get a lot of "muxclient: master hello exchange failed" but it downloads everything

reclean found no deleted documents, but returned 2142 files and directories without metadata, causing the script to crash with an argument list too long.

The few returned items I've looked for manually do not exist, moreover the script returns more "junk" than the total number of files/directories in /xochitl/

$ reclean.sh
Attempting to establish connection with the device...
Successfully established connection, please do not lock your device until the script has completed!
Search for metadata from deleted documents [y/N]: y

Note: Documents previously pushed to the cloud will be re-downloaded as soon as connection is established

Searching for deleted files...

No deleted documents found, nothing to clean here...

Search for files and directories without metadata [y/N]: y
Searching for junk...

The following documents have been found:

~/.local/share/remarkable/xochitl/0038e664-71c1-4968-b487-889052f4a6d1*
...
~/.local/share/remarkable/xochitl/ffc641b5-bce5-4bbc-926e-daa9904e267e*
~/.local/share/remarkable/xochitl/fff95313-0f3e-4897-8b61-a905aff40401*
Proceed: [y/n]: y
Deleting files...
~/bin/reclean.sh: line 135: /usr/bin/ssh: Argument list too long

Trying to delete them manually:

$ ssh remarkable "rm -R ~/.local/share/remarkable/xochitl/fff95313-0f3e-4897-8b61-a905aff40401*"
rm: can't remove '/home/root/.local/share/remarkable/xochitl/fff95313-0f3e-4897-8b61-a905aff40401*': No such file or directory

in total, the script identified 2142 junk items, but:

remarkable: ~/.local/share/remarkable/xochitl/ ls -l | wc -l
1775
remarkable: ~/.local/share/remarkable/xochitl/ ls *.metadata | wc -l
215

OSX no longer supports -P so the grep command fails.

First of all, thanks a lot for your work!

I had no success with repull so far. It hangs at the following step:

Attempting to establish connection with the device...
Enter passphrase for key '/home/wagner/.ssh/id_rsa': 
Successfully established connection, please do not lock your device until the script has completed!
repull: Checking device for documents...
Searching for Notebook...

Although there is a file called "Notebook". Any ideas?

As per title, while using the -r option, the script reports to me repull: Pulling '/filename', but never actually delivers.
In the folder, a socket file called remarkable-ssh is created, but the script never ends.
I tried it with moderately sized files (5 pages) as well as larger ones.
P.S. I currently have no way of trying it via USB.

This is a known issue due to changes on how /dev/fb0 works on the new remarkable 2.

Check rien/reStream#28 .

I cloned the repo, connected to my macbook pro via usb, and turned on "USB web interface" on the RM2.
I then ran "bash repush.sh /path/to/file.pdf" and the command just hangs.

The file is created on my RM2, but it's just a blank document with the correct name in the root dir.

I have tried specifying an out folder as well, and had the same result. It just drops it in the root folder and fails to copy the pdf.
I have run it with bash and zsh to no avail. I was able to ssh in and set up my keys no problem. And it's creating a file. So I know my connection is good.

Should I try again on a linux machine?

line 552: unexpected EOF while looking for matching `"'

I suspect the culprit s in creating the epub placeholder.

Commenting that out resulted in some grep help messages and error: "repush: File extension invalid or missing: pump_doc.pdf".

Using script version 3.0 on MacOS 10.14.5.

Hi,
Here is another little feature to enhance the repush.sh script

Currenty, after finishing the data transfer, the Wifi is always switched on again.
I am not using the wifi at all (because I love my privacy, and do not want to connect/transfer any data into the cloud). Therefore it would be better if the script determines the state of the wifi before the data transfer, then switches the wifi off, and then - at the end of the script - sets it back to the same state as it was in the beginning.

Thanks a lot for considering my request.

Hi,

currently rM displays the pdf pages on top left corner of the screen if the page size has a different ratio as the screen.

Is it possible to display the pages by default in middle?

I have been using repush.sh for a while and really love it. Unfortunately, I guess after some recent major updates by reMarkable, the script doesn't work any more, at least for me. I could identify a problem with grep being called via ssh, which now gives an error:

unrecognized option '--color=auto'

which is easily solvable by replacing the remote grep instances in the script with /bin/grep. After fixing that, I can upload a pdf and see it on the "My Files" page, and redownload it from the WebUI, but cannot open it on the tablet. If I do so, it endlessly shows the 3 dots of loading. I think it gets stuck somewhere in either 1) generating thumbnails or 2) page information. For example, the content file shows sth like

    "pageCount": 0,
    "pages": [
    ],

I tried with a couple of different pdfs and the behavior is the same.

I would really love to solve this issue as I find this script the best way to interact with my reMarkable.
Thanks

With a remarkable 2 running 2.4.1.30, the ePUB files installed with repush.sh never seem to be updated beyond the two page placeholder added in commit 67b15a3

The $UUID.ePUB file in ~/.local/share/remarkable/xochitl/ has the right sha1sum, so the file transfer worked, although the corresonding $UUID.content file still only has the two pages listed rather than the few hundred that are expected, and viewing the document on the table only shows the two empty placeholder pages.

Am I missing a step to have the tablet process the .epub file to populate the .content?

With version 2.5.0.27, repush.sh is no longer able to transfer files onto the remarkable. The reason for this is a change in the setting of teh visibleName in the metadata field. While before this was the filename including the extension, now the metadata is generated with the filename without the extension in the visibleName-field. This causes repush.sh to hang indefinitely as internal checks fail.

The fix for this has to be applied in two locations:
This line has to be changed to placeholder_basename="$(basename "$1" .$extension)", as $extension is available from a couple of lines above.

The second location is this, in principle the same fix applies, however, $extension is not set. A quick test of mine using "$(basename "$f" | cut -f1 -d '.')" works like a charm, however, this will fail the moment the filename contains another . besides the one separating the extension. As I have just briefly investigated the problem, I am not sure what's the cleanest fix for this location, hence an issue describing the fix instead of pull requesting a fix. The hack with cut, while working, is certainly not the optimal solution.

Addendum: Due to the handling of directories (dropping a placeholder pdf and turning that into a directory), the aforementioned fix is likely not that straightforward. I currently settled for using basename twice at that very location to first remove a .pdf-extension and then removing a .epub-extension, which works well enough for me. However, this is likely breaking directory pushing, so it likely needs to be implemented a little more carefully, with consideration for the directory pushing implementation.

Hi, could we adapt this script to be able to take a screenshot directly on the RM ?
This is a question to enhance drawing workflow, as very large drawing tends to become unresponsive exporting them would flatten things into a single image that could then be used to continue the work on it.
Thanks for your answer

Hey guys,

i have set up the repull script with my remarkable2. This works already. I also have set up ssh-keys like the method in the wiki. If I connect from host to remarkable with "ssh 'root@ip adress'" i don't need the password. But if I start the repull.sh I still need a password. Is there any parameter when executing the repull.ssh wich i have to choose?
I call it like:

sudo bash /usr/local/bin/repull.sh -o /home/pi/folder -r xxx.xxx.xxx.xxx /folder/document

Two things:

1. Pushing files to the remarkable did not work so far Failed to push file! Retry? [Y/n]: Y What could be the reason? I also tried the pdf2remarkable script. While this copies the files to the device, they don't appear under "My files".
2. Saying "no" initializes a retry anyway:

Failed to push file! Retry? [Y/n]: n
Pushing 'paper.pdf'...

Out of curiosity and in hope to learn more about binary patching I've attempted to follow the manual patching guide for webui_invincibility (with Cutter 1.12.0).

I can find the correct string and copy its adress (0x003bf809).

But "axt @ 0x003bf809" won't report any references at all.
So with the steps provided it's not possible to find the correct function.

I've tried the addresses of other strings successfully (e.g. "Supplicant interface already enabled and valid, but reconnecting just in case"). So I believe it is not a mistake in how I've entered the command.

It's great that you have your approach documented. A great practical learning resource :) Thanks!