fishy / url2epub

Create ePub files from URLs

url2epub

Create ePub files from URLs

Overview

The root directory provides a Go library that creates ePub files out of URLs, with limitations.

rmapi/ directory provides a Go library that implements reMarkable API, so that the ePub files generated can be sent to reMarkable paper tablet directly.

tgbot/ directory provides a Go library that implements partial Telegram bot API, so all this can be done in a Telegram message.

cloudrun/ directory provides the Google Cloud Run implementation of the Telegram Bot that does all this, and also serving REST APIs.

License

BSD 3-Clause.

url2epub's Issues

Idea for improving this Telegram Bot

Hi, thank you for this convenient way of uploading websites to the remarkable!

It would be really great if you could send a PDF to the telegram bot which was then uploaded to the remarkable cloud. Just an idea... Would that be
too complicated?

Images not showing on sanjosespotlight.com

Example URL: https://sanjosespotlight.com/san-jose-legends-rod-diridon-launched-the-citys-light-rail-but-got-into-transit-by-accident/

This is actually an interesting case. The site doesn't use AMP, and their images look like this:

<img loading="lazy"   width="1024" height="683" alt="" data-src="https://sanjosespotlight.s3.us-east-2.amazonaws.com/wp-content/uploads/2020/12/26233502/Rod-Diridon-3.jpeg-1024x683.jpg" class="size-large wp-image-66602 lazyload" src="data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw=="><noscript><img loading="lazy" src="https://sanjosespotlight.s3.us-east-2.amazonaws.com/wp-content/uploads/2020/12/26233502/Rod-Diridon-3.jpeg-1024x683.jpg" class="size-large wp-image-66602" width="1024" height="683" alt=""></noscript>

So basically they try to put a placeholder image there, lazy loading the actual image async.

I think a potential solution is to try to get the noscript -> img tag inside img tag if it's there.

Failures on ChromeUnboxed articles

Got errors in ChromeUnboxed articles with logs like this:

E tg_webhook.go:76: urlHandler: getEpub failed for "https://chromeunboxed.com/how-to-enable-nearby-sharing-in-chrome-os-87-right-now/": ziputil.WriteFile: unable to write "content/article.xhtml": html: void element <img> has child nodes

The reason is they use fallback amp-img tags:

<figure class="amp-featured-image ">
  <amp-img src="https://chromeunboxed.com/wp-content/uploads/2020/04/AndroidNearbyShareChromeOSChromebooks.jpg" srcset="https://chromeunboxed.com/wp-content/uploads/2020/04/AndroidNearbyShareChromeOSChromebooks.jpg 1920w, https://chromeunboxed.com/wp-content/uploads/2020/04/AndroidNearbyShareChromeOSChromebooks-300x169.jpg 300w, https://chromeunboxed.com/wp-content/uploads/2020/04/AndroidNearbyShareChromeOSChromebooks-1024x576.jpg 1024w, https://chromeunboxed.com/wp-content/uploads/2020/04/AndroidNearbyShareChromeOSChromebooks-768x432.jpg 768w, https://chromeunboxed.com/wp-content/uploads/2020/04/AndroidNearbyShareChromeOSChromebooks-1536x864.jpg 1536w" width="1920" height="1080" layout="responsive" alt="How to enable Nearby Sharing in Chrome OS 87 right now">
    <amp-img fallback="" src="https://chromeunboxed.com/wp-content/plugins/accelerated-mobile-pages/images/SD-default-image.png" srcset="https://chromeunboxed.com/wp-content/plugins/accelerated-mobile-pages/images/SD-default-image.png 1920w, https://chromeunboxed.com/wp-content/uploads/2020/04/AndroidNearbyShareChromeOSChromebooks-300x169.jpg 300w, https://chromeunboxed.com/wp-content/uploads/2020/04/AndroidNearbyShareChromeOSChromebooks-1024x576.jpg 1024w, https://chromeunboxed.com/wp-content/uploads/2020/04/AndroidNearbyShareChromeOSChromebooks-768x432.jpg 768w, https://chromeunboxed.com/wp-content/uploads/2020/04/AndroidNearbyShareChromeOSChromebooks-1536x864.jpg 1536w" width="1920" height="1080" layout="responsive" alt="How to enable Nearby Sharing in Chrome OS 87 right now">
    </amp-img>
  </amp-img>
</figure>

The solution would be to use ignore the childrens of an amp-img node.

Can't find articles in Remarkable

Hi!
I configured the Telegram bot nicely. When I try to send an article, everything works:

However, the articles don't appear in the Remarkable desktop app or in the Remarkable device itself. However, they do appear in the Remarkable web app.

Any clue?

https://gizmodo.com/these-ultralight-carbon-fiber-headphones-are-perfect-fo-1845915175 appears to be empty in ePub form

Likely because I missed a few html nodes in readable filter. Added header but that doesn't seem to be enough to fix it, need more investigation.

Last modified timestamp

Hi,

Thanks for this Telegram bot; I've been using it, and love how convenient it is to use.

One issue I have is that when sorting by "last updated" (most recent first), the ePub files uploaded by this bot show up at the end rather than the beginning, so they're easy to miss.

I think the fix may be a simple matter of adding a timestamp to the json somewhere and I'll try to take a look too.