GithubHelp home page GithubHelp logo

nicoweio / blinkist Goto Github PK

View Code? Open in Web Editor NEW
44.0 5.0 10.0 100 KB

Tool to download Blinkist's free offerings, namely "Free Daily" and free curated lists

Home Page: https://nicoweio.github.io/blinkist/

License: GNU Affero General Public License v3.0

Python 97.43% HTML 0.96% Dockerfile 1.61%
blinkist markdown-export scraper audiobooks

blinkist's Introduction

About this repository

Features

File formats

This program downloads books from Blinkist. To be precise, it saves…

  • the (almost) raw API responses
    • as YAML
    • This includes things like categories or ratings which are not saved elsewhere.
  • the text version
    • as Markdown
      • the unmodified HTML-formatted content + custom Markdown-formatted structure
      • 🛈 Consider using Pandoc to convert to other formats like HTML, PDF, EPUB, etc.
  • the audio version
    • as one M4A file per chapter (no modifications)
  • the cover image
    • as JPEG (no modifications)
    • in the highest resolution available (1080×1080)

Book selection

  • --book-slug: the book slug (e.g. get-smart-en)
  • --freedaily: Each day, Blinkist offers a free book for each locale.
  • Further options include --book-slug, --latest, --latest-collections, --search, and --trending. Refer to main.py --help for more information.
  • 🛈 If you pass multiple of these options, all of them will be used.

Important

It seems like Blinkist now prevents unauthorized access to books. If you have a Blinkist Premium account, check out the feature/import-session branch.

Installation

# First, clone the repository

# Then, install the dependencies
$ pip install -r requirements.txt

Installation via setuptools and pip is currently not supported. Python ≥3.9 is recommended.

Usage

Just run the main.py executable, stating what to download and providing a path to a “library directory”, where every book will be saved in its own subdirectory. Here's an example:

$ ./main.py --freedaily ~/Library/Blinkist

For an overview of all CLI options, see main.py --help. For other options, you need to modify blinkist/config.py to your needs.

Motivation

While https://github.com/ptrstn/dailyblink is/was broken due to changes to Blinkist's frontend (ptrstn/dailyblink#32), I wrote my own Blinkist downloader from scratch. In contrast to the original, this one relies on Blinkist's API, which might be more stable than scraping with beautifulsoup. Now, some aspects are even better than in the original, while others are sill lacking.

Limitations

  • Downloading arbitrary books with Blinkist Premium is not supported out of the box. It might be easy to get it working, but I didn't test it. There's other tools for that anyway.
  • ID3 tags are not set.
  • The formatting of the markdown export differs a little from @ptrstn's.
  • Occasionaly, Cloudflare gets in the way. [1] [2]

Comparison

Example data

🛈 Also check out the auto-generated Swagger API documentation.

https://www.blinkist.com/api/free_daily
{
    "book": {
        "id": "628223936cee0700089119c9",
        "kind": "book",
        "slug": "the-4-stages-of-psychological-safety-en",
        "title": "The 4 Stages of Psychological Safety",
        "subtitle": "Defining the Path to Inclusion and Innovation",
        "subtitleHtmlSafe": "Defining the Path to Inclusion and Innovation",
        "aboutTheBook": "<p><em>The 4 Stages of Psychological Safety </em>(2020) is a practical handbook for creating and maintaining psychological safety in the workplace. In order for employees to take risks, ask questions, challenge the status quo, and make mistakes –&nbsp;all while learning and growing –&nbsp;they have to feel included and safe. This book shows how leaders can reduce social friction while encouraging collaboration and innovation.</p>",
        "buyOnAmazonUrl": "/en/books/the-4-stages-of-psychological-safety-en/purchase",
        "author": "Timothy R. Clark",
        "truncatedAuthor": "Timothy R. Clark",
        "sourceAuthor": "Timothy R. Clark",
        "url": "/en/books/the-4-stages-of-psychological-safety-en",
        "browseUrl": "/en/nc/browse/books/the-4-stages-of-psychological-safety-en",
        "previewUrl": "/en/books/the-4-stages-of-psychological-safety-en",
        "readingDuration": 9,
        "minutesToRead": 9,
        "isAudio": true,
        "readCount": null,
        "image": {
            "default": {
                "src": "https://images.blinkist.io/images/books/628223936cee0700089119c9/1_1/470.jpg",
                "srcset": {
                    "2x": "https://images.blinkist.io/images/books/628223936cee0700089119c9/1_1/640.jpg"
                }
            },
            "sources": [
                {
                    "media": "xs",
                    "src": "https://images.blinkist.io/images/books/628223936cee0700089119c9/1_1/470.jpg",
                    "srcset": {
                        "2x": "https://images.blinkist.io/images/books/628223936cee0700089119c9/1_1/640.jpg"
                    }
                },
                {
                    "media": "s",
                    "src": "https://images.blinkist.io/images/books/628223936cee0700089119c9/1_1/640.jpg",
                    "srcset": {
                        "2x": "https://images.blinkist.io/images/books/628223936cee0700089119c9/1_1/1080.jpg"
                    }
                },
                {
                    "media": "m",
                    "src": "https://images.blinkist.io/images/books/628223936cee0700089119c9/1_1/250.jpg",
                    "srcset": {
                        "2x": "https://images.blinkist.io/images/books/628223936cee0700089119c9/1_1/470.jpg"
                    }
                }
            ]
        },
        "audioUrl": "https://hls.blinkist.io/bibs/628223936cee0700089119c9/628223936cee0700089119cb-T1652696046.m4a?Expires=1654617121&Signature=VuyioBHQEDE~ExCpKbib9rBYzjtxsls3EQo6ZCLN0fY~GaFiU9Cb1pV5Xzo1-4Xdef8IlRMWHXZdLFtAOpmmWqcnC2z8ySekv8wFrSmZcPxbQGdi-AstNtVMzTRQVKniy6Kx3Xc2lCswJdnwP0j3okC4Z~ijkcEn91EqTHZhtpEwkBjPEg2hX433tKnc1yFHU4DQpcbe6977fuaCyKZZjXRL4jYXRhRXgvMcqLs8ST3cS49lfzuqfG1kSJxBo7PJ~mvT9HsrSH91aEHW2XBtgfoiwrNVdxQBm9gGSHNoVun0kJa8DABagDRMdFHkr0~pF7XPfrNJGGO6DhIUVNdCmw__&Key-Pair-Id=APKAJXJM6BB7FFZXUB4A",
        "chaptersLength": 5,
        "hasAudio": true,
        "language": "en",
        "freeDaily": null,
        "category": {
            "title": "Management & Leadership",
            "sprite": "management-and-leadership",
            "slug": "management-and-leadership-en"
        },
        "averageRating": 4.2,
        "categories": [
            {
                "id": "54788e1066333100094b0000",
                "url": "/en/nc/categories/management-and-leadership-en",
                "sprite": "management-and-leadership",
                "slug": "management-and-leadership-en",
                "title": "Management & Leadership",
                "subtitle": "Every great leader has unique secrets to success, but what’s one they all agree on? Books! Read up, step up, and shine."
            },
            {
                "id": "5b868435b238e1000726ccba",
                "url": "/en/nc/categories/career-and-success-en",
                "sprite": "career-and-success",
                "slug": "career-and-success-en",
                "title": "Career & Success",
                "subtitle": "With these titles, climbing the job ladder will be as easy as 1-2-3."
            }
        ]
    },
    "endTimestamp": 1654639199
}
https://www.blinkist.com/api/books/{book_slug}/chapters
{
   "book":{},
   "chapters":[
      {
         "id":"628223936cee0700089119ca",
         "order_no":0,
         "action_title":"What’s in it for me? Learn how to encourage innovation through inclusion in your team or organization."
      },
      {
         "id":"628223936cee0700089119cb",
         "order_no":1,
         "action_title":"To create inclusion safety, make sure team members feel unconditionally included from the very beginning."
      },
      {
         "id":"628223936cee0700089119cc",
         "order_no":2,
         "action_title":"To provide learner safety, create an environment where failure isn’t just accepted – it’s rewarded."
      },
      {
         "id":"628223936cee0700089119cd",
         "order_no":3,
         "action_title":"To provide contributor safety, get to know your team, limit your tell-to-ask ratio, and help colleagues think beyond their roles."
      },
      {
         "id":"628223936cee0700089119ce",
         "order_no":4,
         "action_title":"Democratize innovation by fostering challenger safety."
      },
      {
         "id":"628223936cee0700089119cf",
         "order_no":5,
         "action_title":"Final summary"
      }
   ],
   "current_chapter_id":"None"
}
https://www.blinkist.com/api/books/{book_id}/chapters/{chapter_id}
{
   "id":"628223936cee0700089119ca",
   "order_no":0,
   "action_title":"What’s in it for me? Learn how to encourage innovation through inclusion in your team or organization.",
   "text":"<p>Congrats! You’re in the luxurious position of choosing between two teams you could work with. Let’s go ahead and meet them.</p>\\n<p>\\n </p>\\n<p>This is the first team’s office. Notice that? The air is stiff. [shortened as to not violate their copyright]</p>",
   "audio_url":"https://hls.blinkist.io/bibs/628223936cee0700089119c9/628223936cee0700089119ca-T1652696046.m4a",
   "signed_audio_url":"https://hls.blinkist.io/bibs/628223936cee0700089119c9/628223936cee0700089119ca-T1652696046.m4a?Expires=1654621635&Signature=PFcksN0ISh~J6YjzWQKsJYaQUbmW0Cl~ct4qtiIsfDPxrXjyYxorafH~TdCP4bYsjSuuOeDp1BCEkLtO0HWm3EsLc1T5Cv7LRIS7yuuHpR6GK~72DjKDQBPGWx4JZsWv0Au1VegwfYHEU4sFaz9VvahcJg5u3~FufSEhgygTC3SOGpgfsRTIAOfkvXPhet-d~8u0KAHZudHHkBEVl1w804abVfW-30uvxyuSBBViTkI7r74RyJt~ui42mMO8s314vz6wdMNSgLmF-blKDwU0xXTnskIdSOHI~PS6TT4PEQS~pf1KfsUDLrhr8P61TzUHkCZtricCD1udzRLjYLpEzA__&Key-Pair-Id=APKAJXJM6BB7FFZXUB4A"
}
https://api.blinkist.com/contentaccess/free_items
{
  "items": [
    {
      "item_type": "curated_list",
      "item_id": "f8a49868-a6f3-40fd-b264-2348770c6815"
    },
    {
      "item_type": "book",
      "item_id": "52bec76a3933330008000000"
    },
    {
      "item_type": "episode",
      "item_id": "1055"
    },
    
  ]
}
https://api.blinkist.com/content/curated_lists/f8a49868-a6f3-40fd-b264-2348770c6815
{
  "curated_list": {
    "id": "1495",
    "position": -1,
    "uuid": "f8a49868-a6f3-40fd-b264-2348770c6815",
    "slug": "how-to-lead-a-team-you-didn-t-hire",
    "title": "How To Lead A Team You Didn't Hire",
    "description": "The Great Resignation has caused many leaders to pursue new roles, often taking charge of pre-existing departments and teams. Starting a new position can be its own steep learning curve, but the margin for error is even smaller when you are managing a team you didn’t hire. How do you earn this team’s respect and make accurate assessments of its current challenges without the context of someone who built it from the ground up? The following two Blinks and two Shortcasts investigate what it means to be a great leader under these circumstances.",
    "short_description": "Four free Blinks and Shortcasts full of tips on how to be a great leader. ",
    "curator_name": "Sally Page",
    "curator_id": "blinkist",
    "etag": 1658845113,
    "language": "en",
    "discoverable": false,
    "published_at": "2022-07-26T14:18:33.000Z",
    "deleted_at": null,
    "kind": "collection",
    "styling": {
      "main_color": null,
      "accent_color": null,
      "text_color": null,
      "text_on_accent_color": null
    },
    "content_items": [
      {
        "id": "14380",
        "position": 1,
        "content_item_type": "book",
        "content_item_id": "52bec76a3933330008000000",
        "description": ""
      },
      {
        "id": "14381",
        "position": 2,
        "content_item_type": "book",
        "content_item_id": "52f1195c35653600110b0000",
        "description": ""
      },
      {
        "id": "14382",
        "position": 3,
        "content_item_type": "episode",
        "content_item_id": "1055",
        "description": ""
      },
      {
        "id": "14383",
        "position": 4,
        "content_item_type": "episode",
        "content_item_id": "1000",
        "description": ""
      }
    ]
  }
}

blinkist's People

Contributors

dependabot[bot] avatar iki avatar manuelschneid3r avatar nicoweio avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

blinkist's Issues

TypeError: argument of type 'NoneType' is not iterable

Hello, I get this error when trying to grab the free curated blinks?

Traceback (most recent call last):
  File "/root/blinkist/./main.py", line 121, in <module>
    main()
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/root/blinkist/./main.py", line 113, in main
    download_book(
  File "/root/blinkist/./main.py", line 58, in download_book
    chapter.download_audio(book_dir)
  File "/root/blinkist/blinkist/chapter.py", line 33, in download_audio
    assert 'm4a' in self.data['signed_audio_url']
TypeError: argument of type 'NoneType' is not iterable

No audio downloaded for this book?

Hello,

Love this tool!

Is there any way to get a verbose or debug log?

This book downloads no audio: https://www.blinkist.com/en/books/the-culture-map-en

It shows downloading chapters, and then this final result:

./main.py -r --book-slug the-culture-map-en ./
Book: “The Culture Map”
~/blinkist/the-culture-map-en# ls -l
total 218
-rw-r--r-- 1 root root  21185 Jan 13 18:25 book.md
-rw-r--r-- 1 root root  25892 Jan 13 18:25 book.yaml
-rw-r--r-- 1 root root 186744 Jan 13 18:25 cover.jpg

object of type 'generator' has no len()

Hi,
Thank you for great!

While downloading this German daily Book (03/30/2024), I found this error. Could please help us fix it?
I put this attribute in next attempt.

[01:23:56] INFO Book: “Das Charisma-Geheimnis” main.py:191
[01:24:43] ERROR Error downloading “Das Charisma-Geheimnis”: object of type 'generator' has no len() main.py:93
INFO Keeping temporary output directory “das-charisma-geheimnis-de.tmp2” main.py:94
INFO Continuing with next book… (--continue-on-error was set) main.py:97
Downloading audio… ⠏ ━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━ 5/12 0:00:55

Thank you.

50% of books can't be downloaded

Not sure why, but out of 10 books I'm trying to download, at least 50% of them is not going through. The tool reporting this error:

[20:08:26] ERROR    Error downloading “Playing to Win”: RetryError[<Future at 0x104f77fd0 state=finished raised CloudflareChallengeError>]   main.py:93

This happens with certain titles and not the others. For example, it works with "myths-of-strategy-en" but fails with "playing-to-win-en-ag-lafley-and-roger-l-martin"

--freedaily option the only one working

Hi,

Maybe it is some web app update on cloudflare that is blocking everything else but --freedaily option?

I'm using the repo's latest code. I'm getting this error message when using --book-slug option:

`
E:\blinkist\software>python main.py --book-slug the-highly-sensitive-person-en e:\blinkist\resumenes
[11:17:52] INFO Book: “The Highly Sensitive Person” main.py:189
[11:17:53] INFO Retrying in 2.0 seconds… common.py:17
[11:17:55] INFO Retrying in 2.0 seconds… common.py:17
[11:17:58] INFO Retrying in 2.0 seconds… common.py:17
[11:18:01] INFO Retrying in 2.0 seconds… common.py:17
[11:18:03] INFO Retrying in 2.0 seconds… common.py:17
[11:18:06] INFO Retrying in 2.0 seconds… common.py:17
[11:18:09] INFO Retrying in 2.0 seconds… common.py:17
[11:18:11] INFO Retrying in 2.0 seconds… common.py:17
[11:18:14] INFO Retrying in 2.0 seconds… common.py:17
[11:18:16] ERROR Error downloading “The Highly Sensitive Person”: RetryError[<Future at 0x17e4afbf940 main.py:83
state=finished raised CloudflareChallengeError>]
WARNING Renamed output directory to “the-highly-sensitive-person-en – ERROR” main.py:92
CRITICAL Exiting… main.py:97
CRITICAL Hint: Try using --continue-on-error. main.py:98
Fetching chapters… ⠦ ---------------------------------------- 0/4 -:--:--
Traceback (most recent call last):
File "C:\Users\sicon\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\tenacity_init_.py", line 382, in call
result = fn(*args, **kwargs)
File "E:\blinkist\software\blinkist\common.py", line 30, in request
raise cloudscraper.exceptions.CloudflareChallengeError()
cloudscraper.exceptions.CloudflareChallengeError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "E:\blinkist\software\main.py", line 198, in
main()
File "C:\Users\sicon\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\click\core.py", line 1157, in call
return self.main(*args, **kwargs)
File "C:\Users\sicon\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\click\core.py", line 1078, in main
rv = self.invoke(ctx)
File "C:\Users\sicon\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\click\core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Users\sicon\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\click\core.py", line 783, in invoke
return callback(*args, **kwargs)
File "E:\blinkist\software\main.py", line 191, in main
download_book(
File "E:\blinkist\software\main.py", line 57, in download_book
_ = book.chapters
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\functools.py", line 981, in get
val = self.func(instance)
File "E:\blinkist\software\blinkist\book.py", line 54, in chapters
chapters = [
File "E:\blinkist\software\blinkist\book.py", line 55, in
Chapter.from_id(self, chapter['id'])
File "E:\blinkist\software\blinkist\chapter.py", line 16, in from_id
chapter_data = api_request_web(f'books/{book.id}/chapters/{chapter_id}')
File "E:\blinkist\software\blinkist\common.py", line 52, in api_request_web
return api_request('https://blinkist.com/api/', endpoint, params=params)
File "E:\blinkist\software\blinkist\common.py", line 43, in api_request
response = request(url, params=params, headers=HEADERS)
File "C:\Users\sicon\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\tenacity_init
.py", line 289, in wrapped_f
return self(f, *args, **kw)
File "C:\Users\sicon\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\tenacity_init
.py", line 379, in call
do = self.iter(retry_state=retry_state)
File "C:\Users\sicon\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\tenacity_init_.py", line 326, in iter
raise retry_exc from fut.exception()
tenacity.RetryError: RetryError[<Future at 0x17e4afbf940 state=finished raised CloudflareChallengeError>]`

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.