GithubHelp home page GithubHelp logo

girbons / comics-downloader Goto Github PK

View Code? Open in Web Editor NEW
478.0 12.0 51.0 60.83 MB

tool to download comics and manga in pdf/epub/cbr/cbz from a website

License: MIT License

Go 98.10% Makefile 1.90%
comics-downloader comicextra go windows osx linux command-line-tool download-comics pdf manga-downloader

comics-downloader's Introduction

Hi there ๐Ÿ‘‹

comics-downloader's People

Contributors

alexzandev avatar bake avatar bhm avatar christoffergreen avatar dependabot[bot] avatar girbons avatar hakuamesan avatar ilmanzo avatar ivanlemeshev avatar jan0660 avatar jkernech avatar jphager2 avatar kavishgr avatar kshunterco avatar nbsmith avatar silvanocerza avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

silvanocerza layzhi nazgaret thanhtoan1196 francoispoinsot cupello jestraw jphager2 eyal-shalev corentinb jnhoang mario-nguyen jayd2446 asdbaihu yang123vc bake alextorquin blisszard oteher jkernech hakuamesan alex-finner blahblahblahh01 wavecoworking ccchou kshunterco anangel vemuaz anjandev justinknight93 lordzizzy heisenbergsongyi sergeypdev kupacariibumu bhm mfrank2016 konstantindjairo j3s5e phantomeis chiwishen simrit1 nakedlittlezombie thepinkpanthercrypto cattingnum cwdmaharanej mohamad-supangat x13igpoppa toxanetoxa aryamanhm anhdung2k1 nbsmith

comics-downloader's Issues

Issue with new cbr/cbz format

Hi there, when downloading a comic using -format=cbz, the cbz reader reports "no images in zip". So I expanded out the cbz to have a look, and the images are in there, but the file names have random numbers appended behind the .jpg. So the reader does not understand.

If I rename them, create a zip file and change the name, I can view the files as intended.

demo ๎‚ฐ tree
.
|-- 0.image.jpg257861582
|-- 1.image.jpg549992149
|-- 10.image.jpg903726552
|-- 11.image.jpg812153687
|-- 12.image.jpg957441226
|-- 13.image.jpg895692705
|-- 14.image.jpg714702988
|-- 15.image.jpg323022971
|-- 16.image.jpg832890270
|-- 17.image.jpg326568037
|-- 18.image.jpg859275904
|-- 19.image.jpg207434207
|-- 2.image.jpg330143280
|-- 20.image.jpg469486002
|-- 21.image.jpg275909993
|-- 22.image.jpg911042484
|-- 23.image.jpg505615747
|-- 24.image.jpg547160838
|-- 25.image.jpg207757485
|-- 3.image.jpg115958223
|-- 4.image.jpg213762274
|-- 5.image.jpg140432089
|-- 6.image.jpg141762148
|-- 7.image.jpg150061683
|-- 8.image.jpg692767542
|-- 9.image.jpg115787037
`-- chapter-1.cbz

I have tested with both cbr and cbz.

Question on mangadex behavior

So when attempting to download all issues of a manga off mangadex say via

 /opt/bin/comic-dl -url=https://mangadex.org/title/23569/gokushufudou-the-way-of-the-house-husband -all -images-only -output /media/Media/Comics/

It will return

[noah@nixos:/media/Media/Comics/manga/mangadex.org]$ /opt/bin/comic-dl -url=https://mangadex.org/title/23569/gokushufudou-the-way-of-the-house-husband -all -images-only -output /media/Media/Comics/
WARN[0000] `all` and `last` parameters are not supported  site="https://mangadex.org/title/23569/gokushufudou-the-way-of-the-house-husband"
INFO[0000] Downloading...                                url="https://mangadex.org/title/23569/gokushufudou-the-way-of-the-house-husband"
ERRO[0000] could not get chapter 23569: got unexpected status: error

What exactly is the correct method to download all issues for a manga on magadex then? I assumed -all would be supported.

panic: runtime error: invalid memory address or nil pointer dereference

Hi!

I encountered this error when downloading many chapters at once.
This particular one got up to 65 chapters, on other 'runs' it would panic somewhere between 30-50 chapters.

Here's the command I used:
.\comics-downloader.exe -url https://mangadex.org/title/22369/jahy-sama-wa-kujikenai -output "T:\Manga\Jahy-sama Won't Be Discouraged!" -format epub

(The output is on a mapped network drive, not sure if relevant, but hey.)

  36% |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ                          |  [3s:5s]panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x20 pc=0x670e20]

goroutine 1 [running]:
image/jpeg.Encode(0x900ec0, 0xc000154000, 0x0, 0x0, 0x0, 0x3, 0x900c80)
        /usr/local/Cellar/go/1.13/libexec/src/image/jpeg/writer.go:576 +0x30
github.com/Girbons/comics-downloader/pkg/util.SaveImage(0x900ec0, 0xc000154000, 0x16a0028, 0xc000188500, 0x869d60, 0x3, 0x0, 0x0)
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/util/util.go:171 +0x228
github.com/Girbons/comics-downloader/pkg/core.(*Comic).DownloadImages(0xc000142b40, 0xc00000c3c0, 0x28, 0x0, 0x0, 0x0, 0x0)
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/core/core.go:293 +0x64d
github.com/Girbons/comics-downloader/pkg/core.(*Comic).makeEPUB(0xc000142b40, 0xc00000c3c0, 0x28, 0x0, 0x0)
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/core/core.go:96 +0x23a
github.com/Girbons/comics-downloader/pkg/core.(*Comic).MakeComic(0xc000142b40, 0xc00000c3c0, 0x28, 0xc0003b8630, 0x0)
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/core/core.go:318 +0xee
github.com/Girbons/comics-downloader/cmd/app.download(0xc00000e380, 0x37, 0xc00000a118, 0x4, 0x0, 0x0, 0x0, 0x869d60, 0x3, 0xc00000c3c0, ...)
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/cmd/app/downloader.go:76 +0x4d9
github.com/Girbons/comics-downloader/cmd/app.Run(0xc00000e380, 0x37, 0xc00000a118, 0x4, 0x0, 0x0, 0x869d60, 0x3, 0x0, 0x258, ...)
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/cmd/app/downloader.go:111 +0x1e1
main.main()
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/cmd/downloader/main.go:44 +0x5a2

I can still download all of the chapters by restarting the download with the same command, but it is a bother still.
Thanks for the nifty scraper though!

build error 

$ make linux-build

cmd/downloader/main.go:42:9: too many arguments in call to app.Run
	have (string, string, string, bool, bool)
	want (string, string, string, bool)
make: *** [Makefile:19: linux-build] Error 2

Installation issue Mac OS

I'm new to GoLang, do I need to install Go on a Mac before running your script?

--> brew install golang

Thanks

runtime error on -all argument with a supported website

Describe the bug
Hello, I got a panic: runtime error: index out of range [4] with length 4
when I try to download all the issue of a manga on mangareader.net
I can download specific issue but not all. I tried with 2 different manga.
maybe I use it wrongly.

To Reproduce
On widows 10 with the command
comics-downloader.exe -all -format=cbz -url=http://www.mangareader.net/akira
or
comics-downloader.exe -all -format=cbz -url=http://www.mangareader.net/mahou-sensei-negima

Expected behavior
The program should create a cbz file of the manga with all issue.

Screenshots

>comics-downloader.exe -url=http://www.mangareader.net/akira -images-only -format=cbz -all
time="2020-04-14T17:58:19+02:00" level=info msg=Downloading... url="http://www.mangareader.net/akira"
panic: runtime error: index out of range [4] with length 4

goroutine 1 [running]:
github.com/Girbons/comics-downloader/pkg/sites.(*Mangareader).GetInfo(0xb96030, 0xc00000e335, 0x20, 0x8, 0xc000107808, 0x74bdd5, 0x0)
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/sites/mangareader.go:104 +0x8a
github.com/Girbons/comics-downloader/pkg/sites.initializeCollection(0xc00010a200, 0x6, 0x8, 0xc000107f08, 0x8dc660, 0xb96030, 0x8, 0x0, 0x0, 0x0, ...)
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/sites/loader.go:21 +0xe8
github.com/Girbons/comics-downloader/pkg/sites.LoadComicFromSource(0xc000107f08, 0x4, 0xc0001079e8, 0x1, 0x1, 0x0)
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/sites/loader.go:75 +0x15c
github.com/Girbons/comics-downloader/cmd/app.download(0xc000107f08, 0x0)
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/cmd/app/downloader.go:79 +0x593
github.com/Girbons/comics-downloader/cmd/app.Run(0xc000107f08)
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/cmd/app/downloader.go:125 +0x90
main.main()
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/cmd/downloader/main.go:83 +0x24c

Desktop (please complete the following information):

  • OS: Windows 10 Pro
  • Version v0.22.2
  • Type CLI

Thank you for your time.

segfault in pre-built binary and locally compiled.

Describe the bug
My 20.04 system throws a segfault when starting to download (it appears). The binary appears to start up fine, then segfaults.

โฏ ./comics-downloader 'https://www.comicextra.com/comic/invincible'   
INFO[0000] Downloading...                                url="https://www.comicextra.com/comic/invincible"
   0% |                                        |  [0s:0s]panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x67bea9]

goroutine 1 [running]:
image/jpeg.Encode(0x8fde20, 0xc0010ce008, 0x0, 0x0, 0x0, 0x0, 0x8fd900)
	/usr/lib/go-1.13/src/image/jpeg/writer.go:576 +0x29
github.com/Girbons/comics-downloader/pkg/util.SaveImage(0x8fde20, 0xc0010ce008, 0x7fc05d748160, 0xc0010a0690, 0x8658d2, 0x3, 0x0, 0xb)
	/home/bdmorin/Projects/comics-downloader/pkg/util/util.go:171 +0x221
github.com/Girbons/comics-downloader/pkg/core.(*Comic).DownloadImages(0xc0001981b0, 0x7ffeee4174ef, 0x1, 0x0, 0x0, 0x0, 0x0)
	/home/bdmorin/Projects/comics-downloader/pkg/core/core.go:262 +0x696
github.com/Girbons/comics-downloader/pkg/core.(*Comic).makePDF(0xc0001981b0, 0x7ffeee4174ef, 0x1, 0x0, 0x0)
	/home/bdmorin/Projects/comics-downloader/pkg/core/core.go:119 +0xcd
github.com/Girbons/comics-downloader/pkg/core.(*Comic).MakeComic(0xc0001981b0, 0x7ffeee4174ef, 0x1, 0x100, 0x0)
	/home/bdmorin/Projects/comics-downloader/pkg/core/core.go:291 +0xbe
github.com/Girbons/comics-downloader/cmd/app.download(0xc000193ee0, 0xc000103e00)
	/home/bdmorin/Projects/comics-downloader/cmd/app/downloader.go:90 +0x65f
github.com/Girbons/comics-downloader/cmd/app.Run(0xc000103ee0)
	/home/bdmorin/Projects/comics-downloader/cmd/app/downloader.go:125 +0x8c
main.main()
	/home/bdmorin/Projects/comics-downloader/cmd/downloader/main.go:83 +0x245

To Reproduce
I've tried with the stock binary and a self-compiled one.

I've tried several urls, this one starts to work, but then fails.

โฏ ./comics-downloader  -all -format cbr -output ~/Files/comics -url 'https://www.comicextra.com/comic/the-boys'
INFO[0000] Downloading...                                url="https://www.comicextra.com/comic/the-boys"
 100% |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ|  [1s:0s]INFO[0007] CBR file correctly saved                     
 100% |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ|  [1s:0s]INFO[0009] CBR file correctly saved                     
 100% |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ|  [1s:0s]INFO[0010] CBR file correctly saved                     
 100% |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ|  [1s:0s]INFO[0011] CBR file correctly saved                     
 100% |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ|  [1s:0s]INFO[0012] CBR file correctly saved                     
 100% |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ|  [1s:0s]INFO[0013] CBR file correctly saved                     
 100% |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ|  [1s:0s]INFO[0014] CBR file correctly saved                     
 100% |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ|  [1s:0s]INFO[0016] CBR file correctly saved                     
 100% |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ|  [1s:0s]INFO[0017] CBR file correctly saved                     
 100% |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ|  [1s:0s]INFO[0018] CBR file correctly saved                     
 100% |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ|  [1s:0s]INFO[0019] CBR file correctly saved                     
   0% |                                        |  [0s:0s]panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x67bea9]

goroutine 1 [running]:
image/jpeg.Encode(0x8fde20, 0xc000904238, 0x0, 0x0, 0x0, 0x0, 0x8fd900)
	/usr/lib/go-1.13/src/image/jpeg/writer.go:576 +0x29
github.com/Girbons/comics-downloader/pkg/util.SaveImage(0x8fde20, 0xc000904238, 0x7f428000c840, 0xc0006ac660, 0x8658d2, 0x3, 0x0, 0xa)
	/home/bdmorin/Projects/comics-downloader/pkg/util/util.go:171 +0x221
github.com/Girbons/comics-downloader/pkg/core.(*Comic).DownloadImages(0xc0000e63f0, 0x7ffc815cc4b8, 0x1, 0x0, 0x0, 0x0, 0x0)
	/home/bdmorin/Projects/comics-downloader/pkg/core/core.go:262 +0x696
github.com/Girbons/comics-downloader/pkg/core.(*Comic).makeCBRZ(0xc0000e63f0, 0x7ffc815cc4b8, 0x1, 0x0, 0x0)
	/home/bdmorin/Projects/comics-downloader/pkg/core/core.go:170 +0x93
github.com/Girbons/comics-downloader/pkg/core.(*Comic).MakeComic(0xc0000e63f0, 0x7ffc815cc4b8, 0x1, 0xc0000e61b0, 0x0)
	/home/bdmorin/Projects/comics-downloader/pkg/core/core.go:289 +0x62
github.com/Girbons/comics-downloader/cmd/app.download(0xc000659ee0, 0x0)
	/home/bdmorin/Projects/comics-downloader/cmd/app/downloader.go:90 +0x65f
github.com/Girbons/comics-downloader/cmd/app.Run(0xc0000efee0)
	/home/bdmorin/Projects/comics-downloader/cmd/app/downloader.go:125 +0x8c
main.main()
	/home/bdmorin/Projects/comics-downloader/cmd/downloader/main.go:83 +0x245

Expected behavior
The URL downloads as expected and packages the download in the requested format.

Desktop (please complete the following information):

โฏ neofetch --off 
bdmorin@phyzz 
------------- 
OS: Pop!_OS 20.04 LTS x86_64 
Host: Oryx Pro oryp5 
Kernel: 5.4.0-7634-generic 
Uptime: 7 hours, 50 mins 
Packages: 3645 (dpkg), 10 (brew), 30 (flatpak), 28 (snap) 
Shell: zsh 5.8 
Resolution: 2560x1080, 1920x1080 
DE: GNOME 
WM: Mutter 
WM Theme: Pop 
Theme: Pop-dark [GTK2/3] 
Icons: Pop [GTK2/3] 
Terminal: gnome-terminal 
CPU: Intel i7-9750H (12) @ 4.500GHz 
GPU: Intel UHD Graphics 630 
GPU: NVIDIA GeForce RTX 2060 Mobile 
Memory: 10573MiB / 31827MiB 

โฏ go version
go version go1.13.8 linux/amd64

Additional context
I'd like to help, but I'm out of my depth with segfaults.

Add option to check if chapter exists before downloading

First thanks for all the work on this project.

It could be usefull to add an option to check if the chapter doesn't exist in the download directory. It would allow user to use the -all option to download new chapter of the comic without re-downloading the previous ones.

Multiple URLs

First off thanks for this nice tool. If possible, being able to pass multiple URLs to the -url parameter and have them download in a single command is something I would find convenient.

[Question][Possible bug] Where are the files stored?

Hi, I have the following problem:

./Downloads/comics-downloader -format cbz -url https://mangarock.com/manga/mrs-serie-162959/chapter/mrs-chapter-162962

This is the command I used to test this application. It proceeds to download the chapter, but the chapter is nowhere to be seen, after it finishes its execution. Application has necessary privileges. I did a
find -name *.cbz, but it failed to find anything. Same happens with pdfs. Also is there an option to download whole manga and not a single chapter? Thanks for your contribution to this project!

Permission Errors

Ok, I have attempted to use the app in the following way:

cd ~/Downloads && mkdir comics
mv /path/to/comics-downloader ~/home/Downloads/comics
~/home/Downloads/comics
./comics-downloader --url=https://www.comicextra.com/captain-marvel-2016/chapter-10/full

This then resulted in the following error(s):

ERRO[0001] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/1.jpgGet http://localhost/images/manga/captain-marvel-2016/10/1.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0001] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/2.jpgGet http://localhost/images/manga/captain-marvel-2016/10/2.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0001] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/3.jpgGet http://localhost/images/manga/captain-marvel-2016/10/3.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0002] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/4.jpgGet http://localhost/images/manga/captain-marvel-2016/10/4.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0002] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/5.jpgGet http://localhost/images/manga/captain-marvel-2016/10/5.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0002] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/6.jpgGet http://localhost/images/manga/captain-marvel-2016/10/6.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0002] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/7.jpgGet http://localhost/images/manga/captain-marvel-2016/10/7.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0003] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/8.jpgGet http://localhost/images/manga/captain-marvel-2016/10/8.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0003] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/9.jpgGet http://localhost/images/manga/captain-marvel-2016/10/9.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0003] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/10.jpgGet http://localhost/images/manga/captain-marvel-2016/10/10.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0004] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/11.jpgGet http://localhost/images/manga/captain-marvel-2016/10/11.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0004] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/12.jpgGet http://localhost/images/manga/captain-marvel-2016/10/12.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0004] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/13.jpgGet http://localhost/images/manga/captain-marvel-2016/10/13.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0004] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/14.jpgGet http://localhost/images/manga/captain-marvel-2016/10/14.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0005] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/15.jpgGet http://localhost/images/manga/captain-marvel-2016/10/15.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0005] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/16.jpgGet http://localhost/images/manga/captain-marvel-2016/10/16.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0005] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/17.jpgGet http://localhost/images/manga/captain-marvel-2016/10/17.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0006] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/18.jpgGet http://localhost/images/manga/captain-marvel-2016/10/18.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0006] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/19.jpgGet http://localhost/images/manga/captain-marvel-2016/10/19.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0006] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/20.jpgGet http://localhost/images/manga/captain-marvel-2016/10/20.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0007] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/21.jpgGet http://localhost/images/manga/captain-marvel-2016/10/21.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0007] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/22.jpgGet http://localhost/images/manga/captain-marvel-2016/10/22.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0007] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/23.jpgGet http://localhost/images/manga/captain-marvel-2016/10/23.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0007] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/24.jpgGet http://localhost/images/manga/captain-marvel-2016/10/24.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0008] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/25.jpgGet http://localhost/images/manga/captain-marvel-2016/10/25.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0008] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/26.jpgGet http://localhost/images/manga/captain-marvel-2016/10/26.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
ERRO[0008] Something went wrong with url: http://readcomics.io/images/manga/captain-marvel-2016/10/27.jpgGet http://localhost/images/manga/captain-marvel-2016/10/27.jpg: dial tcp 127.0.0.1:80: connect: connection refused 
 100% |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ|  [8s:0s]ERRO[0008] There was an error while making the PDF: Get http://localhost/images/manga/captain-marvel-2016/10/1.jpg: dial tcp 127.0.0.1:80: connect: connection refused 

INFO[0008] Download failed...

And the permissions on the directoies are:

ls -al ../comics/
total 8136
drwxrwxr-x 3 dustin dustin    4096 Dec 21 12:26 .
drwx------ 4 dustin dustin    4096 Dec 21 12:26 ..
drwxrwxr-x 3 dustin dustin    4096 Dec 21 12:26 comics
-rwxrwxr-x 1 dustin dustin 8312682 Dec 21 12:23 comics-downloader

Am I missing something?

Exec format error

Hi there - I was trying to run the release on Ubuntu 18.04 (64 bit), and received:

Error

./comics-downloader: cannot execute binary file: Exec format error

System

$ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.1 LTS"

File Type

$ file comics-downloader 
comics-downloader: Mach-O 64-bit x86_64 executable

Permissions

ls -al | grep comic
-rwxr-xr-x 1 dustin dustin 8268964 Dec 20 18:10 comics-downloader

Any clues?

comicextra -all flag is not using issueNumber on the output filename

Describe the bug
The chapter (aka issues) are overwriting each other's output files. Every file is named the same xyz.cbz in the same directory.

The issueNumber is not being included in the output filename. This causes the next chapter's .cbz or .pdf file to override previously completed one because the destinations are always the same.

To Reproduce

./comic-dl -format cbz -all -url https://www.comicextra.com/comic/xyz

Expected behavior

I expected to have each chapter/issue as its own new file: xyz-001.cbz, xyz-002.cbz, xyz-003.cbz

Desktop (please complete the following information):

  • OS: Linux Ubuntu 5.3.0-46-generic x86_64
  • Version: v0.22.2
  • Type: CLI

Additional context

Renaming each .cbz manually in another shell before the next issue completes shows me it is correctly grabbing the next issue. The same thing happens when using .pdf as the output.

cd comics/www.comicextra.com/comic
mv xyz.cbz xyz-100.cbz
# wait manually to see the new file show up
mv xyz.cbz xyz-099.cbz

etc...

stat -c "%y %s %n" *
2020-04-21 21:09:01.471668081 +0000 23640516 xyz-098.cbz
2020-04-21 21:08:42.815279882 +0000 23423476 xyz-099.cbz
2020-04-21 21:08:23.302851654 +0000 37556936 xyz-100.cbz

Question on behavior of downloader

Does this currently support the ability to check if issues are already downloaded when using say -all and skipping them instead of re-downloading?

Fails to download chapters which have a . in the name 

Version: Comics Downloader Windows GUI v0.14
URL: https://mangarock.com/manga/mrs-serie-100399051
Output: PDF

I faced an issue while trying to download Chapter 13.5

  1. It was saved without a .pdf extension
  2. It was 0 bytes in size
  3. Other chapters all downloaded fine

Error on new download - comicextra

Version

./comics-downloader --version
comics-downloader version v0.22.0

cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=19.10
DISTRIB_CODENAME=eoan
DISTRIB_DESCRIPTION="Ubuntu 19.10"

Command

./comics-downloader -url=https://www.comicextra.com/comic/xxxxxxxxxxxxx -format=cbz -all

Error

INFO[0000] Downloading...                                url="https://www.comicextra.com/comic/xxxxxxxxxxxxx"
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x77115a]

goroutine 1 [running]:
github.com/anaskhan96/soup.findAllofem.func1(0x0, 0xc0003a8c20, 0x1, 0x1, 0x0)
	/Users/girbons/go/pkg/mod/github.com/anaskhan96/[email protected]/soup.go:370 +0x8a
github.com/anaskhan96/soup.findAllofem(0x0, 0xc0003a8c20, 0x1, 0x1, 0x7b7700, 0x7ddaa0, 0xc000127001, 0x8)
	/Users/girbons/go/pkg/mod/github.com/anaskhan96/[email protected]/soup.go:374 +0xf0
github.com/anaskhan96/soup.Root.FindAll(0x0, 0x0, 0x0, 0x8df940, 0xc0003a8c10, 0xc0003a8c20, 0x1, 0x1, 0x0, 0x0, ...)
	/Users/girbons/go/pkg/mod/github.com/anaskhan96/[email protected]/soup.go:136 +0x6c
github.com/Girbons/comics-downloader/pkg/sites.(*Comicextra).RetrieveIssueLinks(0xba55e8, 0x7ffc8b0bb1c7, 0x38, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/sites/comicextra.go:100 +0x423
github.com/Girbons/comics-downloader/pkg/sites.LoadComicFromSource(0xc00007ff08, 0x4, 0xc0000439e8, 0x1, 0x1, 0x0)
	/Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/sites/loader.go:70 +0xd5
github.com/Girbons/comics-downloader/cmd/app.download(0xc00007ff08, 0x0)
	/Users/girbons/go/src/github.com/Girbons/comics-downloader/cmd/app/downloader.go:76 +0x546
github.com/Girbons/comics-downloader/cmd/app.Run(0xc000043f08)
	/Users/girbons/go/src/github.com/Girbons/comics-downloader/cmd/app/downloader.go:122 +0x89
main.main()
	/Users/girbons/go/src/github.com/Girbons/comics-downloader/cmd/downloader/main.go:83 +0x245

Thank you.

Output File name option

Currently, the app outputs files with a name such as:

$ ls                                                       
chapter-10.cbz  chapter-1.cbz  chapter-4.cbz  chapter-7.cbz
chapter-11.cbz  chapter-2.cbz  chapter-5.cbz  chapter-8.cbz

Sitting in a folder something like /home/user/path/to/comics/www.site.com/comic-name.

It would be great if the output name was comic-name-0001.cbz. The reason is then the sorting in a directory of large comics is displayed properly (say you had something like Action COmics that is over issue 1000). And the other reason for the file name is that for the comic readers the simply scan a single directory - you can dump them all into a single directory if desired. Then if you combine this request with the other request that was open to specify an output path, you can automate downloads quite easy.

path option

User should be able to specify where the comics should be downloaded.
This option should be included even in the config file.

not working on rapsberry pi

in fact. it returns
-sh: ./comics-downloader: cannot execute binary file: Exec format error
I believe you've compiled the linux version for amd64, right?

it would be nice having comics-downloader for standard raspian on a raspberry pi. Just with a daily cron script, a bit of bash/python & a syncthing folder, you can have manga issues delivered to your GUI device of choice automatically

Awesome program by the way

Mangadex no longer working, the sequel

Had this exact problem a couple of months ago, but then it started working again after just a couple of hours which made me remove the issue, now it's been two days where I get this same error message: 'could not unmarshal chapter XXXXXXX: json: cannot unmarshal bool into Go struct field Chapter.long_strip of type int' on everything I try to download on Mangadex.

I use the following command: comics-downloader -format cbz

Have tried using the 0.23.0 release as well as compile it myself from source

Mangadex no longer working

Getting:

ERRO[0000] could not unmarshal chapter XXXXXX: json: cannot unmarshal bool into Go struct field Chapter.long_strip of type int

Using latest version v0.22.4 on Linux 64bit

MangaDex: Revert to .org

Hi, MangaDex' .cc domain was only a temporary solution, it's back on .org. It might be a good idea to revert e1223aa and/or add .cc and .org to the list of supported domains. In the same commit, you accidentally removed a slash :)

SIGSEGV segmentation fault with runtime error : MANGATOWN

Describe the bug
Runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x1375265]

To Reproduce
Steps to reproduce the behavior:
Running the cmd:
./comics-downloader -url=https://www.mangatown.com/manga/monster/v01/c001

Expected behavior
Manga to be downloaded successfully.

Screenshots
Screenshot 2020-04-21 at 02 03 22

Desktop (please complete the following information):

  • OS: MacOS
  • Version: v0.22.1
  • Type: cmd

Alternate provider

Do you plan to add any alternative providers to comicextra? Sometimes certain issues are missing. Could be a good idea to be able to specify a backup URL if an issue is missing.

Just an idea.

Mangadex no longer working

Just putting it here in the hopes that this great application hasn't been abandoned. Mangadex naturally no longer working after moving to a new host and address.

specifying -format cbr creates a zip instead

When executing with -format cbr flag, it should create a .cbr file, rar compressed. Instead the utility creates a zip compressed file with a .cbr extension.


$ file one-piece-891.cbr

one-piece-891.cbr: Zip archive data, at least v2.0 to extract

$ unzip -q -t one-piece-891.cbr
No errors detected in compressed data of one-piece-891.cbr.

$ rar t one-piece-891.cbr

RAR 5.30 beta 2   Copyright (c) 1993-2015 Alexander Roshal   4 Aug 2015
Trial version             Type RAR -? for help

one-piece-891.cbr is not RAR archive```

Last Issue option

Currently, the tool has an -all option. It would be really great if there was a -last option which would simply download the last issue. And if the tool was intelligent enough to test for the existence of the file, you could easily add this to a cron job to check for the latest issue and auto download (if you defined everything in a shell script) as things become available.

comicextra fails -all if there are some missing issues in a series

Describe the bug
-all flag crashes the program if there is a missing issue inbetween.

To Reproduce

./comic-dl -format cbz -all -url https://www.comicextra.com/comic/abc

Once it reaches a chapter page with no image files, the program completely exits with this error:

INFO[0000] Downloading...
[0s:0s]panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x6774f9]
goroutine 1 [running]:
image/jpeg.Encode(0x8e1040, 0xc0000a4050, 0x0, 0x0, 0x0, 0x0, 0x8e0b20)
        /usr/local/go/src/image/jpeg/writer.go:576 +0x29
github.com/Girbons/comics-downloader/pkg/util.SaveImage(0x8e1040, 0xc0000a4050, 0x7fe8f7db6130, 0xc00074c9c0, 0x8450ef, 0x3, 0x0, 0xb)
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/util/util.go:171 +0xcb
github.com/Girbons/comics-downloader/pkg/core.(*Comic).DownloadImages(0xc000142120, 0x7ffe672d1bed, 0x1, 0x0, 0x0, 0x0, 0x0)
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/core/core.go:262 +0x685
github.com/Girbons/comics-downloader/pkg/core.(*Comic).makeCBRZ(0xc000142120, 0x7ffe672d1bed, 0x1, 0x0, 0x0)        /Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/core/core.go:170 +0xab
github.com/Girbons/comics-downloader/pkg/core.(*Comic).MakeComic(0xc000142120, 0x7ffe672d1bed, 0x1, 0x100, 0x0)
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/pkg/core/core.go:289 +0x65
github.com/Girbons/comics-downloader/cmd/app.download(0xc0001a5f08, 0x0)
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/cmd/app/downloader.go:90 +0x65f
github.com/Girbons/comics-downloader/cmd/app.Run(0xc000049f08)
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/cmd/app/downloader.go:125 +0x89
main.main()
        /Users/girbons/go/src/github.com/Girbons/comics-downloader/cmd/downloader/main.go:83 +0x245

Expected behavior
The program would skip that chapter and move on to the next one until the list is exhausted. The program starts comicsextra at the latest chapter when -all is used, so it would finally stop after getting past #1.

Desktop (please complete the following information):

  • OS: Linux Ubuntu 5.3.0-46-generic x86_64
  • Version: v0.22.3
  • Type: CLI

Additional context
Attempts to work around this using bash

  1. The .cbz only contains the (first) cover image even if the issue is complete:
for xx in {1..240}; do ./comic-dl -format cbz -url https://www.comicextra.com/abc/chapter-$xx; done
  1. Produces a .pdf only with the (first) cover image.
for xx in {1..243}; do ./comic-dl -images-format jpg -url https://www.comicextra.com/abc/chapter-$xx; done
  1. Will only download the (first) cover image to issue dirs
for xx in {1..240}; do ./comic-dl -images-only -images-format jpg  -url https://www.comicextra.com/abc/chapter-$xx; done

[Suggestion] - Output to CBR, or CBZ instead of PDF

I have a suggestion since the tool is just downloading the jpg files anyways, you should allow the tool to output to a CBR or CBZ file. This is a standard format used by comic readers. These formats are simply a zip file (CBZ) renamed with the cbz file extension, or a rar file (CBR) renamed with the cbr file extension.

This way people can consume the files within their favourite comic reader.

In theory, could also support the epub format for those who read them on an e-ink reader (in black and white).

Mangatown download ERROR 0024

Screenshot 2020-04-08 at 03 33 36

There seems to be an error (somewhat like 0024) and says GET from a particular URL has unsupported protocol scheme.

I've attached a screenshot of the same.

It says "downloading...", but after a few seconds, I get the error message.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.