ahmadibrahiim / website-downloader

💡 Download the complete source code of any website (including all assets). [ Javascripts, Stylesheets, Images ] using Node.js

Complete Website Downloader 💾

Download the complete source code of any website (including all assets) 🔨.

👉 Live Demo: https://website-downloader.onrender.com

Description 📒

Website downloader works with wget and archiver to download all websites assets and compress then sends it back to the user through socket channel

wget params the being used

wget --mirror --convert-links --adjust-extension --page-requisites --no-parent http://example.org

Explanation of the various flags:

• --mirror – Makes (among other things) the download recursive.
• --convert-links – convert all the links (also to stuff like CSS stylesheets) to relative, so it will be suitable for offline viewing.
• --adjust-extension – Adds suitable extensions to filenames (html or css) depending on their content-type.
• --page-requisites – Download things like CSS style-sheets and images required to properly display the page offline.
• --no-parent – When recursing do not ascend to the parent directory. It useful for restricting the download to only a portion of the site

Deploy on cloud providers

How to run it 🤔

• git clone https://github.com/AhmadIbrahiim/Website-downloader.git
• cd website-downloader
• $ npm install
• $ npm start
• http://localhost:3000/

How To Contribute:

• Open Issue(s) with any bugs you notice.
• Please create Pull Requests if you think it would be an added value towards our program.

Liked it ? You can buy a coffee:

Thank you,

Email: [email protected]

https://www.ahmed-ibrahim.com

website-downloader's Issues

Website need to upgrade!

I just changed the CSS code.

Collapsible Menu not working

Firstly, thank you for this tool!
I use it on a regular basis to archive anything from good blogs to guides I come across.
However, I recently stumbled upon a guide that I could not download properly, as the collapsible menus do not work in the downloaded version of the website.

Looking at the source, the text seems to be there. Opening the items just doesn't work.
Since I have no experience at all in web development, I sadly cannot give you more details on the problem.

If you need anything, please let me know.

Website

Doesn't work on some websites, like chat.openai.com, for example

It gives me an empty zip file.
Seems like even the web version of OpenAI's models is Closed.

Can not use in local?

Hello,guys,
i clone the repo and run npm install to install packages success.

then i run npm start, all look perfect,but when i use chrome to vist 127.0.0.1:3000 i always get Total Downloaded file: 0.

After that,the page looks block on comperssing and return nothing

This code is not working

To now I downloaded this script, open in the browser, test for download google.com ... but script is no working.

I have wget and...other packets in my VPS.

So how to understand the problem? No see errors... ?

I try with this page of LOL

Web is down (again)

Website is down again, kindly fix chief

Error

When trying to download any website, you receive this error:
NotFoundError: Not Found at /workspace/Website-downloader/app.js:27:8 at Layer.handle [as handle_request] (/workspace/Website-downloader/node_modules/express/lib/router/layer.js:95:5) at trim_prefix (/workspace/Website-downloader/node_modules/express/lib/router/index.js:317:13) at /workspace/Website-downloader/node_modules/express/lib/router/index.js:284:7 at Function.process_params (/workspace/Website-downloader/node_modules/express/lib/router/index.js:335:12) at next (/workspace/Website-downloader/node_modules/express/lib/router/index.js:275:10) at /workspace/Website-downloader/node_modules/express/lib/router/index.js:635:15 at next (/workspace/Website-downloader/node_modules/express/lib/router/index.js:260:14) at Function.handle (/workspace/Website-downloader/node_modules/express/lib/router/index.js:174:3) at router (/workspace/Website-downloader/node_modules/express/lib/router/index.js:47:12)

Not working anymore

trash

Add api please

please add api for fetch using axios then response as json

It can download websites with login?

I need to download sites witch use login do you support that.

website is down

https://site-dwloader.herokuapp.com/ is giving me an error

Stuck at Last

It stuck their:
`> [email protected] start C:\Users\Fasih4444\Website-downloader

node ./bin/www`

404 Not Found on form submission

Public Server:

Local Server:

Would return an empty directory without contents

I tried to download contents of this specific website🔗
although it returns nothing but a blank directory

compress is getting a lot of time and it not working

website doesnt work

assets

Clicking "Enter" pulls a 404 page

If I click "Enter" inside the text field it goes to a 404 Not Found page. Here's the error that is printing:

NotFoundError: Not Found
    at /var/www/html/dloader/app.js:27:8
    at Layer.handle [as handle_request] (/var/www/html/dloader/node_modules/express/lib/router/layer.js:95:5)
    at trim_prefix (/var/www/html/dloader/node_modules/express/lib/router/index.js:317:13)
    at /var/www/html/dloader/node_modules/express/lib/router/index.js:284:7
    at Function.process_params (/var/www/html/dloader/node_modules/express/lib/router/index.js:335:12)
    at next (/var/www/html/dloader/node_modules/express/lib/router/index.js:275:10)
    at /var/www/html/dloader/node_modules/express/lib/router/index.js:635:15
    at next (/var/www/html/dloader/node_modules/express/lib/router/index.js:260:14)
    at Function.handle (/var/www/html/dloader/node_modules/express/lib/router/index.js:174:3)
    at router (/var/www/html/dloader/node_modules/express/lib/router/index.js:47:12)

app does not launch

https://prnt.sc/ZA6hnaT0hi54

I try with a page of LCU league of legends

I'm trying to use the tool with the league of legends LCU Homepage where all the assets of the league of legends events are stored, but...

It's been like this for a long time.

It might not work, but I'd love to extract the endpoints Riot is using on this page and all the assets.

Thank you!

Demo down

https://site-dwloader.herokuapp.com/

Important

The website is down / not working

How to move the Star github button or find it in code

How could I do this!

I love it

I love it! The single tool that does it's job and it's simple to use. Please continue to maintain it.
I wish you all the best!

Demo site can't download media in CSS sheet

Demo site can't download media in CSS sheet

the web is down

as tiltle said, the live demo is down

The website is down again

Website not working

When you try to use https://yell0wsuit.page/html5-games/games/angry-birds-chrome/ it does not work.

Link to zip file: yell0wsuit.page.zip

code php

Parameters to change the way we download sites?

Hi Ahmed,

I didn't know how best to explain this so I've added a screenshot in the hope that a picture really is worth a thousand words. :-)

I have an old Mac app called SiteSucker which downloads websites. I'd prefer to move to your Website Downloader tool, but for some reason, I get "different" results from Website Downloader. I'm wondering if this is down to a wget parameter?

• On the left of the screenshot is the result of downloading my WordPress site with SiteSucker. I have a single index.html at the root of the download and all is well.
• On the right of the screenshot is the result of downloading my WordPress site with Website Downloader - the same index.html exists but in addition a number of other index.html.... files all suffixed with parameters. Eg; index.html?p=26446.html. These files are other "pages" on my site and blog posts.

Having "multiple index.html" files causes issues when deploying my site to Netlify, where it doesn't seem to recognize the solitary index.html as my index and instead picks one of the others.

I could always remove the offending files and manually update various links in my index.html to point to the correct locations, but if there was a way to download in "the same way as SiteSucker does", that would be fantastic.

Does any of that make sense? :-)

Thanks,

Steve

Some websites not working

when it is told to download itself, it says "not found" this also happens with other sites

The demo of this tool gives an error

Hello, thank you very much for developing this tool, I really wanted to use this tool, but the demo tool does not load

Demo is not working

When launching demo to test the code - the following error message is displayed:

Application error
An error occurred in the application and your page could not be served. If you are the application owner, check your logs for details. You can do this from the Heroku CLI with the command
heroku logs --tail

i have tried this one on but not able to run it

Hi Ahmad,
I have tried to use it with but not able to download the please help me