motherboardgithub / mass_archive Goto Github PK

Hi thanks for writing this it's amazing!

• Convert your script to a repo for pip / snap for one command install with dependencies gotten and updated in one shot super easy for people.
• Make it a library so other programs can call it without hacky stuff
• Make perma.cc support a flag so we don't have to manually edit the file with a API key.

Is it possible to add a option to archive a whole site or part of a site?

For each starting page you have to get the links within the site or a list of to follow sites and push each into the archive service.

If one is mass archiving sites of oneself, it can be good to include an http://archiveready.com/ report per URL. From those one could collect suggestions how to improve archiveability.

Also see http://archiveready.com/docs/api.html

Support the nesting of Hypothesis links as an archive link.
e.g. http://archive.today/?run=1&url=https://via.hypothes.is/https://www.agileservicemanifesto.org/
where http://archive.today/?run=1&url=https://via.hypothes.is/ is the prefix and https://www.agileservicemanifesto.org/ is the URL (encodeURIComponent in JS function)
Same should be applied to WayBack Machine and others

For those who blocks certain URLs

• archive.today
• archive.li
• archive.vn
• archive.ph
• archive.md

https://github.com/oduwsdl/archivenow#example-4 have a big list, and is written in Python, is it possible to integrate some of the code?

• https://megalodon.jp/
• https://archive.st/
• (Perma and WebCitation will not accept any more new pages)

How about adding support for WebCite http://www.webcitation.org/ ? It requires an email though.

http://www.webcitation.org/archive?url=yoururl&email=yourmail