Name: [Unofficial] UK Government Mirror
Type: Organization
Bio: An unofficial mirror of every UK Government Github repository, to preserve repositories that are removed or made private. Updated at least weekly.
Location: UK
Blog: https://government.github.com/community/#governments-uk-central
[Unofficial] UK Government Mirror's Projects
WARC and ARC indexing and discovery tools.
Tools for exploring the contents of web archive files.
Use FUSE-J to mount web archive files as filesystems.
Utilities for working with WARC files stored on HDFS.
A set of test files for web archiving.
WAT (web archive transform) metadata mining
A collection of Jupyter notebooks for working with web archive data, tools and APIs
A RESTful API for rendering web pages
Daemon intended to monitor a queue to which Heritrix will submit URLs. On receipt, the URL is submitted to a webservice (currently via django-phantomjs) and stores the response, a modified HAR record, in a WARC file.
A RESTful API for rendering web pages in PhantomJS
Web page rendering service based on Google's Puppeteer
Experiments in testable, scaleable crawler architectures