macbre / 3pc

3rd party web content database

Hi there 👋

My GitHub Statistics

Read more

• My articles on Medium

3pc's Issues

Import CDN host names and headers from webpagetest

https://github.com/WPO-Foundation/webpagetest/blob/master/agent/wpthook/cdn.h#L46

macbre/phantomas#210

Import 3pc domains from HTTP Archive dumps

http://httparchive.org/downloads.php

See macbre/phantomas#210

Use the list from someonewhocares.org/hosts/

$ curl http://someonewhocares.org/hosts/hosts

Get # ads section

Provide nodejs API and publish to npm

• Provide nodejs API

var thirdParty = require('3pc');

console.log(thirdParty.cdn.matchByUrl('http://vignette3.wikia.nocookie.net/nordycka/images/e/ee/Tj%C3%B8rnuv%C3%ADk.jpg/revision/latest/scale-to-width-down/640?cb=20150904165805&path-prefix=pl'));

console.log(thirdParty.trackers.matchByUrl('http://www.google-analytics.com/analytics.js'));

• Make 3pc JSON "database" available via npm

Create a list CDNs

https://github.com/WPO-Foundation/webpagetest/blob/master/agent/wpthook/cdn.h

Generate list of ads hosts from AdBlock rules

https://easylist.adblockplus.org/en/development

Import 3pc tracking domains from Ghostery JSON file

https://github.com/jonpierce/ghostery/blob/master/firefox/ghostery-statusbar/ghostery/chrome/content/ghostery-bugs.js

Read http://calendar.perfplanet.com/2013/thirdpartycontent/ for more data

See macbre/phantomas#210

Extend trackers DB

Extend trackers DB with the following:

• http://cdn.krxd.net/controltag?confid=...
• http://callingjustified.com/bcn?vqwo=1&deo=1&fe=*
• http://flappybadger.net/sp?cb=*