mmg1 / webanalyze

Port of Wappalyzer (uncovers technologies used on websites) in Go to automate scanning.

webanalyze

This is a port of Wappalyzer in Go. This tool is designed to be performant and allows to test huge lists of hosts.

Installation and usage

Precompiled releases can be downloaded directly here.

If you want to build for yourself:

$ go get -u github.com/rverton/webanalyze/...
$ webanalyze -update # loads new apps.json file from wappalyzer project
$ webanalyze -h
Usage of webanalyze:
  -apps string
        app definition file. (default "apps.json")
  -crawl int
        links to follow from the root page (default 0)
  -host string
        single host to test
  -hosts string
        filename with hosts, one host per line.
  -output string
        output format (stdout|csv|json) (default "stdout")
  -search
        searches all urls with same base domain (i.e. example.com and sub.example.com) (default true)
  -silent
	    avoid printing header (default false)
  -update
        update apps file
  -worker int
        number of worker (default 4)

The -update flags downloads a current version of apps.json from the wappalyzer repository to the current folder.

Development / Usage as a lib

See cmd/webanalyze/main.go for an example on how to use this as a library.

Example

$ ./webanalyze -host robinverton.de -crawl 1
 :: webanalyze        : v1.0
 :: workers           : 4
 :: apps              : apps.json
 :: crawl count       : 1
 :: search subdomains : true

https://robinverton.de/hire/ (0.5s):
    Highlight.js,  (Miscellaneous)
    Netlify,  (Web Servers, CDN)
    Google Font API,  (Font Scripts)
http://robinverton.de (0.8s):
    Highlight.js,  (Miscellaneous)
    Netlify,  (Web Servers, CDN)
    Hugo, 0.42.1 (Static Site Generator)
    Google Font API,  (Font Scripts)

$ ./webanalyze -host robinverton.de -crawl 1 -output csv
 :: webanalyze        : v1.0
 :: workers           : 4
 :: apps              : apps.json
 :: crawl count       : 1
 :: search subdomains : true

Host,Category,App,Version
https://robinverton.de/hire/,Miscellaneous,Highlight.js,
https://robinverton.de/hire/,Font Scripts,Google Font API,
https://robinverton.de/hire/,"Web Servers,CDN",Netlify,
http://robinverton.de,"Web Servers,CDN",Netlify,
http://robinverton.de,Static Site Generator,Hugo,0.42.1
http://robinverton.de,Miscellaneous,Highlight.js,
http://robinverton.de,Font Scripts,Google Font API,