#Web-crawler-demo
To Run, Just execute "mvn clean compile exec:java" If you want to change domain url, go to demo.crawler.Main and change there in the constant.
Points to improve:-
- This is a single threaded web crawler. Multi-threaded would give better performance. For bigger sites, this may run for many minutes.
- it assumes https is different from http. this could be fixed