DarkSpider is a multithreaded crawler and extractor for regular or onion webpages through the TOR network, written in Python.
See the Docs for full documentation, examples, and other information.
For bugs, questions and discussions please use the GitHub Issues.
- @MikeMeliz for the initial
TorCrawlcodebase. - Logo: Spider Icon by Ayu
- A. Alharbi et al., "Exploring the Topological Properties of the Tor Dark Web", IEEE Access, vol. 9, pp. 21746-21758, 2021.
“GPL” stands for “General Public License”. Using the GNU GPL will require that all the released improved versions be free software. source & more
v2.0.1:
- Add: Project IEEE citation by @PROxZIMA in PROxZIMA#8
- Fix: Ignore Gooey installation by default by @PROxZIMA in PROxZIMA#9
- Add: CLI command to include/exclude external links by @PROxZIMA in PROxZIMA#11
- Fixed Issue related to Graphical Analysis by @knightster0804 in PROxZIMA#10
- Revert "Fixed Issue related to Graphical Analysis" by @PROxZIMA in PROxZIMA#12
- Creating json by @r0nl in PROxZIMA#14
- Added 'exclusion' Argument by @r0nl in PROxZIMA#16
- Added required functionalities and Images by @knightster0804 in PROxZIMA#13
- White box unit test cases for modules by @PROxZIMA in PROxZIMA#7
- New graphical visualization using
seabornlibrary by @PROxZIMA in PROxZIMA#17 - Added image and script crawler by @ytatiya3 in PROxZIMA#15
- Crawler multi-threaded implementation by @PROxZIMA in PROxZIMA#18
- @PROxZIMA made their first contribution in PROxZIMA#8
- @knightster0804 made their first contribution in PROxZIMA#10
- @r0nl made their first contribution in PROxZIMA#14
- @ytatiya3 made their first contribution in PROxZIMA#15
Full Changelog: https://github.com/PROxZIMA/DarkSpider/compare/1.0.0...2.0.1
v1.0.0:
- Initial project setup
Full Changelog: https://github.com/PROxZIMA/DarkSpider/commits/1.0.0
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent