Unicode error (ghdb-scraper) about pagodo

[opsdisk]

Thanks for reporting this @bimarifin . What exact version of Python are you using? Looks like 3.7 from your output. I just tried with Python 3.6.7 and 3.7.5 and didn't have any issues, but I'll take a look at adding some defensive measures.

[bimarifin]

btw i also making changes with your code, i think people like me doesnt want to get all dorks from ghdb, just spesific category, so i making changes and looks like that.

but i dont know how to changes / commit to your repo, sorry newbie with github :(

[bimarifin]

but you can use this :

_categories = {'1': 'Footholds', '2': 'File_Containing_Usernames', '3': 'Sensitives_Directories', '4': 'Web_Server_Detection', '5': 'Vulnerable_Files', '6': 'Vulnerable_Servers', '7': 'Error_Messages', '8': 'File_Containing_Juicy_Info', '9': 'File_Containing_Passwords', '10': 'Sensitive_Online_Shopping_Info', '11': 'Network_or_Vulnerability_Data', '12': 'Pages_Containing_Login_Portals', '13': 'Various_Online_devices', '14': 'Advisories_and_Vulnerabilities'}

and just add for loop into retrieve google_dorks

for key, value in _categories.items():
        
        url = "https://www.exploit-db.com/google-hacking-database?category={}".format(key)

[opsdisk]

Great idea! That capability would be awesome. Check out this branch and let me know what you think:

git clone https://github.com/opsdisk/pagodo.git
git branch -a
git checkout issue-37-ghdb_scraper-unicode-error

Pull request is here: #38

[opsdisk]

Merged into master

• Added encoding="utf-8" when writing files to disk
• Added -i switch to separate dorks into different individual categories