option with --proxy about pagodo

[opsdisk]

Hi @melmel27 appreciate the question. That might work with the current version of pagodo and proxychains. There are some new pagodo updates coming soon that I mentioned in another issue today. Biggest being native proxy support :)

#55 (comment)

[opsdisk]

Just released v2! https://github.com/opsdisk/pagodo/releases/tag/v2.0.0

Going to close this one out.

[melmel27]

Hi, thanks for the very fast update.

[melmel27]

i get a certificate error, looks like adding verify=False will fix this?

python3 pagodo.py -g g.txt -o -s -p http://127.0.0.1:8080
2021-09-02 01:55:53,960 [MainThread ] [INFO] Initiation timestamp: 2021-09-02T01:55:53.960795
2021-09-02 01:55:53,961 [MainThread ] [INFO] Search ( 1 / 1 ) for Google dork [ intext:test ] using User-Agent 'Mozilla/5.0 (Windows; U; Windows NT 6.0; fr-FR) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5' through proxy 'http://127.0.0.1:8080'
2021-09-02 01:55:53,961 [MainThread ] [INFO] Requesting URL: https://www.google.com/
2021-09-02 01:55:53,990 [MainThread ] [ERROR] Error with dork: intext:test
2021-09-02 01:55:53,990 [MainThread ] [ERROR] EXCEPTION: HTTPSConnectionPool(host='www.google.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1056)')))
2021-09-02 01:55:53,991 [MainThread ] [INFO] Total URLs found for the 1 total dorks searched: 0
2021-09-02 01:55:53,992 [MainThread ] [INFO] Completion timestamp: 2021-09-02T01:55:53.992034

[opsdisk]

EDIT: Just noticed it's not an HTTPS proxy. Mind sharing which proxy software you're using? Also, would appreciate you still trying out my request below.

Ahhh, didn't think of self-signed certs for proxies. So I think this will be an issue with the yagooglesearch library, and not pagodo.

To help me out, can you open yagooglesearch/__init__.py (should be something like .venv/lib/python3.7/site-packages/yagooglesearch/__init__.py if you are using Python virtual environments. Go to line 297 (https://github.com/opsdisk/yagooglesearch/blob/master/yagooglesearch/__init__.py#L297) and change it to:

response = requests.get(url, proxies=self.proxy_dict, headers=headers, cookies=self.cookies, timeout=15, verify=False)

then try re-running it and letting me know if that works.

[melmel27]

yes i edited it and it's working fine, about dork results i tried to search subdomain of *.site.com but the result shows only 100 results, can the result be more than this? Thank you in advance

[opsdisk]

Thanks for confirming...I'll add the option to yagooglesearch soon.

To your question, you can specify the number of valid URLs to return using the -m switch...100 is the default. If you want to pull back something like 500, use -m 500. See https://github.com/opsdisk/pagodo#number-of-results-to-return

[opsdisk]

Which HTTPS proxy software were you using @melmel27 ?

[melmel27]

hi thanks for the information. i am using IP-Rotate from aws which rotates the IP address on every request.

[melmel27]

this tool can be found in burpsuite. I have another question, some user-agents directed me to consent.google.com. is it possible that google doesn't accept too old user-agent?

[opsdisk]

@melmel27

I just pushed out version 2.1.0 (#62)

In your case, add the -l switch to disable SSL/TLS verification. Let me know if you run into any other issues.

Regarding the old user agent issue...maybe? pagodo should only make requests to https://www.google.com and not any subdomains like consent.google.com. Perhaps it was from a browser you were utilizing that was using burp as its proxy? Definitely let me know if you're seeing something different though.

[melmel27]

i confirm that the subdomain generated due to burp follows redirects like

HTTP/1.1 302 Found
Date: Mon, 06 Sep 2021 02:21:13 GMT
Content-Type: text/html; charset=UTF-8
Content-Length: 486
Connection: close
x-amzn-RequestId: ec8bc2eb-4442-43c3-b5d5-66000afef4a1
X-XSS-Protection: 0
x-amzn-Remapped-Content-Length: 486
X-Frame-Options: SAMEORIGIN
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
x-amz-apigw-id: FOArgHrarPEFcfg=
x-amzn-Remapped-Server: gws
Cache-Control: private
**Location: https://consent.google.com/ml?continue=https://www.google.com/[google-search-dork**
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
x-amzn-Remapped-Date: Mon, 06 Sep 2021 02:21:13 GMT