creativejoe007 / google-leads-scraper

A simple tool that makes it easy for marketers to extract emails and phone numbers (leads) from google search results.

Google-Leads-Scraper

A simple web scraper that makes it easy for marketers to extract emails and phone numbers (leads) from google search results.

Process

Quite a simple process, you simply enter your search query, file-name to store the data, start and stop to denote number of pages to scrape. The system will load up your query headlessly, meaning you wouldn't even see it while it does its scrapping. This scraper will scrape phone numbers and email addresses including a full screenshot of the webpages and navigate google search pages until it gets to your inputed stop page.

Get Started:

To start you would need to activate the virtual environment

For linux/Mac:

   source env/bin/activate

For Windows (not supported):

.\env\Scripts\activate

1. python3 --version (supports 3.7 and above)
2. pip3 install -r requirements.txt
3. python3 main.py "my query" --start=1 --stop=5 --file="a_file_name_to_save_data"

Note:

A screenshot of websites the scraper visits is taken and stored in the STATIC folder. Check the EXTRACTED folder to view content extracted by this tool (usually in csv format)

Please share and support this libary with your code contributions.

google-leads-scraper's Issues

Selenium error: Message: no such element

I get this error while I try to pull data:
Selenium error: Message: no such element: Unable to locate element: {"method":"css selector","selector":"div.tF2Cxc"}
(Session info: headless chrome=103.0.5060.114)
The error repeats exactly the number of pages I input to be scraped.
Do you have any idea what could cause such error? Also, is there any fix for it?

program closes

whenever i try to run the program closes itself and opens the code in vs code

I can't get it to run

Tried installing chrome browser as well, still getting error.

I'm on windows machine

Unable to locate element

Hello !

I've a Selenium Error during the scraping process :
`Selenium error: Message: no such element: Unable to locate element: {"method":"css selector","selector":"div.tF2Cxc"}
(Session info: headless chrome=91.0.4472.114)

Selenium error: Message: no such element: Unable to locate element: {"method":"css selector","selector":"span.aCOpRe"}
(Session info: headless chrome=91.0.4472.114)

Selenium error: Message: no such element: Unable to locate element: {"method":"css selector","selector":"span.aCOpRe"}
(Session info: headless chrome=91.0.4472.114)

...
`

Maybe a Google CSS skeleton modification.

ModuleNotFoundError: No module named 'webdriver_manager.utils'

Webdriver manager is already satisfied (Requirement already satisfied: webdriver_manager in /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages (from -r requirements.txt (line 2)) (3.7.0))

however I am receiving the error

from webdriver_manager.utils import ChromeType
ModuleNotFoundError: No module named 'webdriver_manager.utils'

OS Error: Failed to parse: https://chromedriver.storage.googleapis.com/LATEST_RELEASE_89.0.4389

Fails as below:

python3 main.py "test" --start=1 --stop=5 --file="emails.csv"

[WDM] - ====== WebDriver manager ======
[WDM] - Current google-chrome version is 89.0.4389
[WDM] - Get LATEST driver version for 89.0.4389

 OS Error: Failed to parse: https://chromedriver.storage.googleapis.com/LATEST_RELEASE_89.0.4389

Oops, something's off here

Anyone else getting the error " Oops, something's off here" when running the script? im getting this error regardless of the query or the amount of pages i set to scrape