GitHubHelp

ABoBo555/Sample

★ 0Forks 0HTMLGitHub ↗Compare

README

Lexus Betting Site Scraper

run first command which launches edge with remote debugging port

then go to the site and login manually and go to Bets History page and do all filtering and finally run the python script

Start-Process msedge -ArgumentList "--remote-debugging-port=9222","--user-data-dir=C:\edge_selenium_profile"

python lexus_scraper.py

Automated web scraper for extracting table data from https://lexus100.com/

Features

✅ Connects to your existing Edge browser (bypasses Cloudflare!)
✅ Handles manual login (you control captcha, credentials, T&C)
✅ Navigates to Bets History page
✅ Sets page filters and clicks "View By Page"
✅ Clicks each "View" button to expand details
✅ Scrapes all data (summary + expanded details)
✅ Exports everything to Excel with timestamp

Installation

Step 1: Install Python packages

pip install -r requirements.txt

Step 2: Install Edge WebDriver

The script uses Selenium with Microsoft Edge. Make sure you have Edge browser installed (it comes with Windows).

Usage

Method 1: Using the Helper Script (Recommended)

# Step 1: Start Edge with remote debugging
.\start_edge.ps1

# Step 2: In Edge, complete login manually:
#   - Navigate to https://lexus100.com/
#   - Complete Cloudflare verification
#   - Log in with credentials
#   - Accept Terms & Conditions
#   - Wait at portal page

# Step 3: Run the scraper
python lexus_scraper.py

Method 2: Manual Edge Start

# Step 1: Start Edge with remote debugging manually
Start-Process msedge -ArgumentList "--remote-debugging-port=9222"

# Step 2: Complete login in Edge (see above)

# Step 3: Run the scraper
python lexus_scraper.py

How It Works:

  1. Edge with Remote Debugging - Edge starts with debugging port 9222
  2. Manual Login - You handle Cloudflare, login, and navigation
  3. Script Connects - Python script connects to your running Edge
  4. Automated Scraping - Script takes over:
    • Navigates to Bets History page
    • Sets page filters (1-9999) or you can set manually
    • Clicks all "View" buttons
    • Extracts all data
  5. Excel Export - Data saved as lexus_data_YYYYMMDD_HHMMSS.xlsx

Output

The Excel file contains these columns:

  • Summary data: Member, Sys Time, Total Stakes, Total Fight, Package, Page No
  • Detail data: Draw Date, Draw Type, Bet Number, Category, Big, Small, A, ABC, 4A-4F, 3B-3E, 2A-2C, 5D, 6D, Total Stakes, Prize, Agent Loss
  • Additional: Record ID, Notes

Troubleshooting

If you get "selenium not found":

pip install selenium pandas openpyxl

If Edge doesn't launch:

  • Make sure Microsoft Edge is installed
  • Update Edge to the latest version

If the script can't find elements:

  • The website structure may have changed
  • Try setting filters manually when prompted

Tips

  • Don't close the browser during manual login steps
  • Wait for pages to fully load before pressing ENTER
  • Check the Excel file after completion to verify data
  • Browser stays open at the end for you to inspect results

Support

If something doesn't work:

  1. Check that you're at the correct page before pressing ENTER
  2. Make sure table data is visible before scraping starts
  3. Try running the script again - sometimes timing issues occur

Contributors

ABoBo555

Issues