StellarStoic/LuckySats-lottery-statistics-API

LuckySats lottery statistic scraper and API

★ 0Forks 0PythonGitHub ↗Compare

README

Lottery Results Scraper & API

A system for non-intrusive scraping lottery results from playtdt.com and serving them through a RESTful API on lottery-api.satoshi.si.

Overview

This project consists of two main components:

  1. Lottery Scraper (lotteryresultsscraper.py) [2] - Collects lottery data from playtdt.com
  2. Lottery API (lotteryAPI.py) [1] - Serves the scraped data through a FastAPI interface

Features

Scraper Features [2]:

  • Bitcoin Block Synchronization: Waits for 100th Bitcoin blocks before scraping the lottery data
  • User Agent Rotation: Uses 30+ different user agents to blend in
  • Dual Scraping: Collects data from both main page and stats page
  • Retry Mechanism: Checks for new results 21 times every 2min if not immediately available
  • Data Validation: Validates scraped data before saving
  • Duplicate Prevention: Prevents adding duplicate rounds to the database

API Features [1]:

  • RESTful Endpoints: /api/latest, /api/all, /api/stats, /api/round/{number}, /api/search
  • Search Functionality: Search winners by address or tier
  • Statistics: Comprehensive stats including averages per round
  • CORS Support: Configured for cross-origin requests
  • JSON Responses: Clean, structured JSON output

Installation

Prerequisites

  • Python 3.9+
  • Virtual environment recommended

Setup

# Clone or create project directory
mkdir lotteryAPI
cd lotteryAPI

# Create virtual environment
python3 -m venv lottery-env
source lottery-env/bin/activate

# Install dependencies
pip install requests beautifulsoup4 fastapi uvicorn

Configuration

Scraper Configuration [2]

The scraper can run in two modes:

  • Production Mode: Waits for Bitcoin 100th blocks (default)
  • Testing Mode: Skips block waiting for development
# Production mode (waits for Bitcoin blocks)
python lotteryresultsscraper.py

# Testing mode (no block waiting)
python lotteryresultsscraper.py false
# or
python lotteryresultsscraper.py test

API Configuration [1]

The API runs on port 8000 by default and serves data from draws_data.json and overview_stats.json.

Usage

Running the Scraper [2]

# Basic usage
python lotteryresultsscraper.py

# With testing flag skipping the mempool check
python lotteryresultsscraper.py false

Running the API [1]

# Development
python lotteryAPI.py

# Production (with systemd)
sudo systemctl start lottery-api

API Endpoints [1]

Endpoint Method Description
/ GET API information and available endpoints
/api/latest GET Get the last 21 lottery rounds
/api/all GET Get all rounds (with optional pagination)
/api/round/{number} GET Get specific round by number
/api/stats GET Get statistics about lottery data
/api/search GET Search winners by address or tier

Example API Responses

Latest Round:

{
  "round": "Round 9268",
  "round_number": 9268,
  "block": "Block 926800",
  "draw_number": "12345678",
  "winning_numbers": 8,
  "total_payout": "2049 sats paid",
  "prize_pool": "59665 sats",
  "hash": "000...",
  "winners": [...],
  "scraped_at": "2025-12-07T14:12:00.669130"
}

Statistics:

{
  "total_rounds": "170",
  "total_bets": "1944",
  "total_winners": "998",
  "total_paid_sats": "886692",
  "current_round": "9271",
  "current_prize_pool": "85026",
  "average_payout_last_21_rounds": 4663.41,
  "rounds_used_for_average": 17,
  "source": "playtdt.com/stats",
  "last_updated": "2025-12-08T21:56:45.382791"
}

Deployment

Systemd Services

API Service (/etc/systemd/system/lottery-api.service):

[Unit]
Description=Lottery API Service
After=network.target

[Service]
Type=simple
User=root
WorkingDirectory=/home/lotteryAPI
ExecStart=/bin/bash -c 'cd /home/lotteryAPI && source lottery-env/bin/activate && python lotteryAPI.py'
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

Scraper Service (optional - runs once on boot):

[Unit]
Description=Lottery Results Scraper
After=network.target lottery-api.service

[Service]
Type=simple
User=root
WorkingDirectory=/home/lotteryAPI
ExecStart=/bin/bash -c 'cd /home/lotteryAPI && source lottery-env/bin/activate && python lotteryresultsscraper.py'
Restart=no

[Install]
WantedBy=multi-user.target

Nginx Configuration

server {
    listen 80;
    server_name lottery-api.mydomain.com;
    
    location / {
        proxy_pass http://127.0.0.1:8000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

Data Structure

The scraper saves data in JSON format with the following structure [2]:

{
  "round": "Round 9261",
  "round_number": 9261,
  "block": "Block 926100",
  "draw_number": "55773559",
  "winning_numbers": 27,
  "total_payout": "6989 sats paid",
  "prize_pool": "81718 sats",
  "hash": "00000000000000000000014f8357c81bdf13b781062222ef948e053c0e378405a",
  "winners": [
    {
      "tier": "3-Match 54690405",
      "address": "di***@bitrefill.me",
      "amount": "683 sats",
      "winning_digits": "5469[0405]",
      "full_winning_number": "54690405"
    }
  ],
  "scraped_at": "2025-12-07T14:12:00.669130"
}

How It Works

  1. Block Monitoring: Scraper checks current Bitcoin block height via multiple APIs [2]
  2. Waiting: Waits for next 100th block, then waits for lottery results
  3. Scraping: Collects data from playtdt.com main page and stats page [2]
  4. Processing: Merges data, validates, and saves to JSON
  5. Serving: API reads JSON file and serves through REST endpoints [1]

if making changes in system files

Reload systemd

sudo systemctl daemon-reload

Start the service

sudo systemctl start lottery-scraper or sudo systemctl start lottery-api

Check status

sudo systemctl status lottery-scraper or sudo systemctl status lottery-api

Enable to start on boot

sudo systemctl enable lottery-scraper or sudo systemctl enable lottery-api

View recent logs

sudo journalctl -u lottery-scraper -n 50 or sudo journalctl -u lottery-api -n 50

Follow logs in real-time

sudo journalctl -u lottery-scraper -f or sudo journalctl -u lottery-api -f

Testing

# Test scraper
python lotteryresultsscraper.py test

# Test API locally
curl http://127.0.0.1:8000/api/latest

# Test API through Nginx
curl https://lottery-api.yourdomain.com/api/stats

Troubleshooting

Logs

# Scraper logs (console output)
python lotteryresultsscraper.py false

# API logs
sudo journalctl -u lottery-api -f

Systemd Service Monitoring

Check service status

sudo systemctl status lottery-api

View recent logs

sudo journalctl -u lottery-api -n 50

Follow logs in real-time

sudo journalctl -u lottery-api -f

License

This project is for educational and personal use. Ensure compliance with playtdt.com's terms of service when scraping.

Contributing

  1. Fork the repository
  2. Create a feature branch
  3. Make changes with proper testing
  4. Submit a pull request

Support

For issues or questions:

  1. Check the troubleshooting section
  2. Review the code comments [1][2]
  3. Create an issue with detailed error logs

Contributors

StellarStoic

Issues