firecrawl/firecrawl
Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
Topic: webscraping
10,377 repositories
D4Vinci/Scrapling
🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
huginn/huginn
Create agents that monitor and act on your behalf. Your agents are standing by!
CloakHQ/CloakBrowser
Stealth Chromium that passes every bot detection test. Drop-in Playwright replacement with source-level fingerprint patches. 50+ tests passed.
ScrapeGraphAI/Scrapegraph-ai
Python scraper based on AI
assafelovic/gpt-researcher
An autonomous agent that conducts deep research on any data using any LLM providers
getmaxun/maxun
Turn any website into a structured API. Extract, automate, search and monitor the web.
pystardust/ani-cli
A cli tool to browse and play anime
daijro/camoufox
🦊 Anti-detect browser
lorien/awesome-web-scraping
List of libraries, tools and APIs for web scraping and data processing.
alirezamika/autoscraper
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
niespodd/browser-fingerprinting
Analysis of Bot Protection systems with available countermeasures 🚿. How to defeat anti-bot system 👻 and get around browser fingerprinting scripts 🕵️♂️ when scraping the web?
jaypyles/Scraperr
Self-hosted webscraper.
Kaliiiiiiiiii-Vinyzu/patchright
Undetected version of the Playwright testing and automation library.
itsOwen/CyberScraper-2077
A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama
feder-cr/invisible_playwright
Playwright that anti-bots cannot see, so no captchas: same API, anti-detect stealth headless Firefox, undetected fingerprint, bypass bot detection, Python web scraping and browser automation.
feder-cr/invisible_selenium
Selenium that anti-bots cannot see, so no captchas: same API, anti-detect stealth headless Firefox, undetected fingerprint, bypass bot detection, Python web scraping and browser automation.
GodsScion/Auto_job_applier_linkedIn
Make your job hunt easy by automating your application process with this Auto Applier
feder-cr/invisible_playwright_mcp
Playwright MCP server undetected by anti-bots and captchas: AI agent browses the web on anti-detect stealth Firefox, Python, undetected browser automation, scraping, computer use.
anaskhan96/soup
Web Scraper in Go, similar to BeautifulSoup
reworkd/tarsier
Vision utilities for web interaction agents 👀
TheWebScrapingClub/webscraping-from-0-to-hero
The web scraping open project repository aims to share knowledge and experiences about web scraping with Python
m8sec/CrossLinked
LinkedIn enumeration tool to extract valid employee names from an organization through search engine scraping
Kaliiiiiiiiii-Vinyzu/patchright-python
Undetected Python version of the Playwright testing and automation library.
zohaibbashir/Google-Maps-Scrapper
This code is used to perform web scraping and data extraction from Google Maps. It is particularly designed for obtaining information about businesses, including their name, address, website, phone number, reviews, and more.
requests-cache/requests-cache
Persistent HTTP cache for python requests
cdpdriver/zendriver
A blazing fast, async-first, undetectable webscraping/web automation framework based on ultrafunkamsterdam/nodriver. Now with Docker support!
jamesturk/scrapeghost
👻 Experimental library for scraping websites using OpenAI's GPT API.
feder-cr/invisible_puppeteer
Puppeteer for Python that anti-bots cannot see, so no captchas: same API, anti-detect stealth headless Firefox, undetected fingerprint, bypass bot detection, web scraping and browser automation.
raznem/parsera
Lightweight library for scraping web-sites with LLMs