firecrawl/firecrawl
Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
Topic: data-extraction
4,679 repositories
D4Vinci/Scrapling
🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
unclecode/crawl4ai
Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key.
ScrapeGraphAI/Scrapegraph-ai
Python scraper based on AI
browserbase/stagehand
The SDK to extract data and interact with any site on the web. Get started with Claude Code, Codex, Eve, Mastra, and more.
getmaxun/maxun
Turn any website into a structured API. Extract, automate, search and monitor the web.
browser-act/skills
Browser automation CLI built for AI agents. Break through anti-bot walls, hand off to humans across platforms when stuck. Parallel multi-task execution, independent multi-session operation, isolated multi-account browsing.
MontFerret/ferret
Declarative data automation language and Go runtime for structured extraction workflows.
vi3k6i5/flashtext
Extract Keywords from sentence or Replace keywords in sentences.
brightdata/brightdata-mcp
A powerful Model Context Protocol (MCP) server that provides an all-in-one solution for public web access.
saifyxpro/HeadlessX
The undetected self-hosted browser automation platform. Powered by Camoufox (Firefox) for 0% detection rates. Built for speed, privacy, and scalability.
hhursev/recipe-scrapers
Python package for scraping recipes data
shcherbak-ai/contextgem
Extract structured data and insights from documents using LLMs and decision models
extractus/article-extractor
To extract article from given URL
JonathanLink/PDFLayoutTextStripper
Converts a pdf file into a text file while keeping the layout of the original pdf. Useful to extract the content from a table in a pdf file for instance. This is a subclass of PDFTextStripper class (from the Apache PDFBox library).
hi-primus/optimus
:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
thinh-vu/vnstock
A beginner-friendly yet powerful Python toolkit for financial analysis and automation — built to make modern investing accessible to everyone
raznem/parsera
Lightweight library for scraping web-sites with LLMs
fastcrw/crw
Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.
soxoj/socid-extractor
⛏️ The extraction engine behind Maigret: turn any profile URL into a structured OSINT record across 150+ sites
yfedoseev/pdf_oxide
The fastest PDF library for Python and Rust. Text extraction, image extraction, markdown conversion, PDF creation & editing. 0.8ms mean, 5× faster than industry leaders, 100% pass rate on 3,830 PDFs. MIT/Apache-2.0.
ScrapingBee/ai-web-scraper
Effortlessly extract data with our AI scraper API. Simplify data extraction, get clean JSON outputs and adapt to page changes. Try it free today!
eclaire-labs/eclaire
Local-first, open-source AI assistant for your data. Unify tasks, notes, docs, photos, and bookmarks. Private, self-hosted, and extensible via APIs.
polyrabbit/hacker-news-digest
:newspaper: Let LLM Summarize Hacker News for You
ScrapeGraphAI/scrapecraft
🤖 AI-powered web scraping editor with visual workflow builder. Build, test & deploy web scrapers using natural language. Powered by ScrapeGraphAI & LangGraph.
adrienjoly/npm-pdfreader
🚜 Parse text and tables from PDF files.
ScrapingBee/google-reviews-scraper
Google Reviews API integration and scalable Google Reviews Scraper for extracting ratings, review text, and business insights from Google Maps using a structured Google Review API.
mrshu/github-statuses
The "Missing GitHub Status Page" -- a Flat Data attempt at historically documenting GitHub statuses
jpjacobpadilla/Stealth-Requests
Undetected web-scraping & seamless HTML parsing in Python!
vakra-dev/reader
Open source web infrastructure for AI. Scrape, crawl, and automate the web, clean markdown, browser sessions, ready for your agents.