firecrawl/firecrawl
Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
Topic: web-data-extraction
63 repositories
ScrapeGraphAI/Scrapegraph-ai
Python scraper based on AI
browser-act/skills
Browser automation CLI built for AI agents. Break through anti-bot walls, hand off to humans across platforms when stuck. Parallel multi-task execution, independent multi-session operation, isolated multi-account browsing.
vakra-dev/reader
Open source web infrastructure for AI. Scrape, crawl, and automate the web, clean markdown, browser sessions, ready for your agents.
lightfeed/extractor
Use LLMs to robustly extract web data
umbrellaDocumentation/Web-Data-Scraper
Web Data Scraper - no-code internet scraping. Extract and export to CSV, Excel, JSON, Google Sheets, and Webhook.
diffbot/diffbot-python
Python client library for Diffbot APIs
crawlcore/qcrawl
qcrawl - fast async web crawling & scraping framework for Python.
MohamedHmini/iww
AI based web-wrapper for web-content-extraction
neurons-me/this.url
The this.url class is designed to fetch and parse URL data, returning an object with structured information that can then be used for machine learning algorithms in a database or other storage.
luminati-io/java-web-scraping
Quick guide with code example how to use Java for web scraping
didvc/jseval
Evaluate JavaScript on a URL through headless Chrome browser.
wyattowalsh/proxywhirl
rotating proxy system
ZenRows/cli
Zero-config CLI that gives AI agents reliable access to protected web data through Zenrows — Fetch, Extract, Batch, and Browser Sessions, plus MCP setup and agent skills. No API key needed to start.
lightfeed/scrapedown
HTML to Markdown with CSS selector and XPath annotations
dstark5/gnews-scraper
GNewsScraper is a TypeScript package that scrapes article data from Google News based on a keyword or phrase. It returns the results as an array of JSON objects, making it convenient to access and use the scraped information
DemonMartin/scrappey-wrapper
An API wrapper for Scrappey.com written in Node.js (cloudflare bypass & solver)
SaurabhSSB/BookMiner
A pipeline to scrape, extract, and analyze book data from web pages to insights.
jjonescz/awe
AI-based web extractor
wbsg-uni-mannheim/WDCFramework
Java Framework which is used by the Web Data Commons project to extract Microdata, Microformats and RDFa data, Web graphs, and HTML tables from the web crawls provided by the Common Crawl Foundation.
spidra-io/spidra-js
The official Node.js SDK for Spidra.
vakra-dev/supermarkdown
High-performance HTML to Markdown converter with full GitHub Flavored Markdown support. Written in Rust, available for Node.js and as a native Rust crate.
webscrape-ai/webscrape-ai
The web scraping API that turns any URL into clean data.
Boomslet/Web_Crawler
Open-source web crawler
lekhmanrus/real-shot-pdf
RealShotPDF is a Chrome extension designed to simplify the process of creating PDF documents from web content. The extension allows users to navigate through selected webpages, parse and display links in a tree view, and generate PDFs for the chosen pages. It operates locally without sending any data to external servers.
kaizenplatform/FacebookInsightsConnector
The Tableau Web Data Connector for Facebook Insights API
oxpath/oxpath
OXPath from Oxford
lightfeed/sdk
Lightfeed SDK to search and filter web data
jacoobwang/beecrawl
The Web Data Infrastructure for AI
wbsg-uni-mannheim/schemaorg-tables
This repository contains the code and data download links to reproduce the building process of the 2021 Schema.org Table Corpus.