scraping-xx Goto Github PK

forms

An easy way to create, parse and validate forms in node.js

formspring_scraper

Scrapes Data for Formspring pages

fuzzer

Fuzzer for the Security Engineering class @ RIT

gae-boilerpipe

Simple implementation of modified Boilerpipe parser on Google App Engine

gatherer

Experimental scrapy client for scraping OpenCourseware related content

gelatin

Transform text files to XML, JSON, or YAML

geodict

A simple Python library/tool for pulling location information from unstructured text

ghost.py

Webkit based scriptable web browser for python.

gighuntr.github.com

A tool for finding jobs or freelance gigs by searching various job boards using a custom Google search.

go-api-backend

Highly specific design, allows http-gets to relay events to a single websocket.

googlesearch

Scrape the Google search result with Scrapy.

googleurl

goose

Html Content / Article Extractor in Scala - open sourced from Gravity Labs - http://gravity.com

grafana

Grafana - A Graphite Dashboard and Graph Editor

grunt-contrib-jasmine

Run jasmine specs headlessly through PhantomJS.

gscraper

A web-scraping interface to various Google Services.

gwt_crawlerrors-php

Download website crawl errors from Google Webmaster Tools as CSV.

haikufinder

Python module to locate haikus in unstructured English text

haskell-pdf-parsing-library

Library for parsing, transforming and producing PDF files

hon-lucene-synonyms

Solr query parser plugin that performs proper query-time synonym expansion.

hook.io

hook.io - the node.js webhook platform

hpricotscape

A browser for master web scrapers. It saves cookies, has a decent user agent, sends referer urls, and let's you (the scraper) easily submit forms on the page. It *doesn't* run the page's javascript yet, however.

html-pipeline

HTML processing filters and utilities

html-structure-view

A simple and useful tool to assist in planning, presentation and visualization of structures of your site.

html2markdown

An HTML-to-markdown conversion helper for PHP

html2text

a Python script that converts a page of HTML into clean, easy-to-read plain ASCII text. Better yet, that ASCII also happens to be valid Markdown (a text-to-HTML format).

html2xml

HTML to XML conversion with WebKit and Tidy

htmlconverter

PHP5 library that provides easy HTML-to-Text conversion

htmltreediff

Structure-aware diff for html and xml documents

hunter-gatherer

NTCIR-1Click