mechanicalsoup's Introduction

MechanicalSoup

A Python library for automating interaction with websites. MechanicalSoup automatically stores and sends cookies, follows redirects, and can follow links and submit forms.

Installation

From PyPI

 pip install MechanicalSoup

Rationale

WWW-Mechanize and Mechanize are similar Perl and Ruby libraries. Unfortunately, their Python sister project has become inactive, and is now incompatible with recent Python versions (Python 3.x). This library provides a similar API, built on Python giants Requests (for http sessions) and BeautifulSoup (for document navigation).

Example

From example.py, code to log into the GitHub website:

import mechanicalsoup

browser = mechanicalsoup.Browser()

# request github login page
login_page = browser.get("https://github.com/login")

# find login form
login_form = login_page.soup.select("#login")[0].select("form")[0]

# specify username and password
login_form.select("#login_field")[0]['value'] = "username"
login_form.select("#password")[0]['value'] = "password"

# submit form
page2 = browser.submit(login_form, login_page.response.url)

# verify we are now logged in
assert page2.soup.select(".logout-form")

# verify we remain logged in (thanks to cookies) as we browse the rest of the site
page3 = browser.get("https://github.com/colonelpanic/MechanicalSoup")
assert page3.soup.select(".logout-form")

For an example with a more complicated form (with checkboxes, radio buttons and textareas), read tests/test_browser.py.