A browser testing and web scraping library for PHP and Symfony
Panther is a convenient standalone library to scrape websites and to run end-to-end tests using real browsers.
Panther is super powerful. It leverages the W3C's WebDriver protocol to drive native web browsers such as Google Chrome and Firefox.
Panther is very easy to use, because it implements Symfony's popular BrowserKit and DomCrawler APIs, and contains all the features you need to test your apps. It will sound familiar if you have ever created a functional test for a Symfony app: as the API is exactly the same! Keep in mind that Panther can be used in every PHP project, as it is a standalone library.
Panther automatically finds your local installation of Chrome or Firefox and launches them (thanks to ChromeDriver and GeckoDriver), so you don't need to install anything on your computer, neither Selenium server nor any other obscure driver.
In test mode, Panther automatically starts your application using the PHP built-in web-server. You can focus on writing your tests or web-scraping scenario and Panther will take care of everything else.
Unlike testing and web scraping libraries you're used to, Panther:
- executes the JavaScript code contained in webpages
- supports everything that Chrome (or Firefox) implements
- allows screenshots taking
- can wait for asynchronously loaded elements to show up
- lets you run your own JS code or XPath queries in the context of the loaded page
- supports custom Selenium server installations
- supports remote browser testing services including SauceLabs and BrowserStack
Use Composer to install Panther in your project. You may want to use the --dev
flag if you want to use Panther for testing only and not for web scraping in a production environment:
composer req symfony/panther
composer req --dev symfony/panther
Warning: On *nix systems, the unzip
command must be installed or you will encounter an error similar to RuntimeException: sh: 1: exec: /app/vendor/symfony/panther/src/ProcessManager/../../chromedriver-bin/chromedriver_linux64: Permission denied
(or chromedriver_linux64: not found
).
The underlying reason is that PHP's ZipArchive
doesn't preserve UNIX executable permissions.
If you intend to use Panther to test your application, we strongly recommended to register the Panther PHPUnit extension. While not strictly mandatory, this extension dramatically improves the testing experience by boosting the performance and allowing to use the interactive debugging mode.
To register the Panther extension, add the following lines to phpunit.xml.dist
:
<!-- phpunit.xml.dist -->
<extensions>
<extension class="Symfony\Component\Panther\ServerExtension" />
</extensions>
Without the extension, the web server used by Panther to serve the application under test is started on demand and
stopped when tearDownAfterClass()
is called.
On the other hand, when the extension is registered, the web server will be stopped only after the very last test.
To use the Panther extension, PHPUnit 7.3+ is required. Nonetheless, a listener is provided for older versions:
<!-- phpunit.xml.dist -->
<listeners>
<listener class="Symfony\Component\Panther\ServerListener" />
</listeners>
This listener will start the web server on demand like previously, but it will stop it after each test suite.
<?php
require __DIR__.'/vendor/autoload.php'; // Composer's autoloader
$client = \Symfony\Component\Panther\Client::createChromeClient();
// Or, if you care about the open web and prefer to use Firefox
$client = \Symfony\Component\Panther\Client::createFirefoxClient();
$client->request('GET', 'https://api-platform.com'); // Yes, this website is 100% written in JavaScript
$client->clickLink('Get started');
// Wait for an element to be present in the DOM (even if hidden)
$crawler = $client->waitFor('#installing-the-framework');
// Alternatively, wait for an element to be visible
$crawler = $client->waitForVisibility('#installing-the-framework');
echo $crawler->filter('#installing-the-framework')->text();
$client->takeScreenshot('screen.png'); // Yeah, screenshot!
The PantherTestCase
class allows you to easily write E2E tests. It automatically starts your app using the built-in PHP
web server and let you crawl it using Panther.
To provide all of the testing tools you're used to, it extends PHPUnit's TestCase
.
If you are testing a Symfony application, PantherTestCase
automatically extends the WebTestCase
class.
It means you can easily create functional tests, which can directly execute the kernel of your application and access all
your existing services. In this case, you can use all crawler test assertions
provided by Symfony with Panther.
<?php
namespace App\Tests;
use Symfony\Component\Panther\PantherTestCase;
class E2eTest extends PantherTestCase
{
public function testMyApp(): void
{
$client = static::createPantherClient(); // Your app is automatically started using the built-in web server
$client->request('GET', '/mypage');
// Use any PHPUnit assertion, including the ones provided by Symfony
$this->assertPageTitleContains('My Title');
$this->assertSelectorTextContains('#main', 'My body');
// Or the one provided by Panther
$this->assertSelectorIsEnabled('.search');
$this->assertSelectorIsDisabled('[type="submit"]');
$this->assertSelectorIsVisible('.errors');
$this->assertSelectorIsNotVisible('.loading');
// Use waitForX methods to wait until some asynchronous process finish
$client->waitFor('.popin'); // element is attached to the DOM
$client->waitForStaleness('.popin'); // element is removed from the DOM
$client->testWaitForVisibility('.loader'); // element of the DOM becomes visible
$client->testWaitForInvisibility('.loader'); // element of the DOM becomes hidden
$client->waitForElementToContain('.total', '25 β¬'); // text is inserted in the element content
$client->waitForElementToNotContain('.promotion', '5%'); // text is removed from the element content
}
}
To run this test:
phpunit tests/E2eTest.php
Panther also gives you instant access to other BrowserKit-based implementations of Client
and Crawler
.
Unlike Panther's native client, these alternative clients don't support JavaScript, CSS and screenshot capturing,
but they are super-fast!
Two alternative clients are available:
- The first directly manipulates the Symfony kernel provided by
WebTestCase
. It is the fastest client available, but it is only available for Symfony apps. - The second leverages Symfony's HttpBrowser. It is an intermediate between Symfony's kernel and Panther's test clients. HttpBrowser sends real HTTP requests using Symfony's HttpClient component. It is fast and is able to browse any webpage, not only the ones of the application under test. However, HttpBrowser doesn't support JavaScript and other advanced features because it is entirely written in PHP. This one is available even for non-Symfony apps!
The fun part is that the 3 clients implement the exact same API, so you can switch from one to another just by calling the appropriate factory method, resulting in a good trade-off for every single test case (Do I need JavaScript? Do I need to authenticate with an external SSO server? Do I want to access the kernel of the current request? ... etc).
Here is how to retrieve instances of these clients:
<?php
namespace App\Tests;
use Symfony\Component\Panther\PantherTestCase;
use Symfony\Component\Panther\Client;
class E2eTest extends PantherTestCase
{
public function testMyApp(): void
{
$symfonyClient = static::createClient(); // A cute kitty: Symfony's functional test tool
$httpBrowserClient = static::createHttpBrowserClient(); // An agile lynx: HttpBrowser
$pantherClient = static::createPantherClient(); // A majestic Panther
$firefoxClient = static::createPantherClient(['browser' => static::FIREFOX]); // A splendid Firefox
// Both HttpBrowser and Panther benefits from the built-in HTTP server
$customChromeClient = Client::createChromeClient(null, null, [], 'https://example.com'); // Create a custom Chrome client
$customFirefoxClient = Client::createFirefoxClient(null, null, [], 'https://example.com'); // Create a custom Firefox client
$customSeleniumClient = Client::createSeleniumClient('http://127.0.0.1:4444/wd/hub', null, 'https://example.com'); // Create a custom Selenium client
// When initializing a custom client, the integrated web server IS NOT started automatically.
// Use PantherTestCase::startWebServer() or WebServerManager if you want to start it manually.
// enjoy the same API for the 3 felines
// $*client->request('GET', '...')
$kernel = static::createKernel(); // If you are testing a Symfony app, you also have access to the kernel
// ...
}
}
Creating Isolated Browsers to Test Apps Using Mercure or WebSocket
Panther provides a convenient way to test applications with real-time capabilities which use Mercure, WebSocket and similar technologies.
PantherTestCase::createAdditionalPantherClient()
creates additional, isolated browsers which can interact with each other.
For instance, this can be useful to test a chat application having several users connected simultaneously:
<?php
use Symfony\Component\Panther\PantherTestCase;
class ChatTest extends PantherTestCase
{
public function testChat(): void
{
$client1 = self::createPantherClient();
$client1->request('GET', '/chat');
// Connect a 2nd user using an isolated browser and say hi!
$client2 = self::createAdditionalPantherClient();
$client2->request('GET', '/chat');
$client2->submitForm('Post message', ['message' => 'Hi folks ππ»']);
// Wait for the message to be received by the first client
$client1->waitFor('.message');
// Symfony Assertions are always executed in the **primary** browser
$this->assertSelectorTextContains('.message', 'Hi folks ππ»');
}
}
Use the Client::ping()
method to check if the WebDriver connection is still active (useful for long running tasks).
Since Panther implements the API of popular libraries, it already has extensive documentation:
- For the
Client
class, read the BrowserKit documentation - For the
Crawler
class, read the DomCrawler documentation - For WebDriver, read the PHP WebDriver documentation
The following environment variables can be set to change some Panther's behaviour:
PANTHER_NO_HEADLESS
: to disable browser's headless mode (will display the testing window, useful to debug)PANTHER_WEB_SERVER_DIR
: to change the project's document root (default to./public/
, relative paths must start by./
)PANTHER_WEB_SERVER_PORT
: to change the web server's port (default to9080
)PANTHER_WEB_SERVER_ROUTER
: to use a web server router script which is run at the start of each HTTP requestPANTHER_EXTERNAL_BASE_URI
: to use an external web server (the PHP built-in web server will not be started)PANTHER_APP_ENV
: to override theAPP_ENV
variable passed to the web server running the PHP app
PANTHER_NO_SANDBOX
: to disable Chrome's sandboxing (unsafe, but allows to use Panther in containers)PANTHER_CHROME_DRIVER_BINARY
: to use anotherchromedriver
binary, instead of relying on the ones already provided by PantherPANTHER_CHROME_ARGUMENTS
: to customize Chrome arguments. You need to setPANTHER_NO_HEADLESS
to fully customize.PANTHER_CHROME_BINARY
: to use anothergoogle-chrome
binary
PANTHER_GECKO_DRIVER_BINARY
: to use anothergeckodriver
binary, instead of relying on the ones already provided by PantherPANTHER_FIREFOX_ARGUMENTS
: to customize Firefox arguments. You need to setPANTHER_NO_HEADLESS
to fully customize.PANTHER_FIREFOX_BINARY
: to use anotherfirefox
binary
Accessing To Hidden Text
According to the spec, WebDriver implementations return only the displayed text by default.
When you filter on a head
tag (like title
), the method text()
returns an empty string. Use the method html()
to get
the complete contents of the tag, including the tag itself.
Panther can make a pause in your tests suites after a failure.
It is a break time really appreciated for investigating the problem through the web browser.
For enabling this mode, you need the --debug
PHPUnit option without the headless mode:
$ export PANTHER_NO_HEADLESS=1
$ phpunit --debug
Test 'App\AdminTest::testLogin' started
Error: something is wrong.
Press enter to continue...
To use the interactive mode, the PHPUnit extension must be registered.
Sometimes, it's convenient to reuse an existing web server configuration instead of starting the built-in PHP one.
To do so, set the external_base_uri
option:
<?php
namespace App\Tests;
use Symfony\Component\Panther\PantherTestCase;
class E2eTest extends PantherTestCase
{
public function testMyApp(): void
{
$pantherClient = static::createPantherClient(['external_base_uri' => 'https://localhost']);
// the PHP integrated web server will not be started
}
}
To use a proxy server, set the following environment variable: PANTHER_CHROME_ARGUMENTS='--proxy-server=socks://127.0.0.1:9050'
To force Chrome to accept invalid and self-signed certificates, set the following environment variable: PANTHER_CHROME_ARGUMENTS='--ignore-certificate-errors'
This option is insecure, use it only for testing in development environments, never in production (e.g. for web crawlers).
Here is a minimal Docker image that can run Panther:
FROM php:latest
RUN apt-get update && apt-get install -y libzip-dev zlib1g-dev chromium && docker-php-ext-install zip
ENV PANTHER_NO_SANDBOX 1
# Not mandatory, but recommended
ENV PANTHER_CHROME_ARGUMENTS='--disable-dev-shm-usage'
Build it with docker build . -t myproject
Run it with docker run -it -v "$PWD":/srv/myproject -w /srv/myproject myproject bin/phpunit
If you are using Alpine Linux, you may need to use another chromedriver
binary.
RUN apk add --no-cache \
chromium \
chromium-chromedriver
ENV PANTHER_CHROME_DRIVER_BINARY /usr/lib/chromium/chromedriver
Panther works out of the box with GitHub Actions.
Here is a minimal .github/workflows/panther.yml
file to run Panther tests:
name: Run Panther tests
on: [ push, pull_request ]
jobs:
tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Install dependencies
run: composer install -q --no-ansi --no-interaction --no-scripts --no-suggest --no-progress --prefer-dist
- name: Run test suite
run: vendor/bin/phpunit
Panther will work out of the box with Travis CI if you add the Chrome addon.
Here is a minimal .travis.yml
file to run Panther tests:
language: php
addons:
# If you don't use Chrome, or Firefox, remove the corresponding line
chrome: stable
firefox: latest
php:
- 7.4
script:
- phpunit
Here is a minimal .gitlab-ci.yml
file to run Panther tests with Gitlab CI:
image: ubuntu:bionic
services:
- postgres:11
variables:
POSTGRES_PASSWORD: root
POSTGRES_USER: root
POSTGRES_DB: db
before_script:
- apt-get update
- apt-get install software-properties-common -y
- ln -sf /usr/share/zoneinfo/Asia/Tokyo /etc/localtime
- apt-get install curl wget php php-cli php7.2 php7.2-common php7.2-curl php7.2-pgsql php7.2-mysql php7.2-intl php7.2-gd php7.2-xml php7.2-opcache php7.2-mbstring php7.2-zip libfontconfig1 fontconfig libxrender-dev libfreetype6 libxrender1 zlib1g-dev xvfb chromium-browser chromium-chromedriver -y -qq
- export PANTHER_CHROME_DRIVER_BINARY="/usr/lib/chromium-browser/chromedriver"
- export PANTHER_NO_SANDBOX=1
- export PANTHER_WEB_SERVER_PORT=9080
- php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
- php composer-setup.php --install-dir=/usr/local/bin --filename=composer
- php -r "unlink('composer-setup.php');"
- composer
- chromedriver --version
- php -v
- php -m
- composer install --ignore-platform-reqs
- bin/console doctrine:schema:update --force
- bin/console doctrine:schema:validate
- bin/console doctrine:fixtures:load
stages:
- test
test:
script:
- vendor/bin/simple-phpunit tests/Controller/E2eTest.php
Panther will work out of the box with AppVeyor as long as Google Chrome is installed.
Here is a minimal appveyor.yml
file to run Panther tests:
build: false
platform: x86
clone_folder: c:\projects\myproject
cache:
- '%LOCALAPPDATA%\Composer\files'
install:
- ps: Set-Service wuauserv -StartupType Manual
- cinst -y php composer googlechrome
- refreshenv
- cd c:\tools\php74
- copy php.ini-production php.ini /Y
- echo date.timezone="UTC" >> php.ini
- echo extension_dir=ext >> php.ini
- echo extension=php_openssl.dll >> php.ini
- echo extension=php_mbstring.dll >> php.ini
- echo extension=php_curl.dll >> php.ini
- echo memory_limit=3G >> php.ini
- cd %APPVEYOR_BUILD_FOLDER%
- composer install --no-interaction --no-progress
test_script:
- cd %APPVEYOR_BUILD_FOLDER%
- php vendor\phpunit\phpunit\phpunit
If you want to use Panther with other testing tools like LiipFunctionalTestBundle or if you just need to use a different base class, Panther has got you covered. It provides you with the Symfony\Component\Panther\PantherTestCaseTrait
and you can use it to enhance your existing test-infrastructure with some Panther awesomeness:
<?php
namespace App\Tests\Controller;
use Liip\FunctionalTestBundle\Test\WebTestCase;
use Symfony\Component\Panther\PantherTestCaseTrait;
class DefaultControllerTest extends WebTestCase
{
use PantherTestCaseTrait; // this is the magic. Panther is now available.
public function testWithFixtures(): void
{
$this->loadFixtures([]); // load your fixtures
$client = self::createPantherClient(); // create your panther client
$client->request('GET', '/');
}
}
The following features are not currently supported:
- Crawling XML documents (only HTML is supported)
- Updating existing documents (browsers are mostly used to consume data, not to create webpages)
- Setting form values using the multidimensional PHP array syntax
- Methods returning an instance of
\DOMElement
(because this library usesWebDriverElement
internally) - Selecting invalid choices in select
Pull Requests are welcome to fill the remaining gaps!
Many of the wild cat species are highly threatened. If you like this software, help save the (real) panthers by donating to the Panthera organization.
Created by KΓ©vin Dunglas. Sponsored by Les-Tilleuls.coop.
Panther is built on top of PHP WebDriver and several other FOSS libraries. It has been inspired by Nightwatch.js, a WebDriver-based testing tool for JavaScript.