Simple utility to go to a URL and wait for the HTTP response
puppeteer-browser-ready is a helper utility to reduce the amount of boilerplate code needed to tell Puppeteer to visit a web page and and retrieve the HTML. It's primarily intended for use within Mocha test cases. In addition to the raw HTML, you get a node-html-parsed root so you can immediately run queries on the DOM.
Install packages:
$ npm install --save-dev puppeteer puppeteer-browser-readyImport packages:
import puppeteer from 'puppeteer';
import { browserReady } from 'puppeteer-browser-ready';Use the browserReady.goto(url, options) function to tell Puppeteer which page to open.
The Promise will resolve with a Web object containing a title field and a html field.
Pass the Web object to the browserReady.close(web) function to disconnect the page.
const url = 'https://pretty-print-json.js.org/';
let web; //fields: browser, page, response, status, location, title, html, root
before(async () => web = await puppeteer.launch().then(browserReady.goto(url));
after(async () => await browserReady.close(web));| Name (key) | Type | Default | Description |
|---|---|---|---|
parseHtml |
boolean | true |
Return the DOM root as an HTMLElement (node-html-parsed). |
verbose |
boolean | false |
Output HTTP connection debug messages. |
| Name (key) | Type | Default | Description |
|---|---|---|---|
autoCleanup |
boolean | true |
Terminate connection on interruption (SIGINT). |
folder |
string | '.' |
Document root for the static web server. |
port |
number | 0 |
Port number for server (0 find open port). |
verbose |
boolean | true |
Output informational messages. |
See the TypeScript declarations at the top of the puppeteer-browser-ready.ts file.
The browserReady.goto(url, options) function returns a function that takes a Puppeteer Browser
object and returns a Promise that resolves with a Web object:
type Web = {
browser: Puppeteer.Browser,
page: Puppeteer.Page,
response: HTTPResponse | null,
location: Location,
title: string,
html: string,
root: HTMLElement | null, //see node-html-parsed library
};The optional browserReady.startWebServer(options) function starts a static web server and returns
a Promise for when the server is ready:
export type Http = {
server: Server,
terminator: httpTerminator.HttpTerminator,
folder: string,
url: string,
port: number,
verbose: boolean,
};Code:
import puppeteer from 'puppeteer';
import { browserReady } from 'puppeteer-browser-ready';
const handleResponse = (web) => {
console.log('Hello, World!');
console.log('web fields:', Object.keys(web).join(', '));
console.log(`The HTML from ${web.location.href} is ${web.html.length} characters`,
`long and contains ${web.root.querySelectorAll('p').length} <p> tags.`);
return web;
};
puppeteer.launch()
.then(browserReady.goto('https://pretty-print-json.js.org/'))
.then(handleResponse)
.then(browserReady.close);Output:
Hello, World!
web fields: browser, page, response, status, location, title, html, root
The HTML from https://pretty-print-json.js.org/ is 8200 characters
long and contains 7 <p> tags.
Code:
// Mocha Specification Suite
// Imports
import puppeteer from 'puppeteer';
import { assertDeepStrictEqual } from 'assert-deep-strict-equal';
import { browserReady } from 'puppeteer-browser-ready';
// Setup
const url = 'https://pretty-print-json.js.org/';
let web; //fields: browser, page, response, status, location, title, html, root
const loadWebPage = async () =>
web = await puppeteer.launch().then(browserReady.goto(url));
const closeWebPage = async () =>
await browserReady.close(web);
/////////////////////////////////////////////////////////////////////////////////////
describe('The web page', () => {
before(loadWebPage);
after(closeWebPage);
it('has the correct URL', () => {
const actual = { status: web.status, url: web.location.href };
const expected = { status: 200, url: url };
assertDeepStrictEqual(actual, expected);
});
it('title starts with "Pretty-Print JSON"', () => {
const actual = { title: web.title.substring(0, 17) };
const expected = { title: 'Pretty-Print JSON' };
assertDeepStrictEqual(actual, expected);
});
it('body has exactly one header, main, and footer -- node-html-parsed', () => {
const getTags = (elems) => [...elems].map(elem => elem.tagName.toLowerCase());
const actual = getTags(web.root.querySelectorAll('body >*'));
const expected = ['header', 'main', 'footer'];
assertDeepStrictEqual(actual, expected);
});
it('body has exactly one header, main, and footer -- page.$$eval()', async () => {
const getTags = (elems) => elems.map(elem => elem.nodeName.toLowerCase());
const actual = await web.page.$$eval('body >*', getTags);
const expected = ['header', 'main', 'footer'];
assertDeepStrictEqual(actual, expected);
});
});
/////////////////////////////////////////////////////////////////////////////////////
describe('The document content', () => {
before(loadWebPage);
after(closeWebPage);
it('has a 🚀 traveling to 🪐!', () => {
const actual = { '🚀': !!web.html.match(/🚀/g), '🪐': !!web.html.match(/🪐/g) };
const expected = { '🚀': true, '🪐': true };
assertDeepStrictEqual(actual, expected);
});
});Output:
The web page
✓ has the correct URL
✓ title starts with "Pretty-Print JSON"
✓ body has exactly one header, main, and footer -- node-html-parsed
✓ body has exactly one header, main, and footer -- page.$$eval()
The document content
✓ has a 🚀 traveling to 🪐!
The startWebServer(options) and shutdownWebServer(http) functions can be used in global fixtures to start and shutdown a static web server.
For example, the spec/fixtures/setup-teardown.js file below starts a web server on port 7123
with the web root pointed to the project's docs folder.
Code:
// Specification Fixtures
import { browserReady } from 'puppeteer-browser-ready';
let http; //fields: server, terminator, folder, url, port, verbose
// Setup
const mochaGlobalSetup = async () => {
http = await browserReady.startWebServer({ folder: 'docs', port: 7123 });
};
// Teardown
const mochaGlobalTeardown = async () => {
await browserReady.shutdownWebServer(http);
};
export { mochaGlobalSetup, mochaGlobalTeardown };Run specification suites with global fixtures:
$ npx mocha spec/*.spec.js --require spec/fixtures/setup-teardown.js
Output:
[2021-07-14T11:38:22.892Z] Web Server - listening: true 7123 http://localhost:7123/
...Output of Mocha specification suites here...
[2021-07-14T11:38:26.704Z] Web Server - shutdown: true
By default Mocha allows a test 2,000 ms to complete before timing out with a failure.
Web page load times can vary significantly, so it's sometimes a good idea to use the timeout
option to bump up the allowed test execution time.
Example configuration in package.json to allow 5,000 ms:
"scripts": {
"pretest": "run-scripts clean build",
"test": "mocha spec/*.spec.js --timeout 7000"
},CLI Build Tools for package.json
- 🎋 add-dist-header: Prepend a one-line banner comment (with license notice) to distribution files
- 📄 copy-file-util: Copy or rename a file with optional package version number
- 📂 copy-folder-util: Recursively copy files from one folder to another folder
- 🪺 recursive-exec: Run a command on each file in a folder and its subfolders
- 🔍 replacer-util: Find and replace strings or template outputs in text files
- 🔢 rev-web-assets: Revision web asset filenames with cache busting content hash fingerprints
- 🚆 run-scripts-util: Organize npm package.json scripts into named groups of easy to manage commands
- 🚦 w3c-html-validator: Check the markup validity of HTML files using the W3C validator
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent