GithubHelp home page GithubHelp logo

burnpiro / puppeteer-screenshot-tester Goto Github PK

View Code? Open in Web Editor NEW
57.0 5.0 11.0 1.54 MB

Small library that allows us to compare screenshots generated by puppeteer in our tests.

License: MIT License

JavaScript 100.00%
puppeteer nodejs testing jest screenshot test javascript testing-tools puppeteer-screenshot

puppeteer-screenshot-tester's Introduction

puppeteer-screenshot-tester

npm

Small library that allows us to compare screenshots generated by puppeteer in our tests.

Installation

To use Puppeteer Screenshot Tester in your project, run:

yarn add --dev puppeteer-screenshot-tester

or

npm install --save-dev puppeteer-screenshot-tester

Usage

Require the puppeteer-screenshot-tester library:

const ScreenshotTester = require('puppeteer-screenshot-tester')

Initialize Screenshot Tester

const tester = await ScreenshotTester()

Optional arguments:

const tester = await ScreenshotTester(
  [threshold = 0][, includeAA = false[, ignoreColors = false[, matchingBox = { ignoreRectangles = [], includeRectangle = [] } [, errorSettings = Object [, outputSettings = Object]]]]]
)
  • threshold <[number]> A threshold value <0,1> default set to 0, max ratio of difference between images
  • includeAA <[boolean]> Should include anti aliasing?
  • ignoreColors <[boolean]> Should ignore colors?
  • matchingBox <[Object]> Restrict what should be compared
  • matchingBox.ignoreRectangles <[Array<Array[x, y, width, height]>]> Should ignore rectangles? example: [[325,170,100,40], [10,10,200,200]], X and Y should be the coordinates of the top-left corner
  • matchingBox.includeRectangle <[Array<Array[x, y, width, height]>]> Compare only part of screen? example: [[325,170,100,40], [10,10,200,200]], X and Y should be the coordinates of the top-left corner
  • errorSettings <[Object]> change how to display errors (errorType: flat | movement | flatDifferenceIntensity | movementDifferenceIntensity | diffOnly):
    {
      errorColor: {
        red: 255,
        green: 0,
        blue: 255
      },
      errorType: 'flat',
      transparency: 0.7
    }
    
  • outputSettings <[Object]> change the output image settings:
    {
      forceExt: 'jpeg' | 'png' | 'webp' | null,
      compressionLevel: 8 // 0-9 for .png, 0-100 otherwise
      overrides: {} // valid sharp options (see bellow)
    }
    
  • returns: <[function]> resolves to function

overrides:

You can override options passed to sharp image processor. Just paste the overrides object that corresponds with sharp options:

Run the test

const result = await tester(page)

Required arguments:

  • page <[BrowserPage]> BrowserPage returned by puppeteer when calling puppeteer.launch().newPage()

Optional arguments:

const result = await tester(page[, name = 'test'[, screenshotOptions = {}]])
  • name <[string]> name of created screenshot 'test' by default
  • screenshotOptions <[Object]> options passed to Puppeteer's screenshot method See the Puppeteer documentation for more info, plus the following keys:
    • saveNewImageOnError: <[boolean]> saves the undiffed new image on error as ${saveFolder}/${name}-new${ext}
    • overwriteImageOnChange: <[boolean]> overwrites the reference image (${saveFolder}/${name}${ext}) on error / change

Returns

  • <[boolean]> true if images are the same or there is no image to compare (first run)

Examples

const puppeteer = require('puppeteer')
const ScreenshotTester = require('puppeteer-screenshot-tester')

describe('google test', () => {
  let originalTimeout

  // extend default interval to 10s because some image processing might take some time
  // we can do it beforeEach or once per test suite it's up to you
  // if you're running that on fast computer/server probably won't need to do that
  beforeEach(function() {
    originalTimeout = jasmine.DEFAULT_TIMEOUT_INTERVAL
    jasmine.DEFAULT_TIMEOUT_INTERVAL = 10000
  })

  // set default interval timeout for jasmine
  afterEach(function() {
    jasmine.DEFAULT_TIMEOUT_INTERVAL = originalTimeout
  })

  it(`check if google exists`, async () => {
    // create ScreenshotTester with optional config
    const tester = await ScreenshotTester(0.8, false, false, [], {
      transparency: 0.5
    }, { compressionLevel: 8 })

    // setting up puppeteer
    const browser = await puppeteer.launch()
    const page = await browser.newPage()
    await page.setViewport({width: 1920, height: 1080})
    await page.goto('https://www.google.com', { waitUntil: 'networkidle0' })
    await page.type('input[title="Search"]', 'Hello', { delay: 100 })

    // call our tester with browser page returned by puppeteer browser
    // second parameter is optional it's just a test name if provide that's filename
    const result = await tester(page, 'test2', {
      fullPage: true,
    })
    await browser.close()

    // make assertion result is always boolean
    expect(result).toBe(true)
  })
})

Ignoring Rectangles and Including rectangles

const tester = await ScreenshotTester(
    0.1, // threshold
    false, // anti-aliasing
    false, // ignore colors
    { 
      ignoreRectangles: [[650, 300, 700, 200]], 
      includeRectangles: [[300, 200, 1100, 1100]]
    }, // rectangles 
    {
       transparency: 0.5
    }
)

./test2-diff.png

Contributors

Thanks goes to these wonderful people ๐Ÿ˜Ž :


Kemal Erdem

๐Ÿ’ป ๐Ÿ“– ๐Ÿ‘€

Max Harris

๐Ÿ› ๐Ÿ’ป

Andi Smith

๐Ÿ“– โš ๏ธ

JacobJust

๐Ÿ’ป ๐Ÿ“–

This project follows the all-contributors specification. Contributions of any kind welcome!

puppeteer-screenshot-tester's People

Contributors

allcontributors[bot] avatar burnpiro avatar jacobjust avatar lifeworks-bot avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

puppeteer-screenshot-tester's Issues

Option 'type' appears to be ignored?

Currently trying to set filetype to jpeg but it is appearing to be ignored.

    it(`Route / should match screenshot on -> [${deviceName}]`, async () => {
      const tester = await ScreenshotTester();
      const result = await tester(page, `screenshot.${deviceName}`, { type: 'jpeg' });
      expect(result).toBe(true);
    });

Passed througth options path is ignored

I try to make screenshot into a path

await tester(page, testName, { fullPage: true, path: path.join(myDirName, `${testName}.png`) });

but image is create in site of module which is call this method

[BUG] PNG output forced into indexed color

Since v1.5.0, PNG files that puppeteer-screenshot-tester generates are forced into indexed color (256 color limit).

This appears to be due to the use of quality on lines 117 and 147. As on the sharp PNG docs, setting quality implicitly sets palette to true.

Incidentally, line 93 sets compressionLevel instead of quality and therefore the diff files are properly saved losslessly.

Workarounds

Fixes

  • Switch lines 117 and 147 to set compressionLevel.

  • Pass outputSettings or one of its keys directly to sharp's output functions, allowing for passthrough control of Sharp. This could also be specified through something like:

    .png({ quality: outputSettings.compressionLevel || DEFAULT_PNG_COMPRESSION, ...outputSettings.overrides })

ignoreRectangles == ignore an area when compared screenshot?

Is your feature request related to a problem? Please describe.
No sure if ignoreRectangles is meant to ignore part of the screenshot when doing comparision. If no, then would like a feature to ignore some part of a screenshot

Describe the solution you'd like
A way to ignore part of the screenshot, like ignoreRectangles that specify x,y,width,height

Test passes when screenshots are different

Describe the bug
await tester(..) returns true even if screenshots differ

Please complete the following information:

  • OS: macOS High Sierra v10.13.6
  • Node v11.5.0

To Reproduce
Steps to reproduce the behavior:

  1. Just take your example and run it once, so picture is saved
  2. Then change next line:
    await page.type('input[title="Search"]', 'Hello', { delay: 100 })
    to
    await page.type('input[title="Search"]', 'something else', { delay: 100 })
  3. Run test again
    Test passes

Expected behavior
Test shouldn't pass, screenshots are different

Setting the optional threshold allows failing screenshots to pass

Describe the bug
Setting the optional threshold does not work.

Please complete the following information:

  • OS: macOS 10.13.5
  • Node v10.11.0

To Reproduce
Steps to reproduce the behavior:

  1. generate a test image
const puppeteer = require('puppeteer')
const ScreenTest = require('puppeteer-screenshot-tester')
const tape = require('tape')

tape.only('initial view, no zoom', async (t) => {
  const tester = await ScreenTest()
  const browser = await puppeteer.launch({ args: ['--no-sandbox', '--disable-setuid-sandbox'] })
  const page = await browser.newPage()

  await page.setViewport({ width: 1315, height: 562, deviceScaleFactor: 1 })
  await page.goto('http://nodesource.com/')

  t.ok(await tester(page, 'screenshots/initial-view-no-zoom'), 'initial view no zoom screenshot')

  await browser.close()

  t.end()
})
  1. take another screenshot with something different that should fail and set a threshold
const puppeteer = require('puppeteer')
const ScreenTest = require('puppeteer-screenshot-tester')
const tape = require('tape')

tape.only('initial view, no zoom', async (t) => {
  const threshold = 0.1 // <-------- NB: setting this threshold won't work as expected
  const tester = await ScreenTest(threshold)
  const browser = await puppeteer.launch({ args: ['--no-sandbox', '--disable-setuid-sandbox'] })
  const page = await browser.newPage()

  await page.setViewport({ width: 1315, height: 562, deviceScaleFactor: 1 })
  await page.goto('http://google.com/') // <-------- NB: note how the host has changed

  t.ok(await tester(page, 'screenshots/initial-view-no-zoom'), 'initial view no zoom screenshot')

  await browser.close()

  t.end()
})

Expected behavior
When setting a threshold, images with pixel differences that exceed the threshold should fail (but currently do not!)

Stack Trace
data.misMatchPercentage appears to always be a value between 0 and 1, but the threshold gets scaled incorrectly in this comparison:

if (data.isSameDimensions === false || Number(data.misMatchPercentage) > threshold * 100) {

Additional context
This same issue appears to be present on Linux (Ubuntu) hosts.

failing tests

Describe the bug
Tests failing on master

Please complete the following information:

  • OS: macOS 10.13.5
  • Node v10.11.0

To Reproduce
Steps to reproduce the behavior:

  1. clone this repo locally
  2. npm i
  3. npm t

Expected behavior
Passing tests

Stack Trace

Maxs-MacBook-Pro:puppeteer-screenshot-tester max$ npm t

> [email protected] test /Users/max/nodesource/puppeteer-screenshot-tester
> jest

 FAIL  examples/google.test.js
  โ— Test suite failed to run

    SecurityError: localStorage is not available for opaque origins

      at Window.get localStorage [as localStorage] (node_modules/jsdom/lib/jsdom/browser/Window.js:257:15)
          at Array.forEach (<anonymous>)

Test Suites: 1 failed, 1 total
Tests:       0 total
Snapshots:   0 total
Time:        0.577s
Ran all test suites.
npm ERR! Test failed.  See above for more details.

Additional context
Add any other context about the problem here.

include rectangle

You already have a exclude rectangle option.

It would be great if the inverse would be possible too: include rectangle. Sometimes I want to compare only a specific part/rectangle of the screenshot, so it would be really helpful if I only have to define the include instead of having to exclude everything else.

Only works with PNG images, fails on JPG or WebP

Describe the bug
Specified a webp first, then tried JPG (trying to keep image sizes small). Both get the same error

    Error [ERR_UNHANDLED_ERROR]: Unhandled error. (TypeError: Cannot read property 'width' of null

      at node_modules/nodejs-resemble/resemble.js:121:27
      at Object.<anonymous>.exports.PNG.<anonymous> (node_modules/pngjs/lib/png.js:81:7)

Please complete the following information:

  • OS: macOS 11.4
  • Node 14.17.6

To Reproduce
Steps to reproduce the behavior:

  const tester = await ScreenshotTester(0.1);
  const result = await tester(testPage, 'test', {
    fullPage: true,
    type: 'webp',
    path: '/screenshots/test.webp',
  });

&

  const tester = await ScreenshotTester(0.1);
  const result = await tester(testPage, 'test', {
    fullPage: true,
    type: 'jpeg',
    path: '/screenshots/test.jpg',
  });

first run works and generates the image but second run throws an error

Expected behavior
Support all three screenshot formats supported by puppeteer https://github.com/puppeteer/puppeteer/blob/main/docs/api.md#pagescreenshotoptions

Stack Trace

    Error [ERR_UNHANDLED_ERROR]: Unhandled error. (TypeError: Cannot read property 'width' of null

      at node_modules/nodejs-resemble/resemble.js:121:27
      at Object.<anonymous>.exports.PNG.<anonymous> (node_modules/pngjs/lib/png.js:81:7)
      at Object.<anonymous>.module.exports.Object.<anonymous>.ParserAsync._handleError (node_modules/pngjs/lib/parser-async.js:36:8)
      at Object.<anonymous>.module.exports.Object.<anonymous>.Parser._parseSignature (node_modules/pngjs/lib/parser.js:53:12)
      at Object.<anonymous>.module.exports.Object.<anonymous>.ChunkStream._processRead (node_modules/pngjs/lib/chunkstream.js:174:13)
      at Object.<anonymous>.module.exports.Object.<anonymous>.ChunkStream._process (node_modules/pngjs/lib/chunkstream.js:193:14)
      at Object.<anonymous>.module.exports.Object.<anonymous>.ChunkStream.write (node_modules/pngjs/lib/chunkstream.js:61:8))
      at Object.<anonymous>.module.exports.Object.<anonymous>.ChunkStream._process (node_modules/pngjs/lib/chunkstream.js:207:10)
      at Object.<anonymous>.module.exports.Object.<anonymous>.ChunkStream.write (node_modules/pngjs/lib/chunkstream.js:61:8)
      at Object.<anonymous>.module.exports.Object.<anonymous>.ChunkStream.end (node_modules/pngjs/lib/chunkstream.js:74:10)
      at Object.<anonymous>.exports.PNG.Object.<anonymous>.PNG.end (node_modules/pngjs/lib/png.js:98:16)
      at Object.<anonymous>.exports.PNG.Object.<anonymous>.PNG.parse (node_modules/pngjs/lib/png.js:88:8)
      at loadImageData (node_modules/nodejs-resemble/resemble.js:120:9)
      at compare (node_modules/nodejs-resemble/resemble.js:504:4)
      at wrapper (node_modules/nodejs-resemble/resemble.js:582:7)

Additional context
Add any other context about the problem here.

Cannot update to 1.3.0

Describe the bug

npm ERR! Error while executing:
npm ERR! /bin/git ls-remote -h -t git://github.com/burnpiro/node-resemble.js.git
npm ERR!
npm ERR! fatal: cannot come back to cwd: Permission denied
npm ERR!
npm ERR! exited with error code: 128

Please complete the following information:

  • OS: Centos 8
  • node v13.13.0, npm 6.14.7

To Reproduce
npm install -g [email protected]

Expected behavior
Update to 1.3.0

Stack Trace

0 info it worked if it ends with ok
1 verbose cli [
1 verbose cli   '/usr/bin/node',
1 verbose cli   '/bin/npm',
1 verbose cli   'install',
1 verbose cli   '-g',
1 verbose cli   '--unsafe-perm',
1 verbose cli   '[email protected]'
1 verbose cli ]
2 info using [email protected]
3 info using [email protected]
4 verbose npm-session 5c430a1aed7d5a5d
5 silly install loadCurrentTree
6 silly install readGlobalPackageData
7 http fetch GET 200 https://registry.npmjs.org/puppeteer-screenshot-tester 75ms (from cache)
8 silly pacote version manifest for [email protected] fetched in 139ms
9 timing stage:loadCurrentTree Completed in 294ms
10 silly install loadIdealTree
11 silly install cloneCurrentTreeToIdealTree
12 timing stage:loadIdealTree:cloneCurrentTree Completed in 1ms
13 silly install loadShrinkwrap
14 timing stage:loadIdealTree:loadShrinkwrap Completed in 12ms
15 silly install loadAllDepsIntoIdealTree
16 silly resolveWithNewModule [email protected] checking installable status
17 silly removeObsoleteDep removing [email protected] from the tree as its been replaced by a newer version or is no longer required
18 silly removeObsoleteDep removing [email protected] from the tree as its been replaced by a newer version or is no longer required
19 silly removeObsoleteDep removing [email protected] from the tree as its been replaced by a newer version or is no longer required
20 silly removeObsoleteDep removing [email protected] from the tree as its been replaced by a newer version or is no longer required
21 silly removeObsoleteDep removing [email protected] from the tree as its been replaced by a newer version or is no longer required
22 silly removeObsoleteDep removing [email protected] from the tree as its been replaced by a newer version or is no longer required
23 silly removeObsoleteDep removing [email protected] from the tree as its been replaced by a newer version or is no longer required
24 silly removeObsoleteDep removing [email protected] from the tree as its been replaced by a newer version or is no longer required
25 silly removeObsoleteDep removing [email protected] from the tree as its been replaced by a newer version or is no longer required
26 silly removeObsoleteDep removing [email protected] from the tree as its been replaced by a newer version or is no longer required
27 silly removeObsoleteDep removing [email protected] from the tree as its been replaced by a newer version or is no longer required
28 silly fetchPackageMetaData error for node-resemble-js@git://github.com/burnpiro/node-resemble.js.git#0.3.0 Error while executing:
28 silly fetchPackageMetaData /bin/git ls-remote -h -t git://github.com/burnpiro/node-resemble.js.git
28 silly fetchPackageMetaData
28 silly fetchPackageMetaData fatal: cannot come back to cwd: Permission denied
28 silly fetchPackageMetaData
28 silly fetchPackageMetaData exited with error code: 128
29 http fetch GET 304 https://registry.npmjs.org/fs 257ms (from cache)
30 http fetch GET 304 https://registry.npmjs.org/parent-module 245ms (from cache)
31 silly pacote range manifest for parent-module@^0.1.0 fetched in 263ms
32 silly resolveWithNewModule [email protected] checking installable status
33 silly pacote range manifest for fs@^0.0.1-security fetched in 280ms
34 silly resolveWithNewModule [email protected] checking installable status
35 http fetch GET 304 https://registry.npmjs.org/path 303ms (from cache)
36 silly pacote range manifest for path@^0.12.7 fetched in 313ms
37 silly resolveWithNewModule [email protected] checking installable status
38 timing stage:rollbackFailedOptional Completed in 1ms
39 timing stage:runTopLevelLifecycles Completed in 648ms
40 verbose stack Error: exited with error code: 128
40 verbose stack     at ChildProcess.<anonymous> (/usr/lib/node_modules/npm/node_modules/pacote/lib/util/finished.js:12:19)
40 verbose stack     at ChildProcess.emit (events.js:315:20)
40 verbose stack     at maybeClose (internal/child_process.js:1026:16)
40 verbose stack     at Socket.<anonymous> (internal/child_process.js:441:11)
40 verbose stack     at Socket.emit (events.js:315:20)
40 verbose stack     at Pipe.<anonymous> (net.js:674:12)
41 verbose cwd /root
42 verbose Linux 4.18.0-147.5.1.el8_1.x86_64
43 verbose argv "/usr/bin/node" "/bin/npm" "install" "-g" "[email protected]"
44 verbose node v13.13.0
45 verbose npm  v6.14.4
46 error Error while executing:
46 error /bin/git ls-remote -h -t git://github.com/burnpiro/node-resemble.js.git
46 error
46 error fatal: cannot come back to cwd: Permission denied
46 error
46 error exited with error code: 128
47 verbose exit [ 1, true ]

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.