nighttrax / mocha-mugshot-reporter

Mocha reporter for the Mugshot visual regression testing lib

mocha-mugshot-reporter

Mocha reporter for the Mugshot visual regression testing lib

Usage:

mocha test -R mocha-mugshot-reporter

Open 'visual-report/statics/index.html' to access the web interface.

mocha-mugshot-reporter's Issues

Update tests

Unit test that an error is displayed if no paths (aka something happened along the way and Mugshot didn't create the result object) and remove this case from the acceptance tests.
Test the case when a test passed and has diffs.

Move dependencies to dev

Some dependencies are no longer needed because the code is bundled in one file before prepublish.

Add Mugshot as a peer dependency

Need

As a user
I want explicit dependencies
So that I don't get surprised when using a package.

Deliverable

• Mugshot is listed as a peer dependency.

Upgrade version

Update version

Clean up after acceptance tests

Problem

When running the acceptance tests, two folders are created: visual-report and visual-test.
When running the second time, the tests could pass even if the code is broken.

Solution

Clean up after testing.

Write acceptance tests

Idea:

Start up the runner, let it generate the HTML report and then make some assertions on it. Use WebdriverIO to open the report and check simple things. The purpose is to make sure that everything clicks in place.

Migrate styles to LESS, add autoprefixer

Deliverables

• Reporter styles using the power of LESS
• No more duplicated prefixed styles

Solution

Prerequisites

• LESS
• webpack
  • less-loader
  • autoprefixer

TODO

• TBD

cc: @adela-istrate-hs

Update linting rules

Deliverables

• Enforce the same coding style as mugshot

Solution

Prerequisites

• mugshot lint rules

TODO

• TBD

Improve bundled files

Need

As a developer
I want git to ignore all bundled files
But I don't want git to ignore index.html

Solution

Put bundled files in a separate folder and ignore that.
Make a bundle file for every bootstrap glyphicon format for better browser support.
Refactor the output config and include path: './statics' so that we can get rid of statics/.

Build a dedicated interface for visual regression tests

Idea

As a user
I want to have an interactive interface
So that I can see why my visual tests fail

What we have now

What we need

An interface dedicated to visual regression tests, more than what other existing mocha reporters have to offer.

Solution

Make it more user friendly.
Add some style.
View images in more interactive way (e.g. like in GitHub).

Use Cases

Given I run the visual tests
When I open the visual report
Then I see a list of my tests description (passed/failed)
and the total number of tests (passed/failed)
and the total time it took to run the tests

Given I am in the visual report home page
When I click the 'passed' button
Then I see only the passed tests

Given I am in the visual report home page
When I click the 'failed' button
Then I see only the failed tests

Given I am in the visual report 'passed' or 'failed' page
When I click the 'all' button
Then I see all the tests (passed/failed)

Given I am in the visual report
When I click a passed test description
Then I see the baseline

Given I am in the visual report
When I click a failed test description
Then I see the visual regression (the baseline and the new screenshot side by side)

Given I see the visual regression
When I am not sure what is different
Then I can choose between this views: 'default' (mugshot diff image), 'swipe', 'fade'

Tools

React-Image-Diff

Tasks

• Think about a design
• Integrate the React-Image-Diff package
• Take input from user (e.g. React-Image-Diff Demo)
• Add some style
• Display error message for failed tests

Lock NPM dependencies to patch version

Need

Prevent compatibility issues due to changed behaviour in minor versions of npm dependencies.

Solution

Change dependencies in package.json to only use latest patch version.

Static index.html is not using the right css file

Webpack config bundles all styles into bundle.css.

index.html from statics folder is importing the old style.css from the previous configuration.

Styling the UI

Need

As a user
I want to have a nice responsive UI
So that I feel good and be in a good mood when I check my tests results

Deliverables

Nice responsive UI.

Solution

Add some CSS.

Reporter Specs

Need

As a user
I need to have the data presented into an appropriate way
So that I can easily check if my tests are passing or not
And to achieve this through baseline - diff - screenshot checking/visual inspection

Deliverables

• cli reporting, using the outbox mocha reporters
• browser reporting, i.e generating html files

Solution

In order to achieve cli reporting we will use as default the Spec reporter, without extending its prototype, simply instantiating it.

var reporters = require('mocha').reporters;

// runner is passed by mocha to out reporter
new reporters.Spec(runner);

The default can be changed by passing through mocha reporter-options, a key=value pair, which will be key: reporter and the value will be any mocha core reporter.

mocha file --reporter ourReporter --reporter-options reporter=nyan

More details in #2.

To present the data in the browser, we will generate an html file.

Following this structure:

The data generator will be a node module, which will export a single function, that will receive Mocha's runner to listen to the Runner events so that it will return a standard data structure (described in #3) of the test files.

The file generator will be simply an fs.writeFile, which will create a js file, that will be later imported in the html with the script tag. Because it is so simple we will include it directly in the reporter. If at some point there will be more logic, it can be moved very easily in its own module.

There are 2 possibilities:

• to pass the this into a before, that all tests, i.e. before, beforeEach, it, after, afterEach etc., will share (test.ctx.X).
• to pass the this of the global describe, i.e that is the first describe. The problem is that this context is shared only between this describe (test.parent.X), if there are nested describes, then we must search through the chain of parents, i.e test.parent.parent...X, until we reach the last describe, i.e the first after the rootSuite, to find the value putted by our plugin.

The second options can be achieved by saving the suite into a variable, and not checking the every time the whole chain, but this is mocha reprorter implementation.

Support generating diffs with negated assertion

Need

As a user
I want support for negated assertions
So that I can expect diffs and pass the test

Solution

Support something like expect(captureItem).to.not.be.identical that will pass and also generate diffs.

Migrate unit tests to react-test-buffet

Deliverables

• Reporter unit tests using react-test-buffet

Solution

Prerequisites

• react-test-buffet

TODO

• TBD

cc: @adela-istrate-hs

Upgrade to [email protected]

Mugshot 0.3 is passing the assets using relative paths.
This upgrade is needed to embrace that.

Bundle Bootstrap

Need

As a developer
I don't want my code to depend on implementation details
So that I can reuse it anywhere.

The above is not satisfied currently and we're depending on the location of Bootstrap inside the node_modules hierarchy.

Deliverables

• Bootstrap is bundled in the report bundle.

Solution

Remove bootstrap.min.css from index.html and instead import it in entry.jsx.

TODO

• import the Bootstrap CSS in the bundle entry point
• remove Bootstrap as a production dependency and move it to dev dependencies