I have recreated the issue i'm seeing here: https://codesandbox.io/s/jn781kl4zy?codemirror=1&fontsize=14

For some reason this very basic form example is not valid using the jest-axe plugin. I have validated that this test should pass using the aXe inline validation playground.

This matcher is AWESOME. I'd love to use it in TypeScript projects at scale. Would you be up for including .d.ts files with it?

Running jest-axe is not enough.

We need to add a little section the the README that explains what kinds of errors aXe can catch, and what requires more work.

Resources:

• https://alphagov.github.io/accessibility-tool-audit/ - ~30% of issues found with aXe
• https://www.gov.uk/service-manual/technology/testing-for-accessibility#what-to-test

Is it possible to specify the run parameters? I'm particularly interested in the Include-Exclude Object

https://github.com/nickcolley/jest-axe/blob/6965b7a11eff021d16785f8c70f6ee1cbb64b0a8/package.json#L27

This is a request to update axe-core version.

Currently my project is using jest-axe and axe-core 4.2.* but would like to update to the newer version of axe-core which has newer features, but can't because jest-axe doesn't support it.

Should include the Design System team, GOV.UK team, Jest and Axe

Hey folks, apologies for the silly question.

I looked around, but couldn't confirm which rule sets are run by default if axe is not configured with custom options. In axe-core's docs, it's mentioned that experimental rules are off by default; does that mean all rules here but experimental rules are run by default through jest-axe?

Thank you.

Is there any estimate on an update to the axe-core version/is it actively being worked?

We use this package and have color contrast tests disabled in a few areas due to bugs existing in axe-core. Since axe-core has been updated (and apparently has fixed for those issues), I wanted to make an issue here for tracking.

Thank you for all you've done with this package!

First, thanks for this break tool.
It was very easy to set up.

The error I get

I get this error:

  ● <KFormInputLabel> › Does not have basic accessibility issues

    No violations found in aXe results object

      64 | 
      65 |     const a11yTestResults = configuredAxe(component.container);
    > 66 |     expect(a11yTestResults).toHaveNoViolations();
         |                             ^
      67 |   });

Here's my code:

import * as React from "react";
import { render, cleanup } from "@testing-library/react";
import { toHaveNoViolations, axe } from "jest-axe";
import "@testing-library/jest-dom";

expect.extend(toHaveNoViolations);

/* more code */

describe("<KFormInputLabel>", () => {
  afterEach(() => {
    jest.resetAllMocks();
    cleanup();
  });

  /* more code */

  it("Does not have basic accessibility issues", async () => {
    const component = render(
      <KFormInputLabel htmlFor="dow3uef98i6">
        Label text
      </KFormInputLabel>
    );

    const a11yTestResults = axe(
      component.container
    );
    expect(a11yTestResults).toHaveNoViolations();
  });
});

I'm not sure what the error means, or what to do about it

"No violations found in aXe results object" sounds like the test should pass. So why does it fail?

Also, I can't see why I'm getting this error when testing this component. It didn't happen it other places where I've used the same test code.

Any thought or guidance would help.

Hey, jest-axe is fantastic and very helpful in teaching my team more a11y responsibility. My tests are working, but on every single file, I am getting the following error:

console.error node_modules/react-dom/cjs/react-dom.development.js:88 Warning: render(): Rendering components directly into document.body is discouraged since its children are often manipulated by third-party scripts and browser extensions. This may lead to subtle reconciliation issues. Try rendering into a container element created for your app.

Mainly, this is inconvenient. It fills up the terminal and constantly prompts questions from the team. I do not see a way to turn it off from the react-dom side, and I'm curious if there is another solution from the jest-axe side. Our app was built with CRA, so our jest customization options are limited as well.

Hi, it seems the types package is co-owned by someone who stopped ownership, so only one left (erbridge) :)

Wouldnt it be handy to just export typings from this main library itself? I do get a type mismatch in RunOptions with 'axe-core' and 'jest-axe'. Or do you want to keep typings explicitly out of this package?

https://github.com/DefinitelyTyped/DefinitelyTyped/blob/master/types/jest-axe/index.d.ts

Version 10 of Node.js (code name Dubnium) has been released! 🎊

To see what happens to your code in Node.js 10, Greenkeeper has created a branch with the following changes:

• Replaced the old Node.js version in your .nvmrc with the new one
• The new Node.js version is in-range for the engines in 1 of your package.json files, so that was left alone

If you’re interested in upgrading this repo to Node.js 10, you can open a PR with these changes. Please note that this issue is just intended as a friendly reminder and the PR as a possible starting point for getting your code running on Node.js 10.

Your Greenkeeper Bot 🌴

I came across this when trying to determine the root cause of what I originally thought was a bug. I had a test that I expected to have an aXe failure, but it turns out it is falling into the incomplete array of rule checks. I assume the toHaveNoViolations matcher essentially checks the for violations.length === 0; I'd like to propose a corresponding toHaveNoIncomplete matcher which checks the incomplete array.

I am writing a sample test for accessibility scan. I could see that the axe run is not able to find the violations in the inner HTML.

Example unit test :

test("Sample Accessibility Test", async () => {
	const sampleHTML: string = "<div class=\"abc1\" role=\"main\"><div class=\"page-content\"><div class=\"abc1-container\"><div class=\"abc1-content\"><div class=\"abc2\"><button type=\"button\" id=\"my-button\" class=\"my-button\" data-is-focusable=\"true\"><span class=\"abc3\" data-automationid=\"splitbuttonprimary\"><span class=\"textContainer\"><span class=\"label\" id=\"my-label\">Sample Button</span></span></span></button></div></div></div></div></div>";
	const results = await axe(sampleHTML);
	expect(results).toHaveNoViolations();
});

Actual -
This test passes with no violations.

Expected -
The test should fail with violations related to aira-labels for the button.

First of all: Great job!

It would be nice if it were possible to define the impact levels that should be used ("critical", "moderate", "etc.") .

This axe plugin for cypress does this in a very nice way (see here - includedImpacts).

Seen at https://github.com/nickcolley/jest-axe/blob/master/index.js#L81

Hi there, thanks for your work on jest-axe!

I just upgraded to 3.4 and I'm getting the following error (I suspect related to the axe update):

 "All page content must be contained by landmarks (region)"

I can set up a global config as you suggest on the readme but I was wondering if it would make sense to disable this rule by default on unit tests

I get the following error when trying to get this working with ts-jest:

TypeScript diagnostics (customize using `[jest-config].globals.ts-jest.diagnostics` option):
    src/components/PriceBreakdown/components/ChargesList/ChargesList.test.tsx:124:19 - error TS2339: Property 'toHaveNoViolations' does not exist on type 'Matchers<any>'.

I am following the documentation correctly so I was just wondering if there is any documentation around using this with typescript.

example of how I am using it:

import { axe, toHaveNoViolations } from "jest-axe";

expect.extend(toHaveNoViolations);

it("should be accessible", async () => {
         // wrap is just a react component
	const { container } = wrap([fee1, fee2, fee3], "Some text", true);
	const results = await axe(container);
	expect(results).toHaveNoViolations();
});

When installing jest-axe with the command yarn add jest-axe -D on a NodeJS 9 system the following error appears:

error [email protected]: The engine "node" is incompatible with this module. Expected version "8.9.4".
error Found incompatible module

If i directly require a copy of the jest-axe/index.js file in my project then all is well. So removing/changing the engines/node variable in package.json might resolve this problem.

https://twitter.com/marcysutton/status/963569635340636160

maybe could add a puppeteer example

Sorry for posting here instead of StackOverflow. There aren't any tags for jest-axe on SO and I couldn't find any spectrum chat either. I'll close this issue as soon as I get a reply.

I have a dialog component that needs to have its state set to open to render it. In react-testing-library I simply include a button in the initial render that sets the dialog state to open onClick (and removes the button from the DOM), so all I need to do is include fireEvent.click(button) before I run any tests to get the component to render. I'm not sure how to do this with jest-axe though and can't find anything about it in the documentation.

How do I go about doing this?

The devDependency @testing-library/react was updated from 9.1.0 to 9.1.1.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

@testing-library/react is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

Hi @NickColley!

first of all thanks for your contribution in a11y tooling, it definitely helps making apps more inclusive. Regarding my issue, some of my tests are timing out after having jest-aXe integrated in my React Testing Library suite. I do have a quite recent version of RTL so cleanup won't help.

It is interesting to point that tests would pass:

• --runInBand flag was used

Regarding the context of tests, all of them are "wrapper" components (like forms) and not bare elements, and they don't really make any extra assertions. They are just rendered and then passed to jest-aXe. It should be noted that prior to aXe's integration, their it renders correctly case would never have any issues. AFAIK, at the time of the timeout <body /> is empty.

Have already tried fiddling with the testTimeout and --max-old-space-size to no avail.

Any ideas of what might be the culprit or steps to identify potential problems are more than welcome.

Thanks in advance!

Here is a simple repro, the test below passes. If you uncomment the jest.useFakeTimers() in the beforeEach the test will timeout no matter what else you do.

import React from 'react'
import { mount } from 'enzyme'

import { toHaveNoViolations, axe } from 'jest-axe'

expect.extend(toHaveNoViolations)

const MyComponent = () => {
    return <div></div>
}

describe('myComponent', () => {
    beforeEach(() => {
        //jest.useFakeTimers()
    })

    it('does not hit a timeout', async () => {
        const wrapper = mount(<MyComponent />)

        const result = await axe(wrapper.getDOMNode())

        expect(result).toHaveNoViolations()
    })
})

Version Details:
jest-axe: 4.1.0
jest: 26.6.3
axe-core: 4.1.1

I'm trying to test the accessibility of the document.

expect(await axe(document.documentElement.outerHtlm)).toHaveNoViolations() does not produce the expected violations, even if you pass is an obviously faulty string (e.g a document without title it isn't flagged, though it is mentioned in the axe-core docs)

Does jest-axe not support the full functionality of axe-core?

Hello, I'm not sure it's directly an issue on our side, but I will ask.

I'm using jest-axe to test Vue component (with vue-test-utils), and since the update of jest-axe v3.1.1, it return errors for duplicate id ("id attribute value must be unique (duplicate-id)") for some of my component. But after verification, the ID are not duplicated, it seem that these false errors are only throw when mounting the component with the option attachToDocument set to true.

Here are a simple example:

testId.vue

<template>
    <div>
        <p id="test-id">Test Id</p>
    </div>
</template>

testId.spec.js

import Vue from "vue";
import { shallowMount} from "@vue/test-utils";

import testId from "testId.vue";

// Use axe accessibility tools
// https://www.deque.com/axe/
// https://github.com/nickcolley/jest-axe
import { axe, toHaveNoViolations } from "jest-axe";
expect.extend(toHaveNoViolations);

describe("CardList.vue", () => {
    let wrapper;

    beforeEach(() => {
        wrapper = shallowMount(testId, {
            attachToDocument: true,
        });
    });

    describe('Mounted', () => {
        test("It should not contain any axe accessibility violation", async () => {
            const html = wrapper.html();

            expect(await axe(html)).toHaveNoViolations()
        });
    });
});

Return:

Expected the HTML found at $('div:nth-child(2) > div
                                                     
<p id="test-id">Test Id</p>                         
                                                     
Received:                                           
                                                    
"id attribute value must be unique (duplicate-id)"  

Any idea ? Thank you

We are using axe as described in the readme and other examples we've looked at online and are getting act warnings when using it.

Expected: Can run axe on react code and assert on violations without act warnings
Actual: act warnings output to the console when test is run

Example code:

describe('tests', () => {
  it('should have no Axe violations', async () => {
    const { container } = render(<MyComponent />);
    const results = await axe(container);
    expect(results).toHaveNoViolations();
  })
});

I'm having to wrap the last two lines in act:

await act(async () => {
  const results = await axe(container);
  expect(results).toHaveNoViolations();
})

In order not to get warnings. I'm not sure how axe is implemented under the hood, so I can't offer much advice, but I do know that react testing library wraps most or all of its actions in act calls under the hood - maybe jest-axe needs to do the same with whatever it's doing?

https://github.com/evcohen/eslint-plugin-jsx-a11y/blob/master/docs/rules/no-redundant-roles.md

my eslint set up points out that no-redundant-roles error, because it doesn't want me to do something like <form role="form></form>

jest-axe though will point out the following if I don't add role property:

    expect(received).toHaveNoViolations(expected)

    Expected the HTML found at $('form') to have no violations:

    <form>

    Received:

    "All page content must be contained by landmarks (region)"

is there a configuration for this? happy to help out if you need extra information or assistance.

Storybook's Storyshots add-on make it easy to create Jest snapshots for each storybook story automatically. That means, there are no individual test files, just one Jest test file with the initialization of Storyshots like:

Storyshots.test.js

import initStoryshots from '@storybook/addon-storyshots';

initStoryshots();

Is it possible to integrate jest-axe so that the accessibility tests are also ran automatically per each story?

I would like to disable the fallback role check.

Using axe directly, I can do this.

axe.configure({
  checks: [
    {
      id: 'fallbackrole'
      enabled: false
    }
  ]
});

However, I don't see a way to configure checks through jest-axe. Thanks

The following sandbox is producing an svg with a title as a child element to the svg however jest-axe is failing the test. This should pass as it meets the accessibility spec.

https://codesandbox.io/s/relaxed-satoshi-0p2jz

Given the limit set in package.json

"engines": {
    "node": ">= 8.0.0"
  },

and because my team uses Yarn, we cannot install jest-axe on node v6 environment - yarn is stricter than npm and completely refuses to install. While this limit is needed to run jest-axe tests, it should not be required for consumers of the library.

While examples provided in the README use some functionalities not available in node 6 (async), anyone can transpile their code and / or use babel to run their tests, therefore it seems that this limitation is not needed.

Is there any chance that the engines section be removed from package.json?
PS: I created PR #30 if this is acceptable. Thank you!

Hi!

At the moment, the matcher passes only when it detects no violations.

Other tools like pa11y and the next version of cypress-axe allow uses to set a tolerance threshold for the test.

What about adding such a threshold to this matcher as well? If it sounds good to you, I could submit a PR with the related changes.

Hi,

I noticed the minimum required Node version in package.json is 8.9.4. I'm wondering if we can lower this to > 8 since it's seem pretty limiting. Is there something that is present in 8.9.4 and above that is required by this library ?

sorry i do not know how to do a multiline code block

const wrapper = mount(<App/>)
const results = await axe(container.getDOMNode())

should be

const wrapper = mount(<App/>)
const results = await axe(wrapper.getDOMNode())

Also I had to change
const App = require('./app')

to

import App from '.App'

because mount was being passed an object instead of the component

First of all, thanks for doing such an amazing job with jest-axe 💓 💯 👏🏻

The issue

We're trying to use jest-axe to verify the output of Reakit components.

Here's the PR that captures the work: ariakit/ariakit#652

We basically configured axe to test components on isolation and now we're [trying to verify the HTML output of a component:

test("renders with no a11y violations", async () => {
  const { container } = render(<ButtonWithTooltip />);
  const results = await axe(container.innerHTML);

  expect(results).toHaveNoViolations();
  cleanup;
});

However, we're facing the following error:

["Error: Uncaught [NotFoundError: The node to be removed is not a child of this node.]
        at reportException (/Users/lluis/Code/reakit/node_modules/jsdom/lib/jsdom/living/helpers/runtime-script-errors.js:62:24)
        at innerInvokeEventListeners (/Users/lluis/Code/reakit/node_modules/jsdom/lib/jsdom/living/events/EventTarget-impl.js:346:9)
        at invokeEventListeners (/Users/lluis/Code/reakit/node_modules/jsdom/lib/jsdom/living/events/EventTarget-impl.js:281:3)

Many Reakit components make use of React Portals, and we noticed when we switch off portals ( through a prop ) then the jest-axe test pass.

So the question being is this library compatible with portals? 🤔

Had this problem recently:

  TypeError: (0 , _prettyFormat.format) is not a function

      48 |     const { container } = render(<Wizard />, { wrapper: MockProviders })
      49 |     const result = await axe(container)
    > 50 |     expect(result).toHaveNoViolations()
         |                    ^
      51 | })

Found out then that the problem was that the Wizard component (which has other nested components) when rendered formed an incorrect sequence of headings: an <h1> was rendered and then later on an <h3> without an <h2> in between.

That error message from axe wasn't very helpful, though. I remember having that issue in the past and getting a helpful error message, something like this.

Is this a bug?

UPDATE

Although this really seems to be the reason why I got that error, the accessibility test is still flaky. I've run the test 10 times for each of these scenarios:

• with the right heading element: FLAKY (passed 5 out of 10)
• commenting the heading element: ALWAYS PASS (passed 10)
• with the wrong heading element: ALWAYS FAILS (failed 10)

Can this be used with react-test-renderer? I'm assuming no but just double checking. Thanks

Violations currently may not have enough context, depending on the rule that's being violated.

Example:

<div id="dialog" aria-labelledby="foo" aria-describedby="desc">
  <h1 id="bar">Title</h1>
  <p id="desc">Description</p>
</div>

Returns:

Expected the HTML found at $('#dialog') to have no violations:

<div id="dialog" aria-labelledby="foo" aria-describedby="desc">
  <h1 id="bar">Title</h1>
  <p id="desc">Description</p>
</div>

Received:

"ARIA attributes must conform to valid values (aria-valid-attr-value)"

Try fixing it with this help: https://dequeuniversity.com/rules/axe/3.1/aria-valid-attr-value?application=axeAPI

If I'm not well versed on accessibility, I may not know exactly what to look for. Which attribute? What's invalid? However, we're in luck! For each node that has a violation, axe has a failure summary that is more specific on what to fix.

Fix all of the following:
Invalid ARIA attribute value: aria-labelledby="foo"

Including this failure summary as part of the violations reporter would provide more clear direction on what to investigate.

🆕🐥☝ First Timers Only

This issue is reserved for people who never contributed to Open Source before. We know that the process of creating a pull request is the biggest barrier for new contributors. This issue is for you 💝

About First Timers Only.

🤔 What you will need to know

Nothing. This issue is meant to welcome you to Open Source :) We are happy to walk you through the process.

📋 Step by Step

• 🙋 Claim this issue: Comment below to let other people know that you're working on this issue.
• 🔀 Fork our repository: You'll need to fork this repository in order to propose a change.
• 📝 Update the file README.md in the your-username-here/jest-axe repository and edit the line as shown below.

@@ -7,6 +7,15 @@
 
 Custom [Jest](https://jest-bot.github.io/jest/) matcher for [aXe](https://github.com/dequelabs/axe-core) for testing accessibility
 
+## ⚠️✋ This project does not guarantee what you build is accessible.
+The GDS Accessibility team found that only [~30% of issues are found by automated testing](https://accessibility.blog.gov.uk/2017/02/24/what-we-found-when-we-tested-tools-on-the-worlds-least-accessible-webpage).
+
+Tools like aXe are similar to [code linters](https://en.wikipedia.org/wiki/Lint_%28software%29) such as [eslint](https://eslint.org/) or [sass-lint](https://github.com/sasstools/sass-lint), they can find common issues but can not guarantee what you build works for users.
+
+You'll also need to:
+- test your interface with the [assistive technologies that real users use](https://www.gov.uk/service-manual/technology/testing-with-assistive-technologies#when-to-test) (see also [WebAIM's survey results](https://webaim.org/projects/screenreadersurvey7/#primary)).
+- include people with disabilities in user research.
+
 ## Installation:
 ```bash
 npm install --save-dev jest-axe

• 🔀 Start a Pull Request. You can create a pull request in Github. Mention closes nickcolley/jest-axe#24 in the description.
• 🏁 Done 👍 We'll review the Pull Request and let you know if any changes are needed.

🤔❓ Questions

Comment on this issue or message Nick on Twitter.

There was a recent advisory posted for lodash and a couple of its sub-packages. Since the dependencies in this package are pegged to specific versions, npm audit fix wasn't able to fix the issue automatically.

Pull request incoming.

Since this matcher appends the rendered content to the document (i.e., document.body.innerHTML = html), I noticed that other tests that render to the document via react-testing-librarycause there to be two instances of the component in the tree.

To avoid this, I added the following:

afterEach(() => {
  document.body.innerHTML = ''
})

However, it seems like this would best be handled by jest-axe itself. I can create a PR if needed.

Master is failing, will need to sort this for the next release. (Which is breaking).

Travis doesnt seem to be running tests on new pull requests either.

https://travis-ci.org/github/nickcolley/jest-axe

Error trying to use jest-axe in a .test.js file using flow (// @flow)

This is the output:

$ xxx/xxx/xxx/node_modules/.bin/flow
Error ┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈ src/components/organisms/BlogCategoryLayout/BlogCategoryLayout.test.js:32:27

Cannot call expect(...).toHaveNoViolations because:
 • Either property toHaveNoViolations is missing in JestExpectType [1].
 • Or property toHaveNoViolations is missing in JestPromiseType [2].
 • Or property toHaveNoViolations is missing in EnzymeMatchersType [3].

           src/components/organisms/BlogCategoryLayout/BlogCategoryLayout.test.js
            29│                 const html = ReactDOMServer.renderToString(
            30│                         <BlogCategoryLayout title="Test Title" standardPosts={standardPosts} />
            31│                 )
            32│                 expect(await axe(html)).toHaveNoViolations()
            33│         })
            34│ })
            35│

           flow-typed/npm/jest_v22.x.x.js
 [1][2][3] 512│         (value: any): JestExpectType & JestPromiseType & EnzymeMatchersType,

I am getting the following error for the following test with jest-axe:

Am I missing something?

Error: Not implemented: window.computedStyle(elt, pseudoElt)

test/setup.ts:

import '@testing-library/jest-dom/extend-expect'
import 'jest-axe/extend-expect'

file.spec.tsx:

const a11y = await axe(container)
expect(a11y).toHaveNoViolations()

jest.config.js

{
  displayName: 'web:test',
  preset: 'ts-jest',
  testEnvironment: 'jsdom',
  testMatch: ['**/__tests__/**/+(*.)+(spec|test).+(js|ts)?(x)'],
  moduleNameMapper: {
    '^~/(.*)$': '<rootDir>/src/$1',
    '^test/(.*)$': '<rootDir>/test/$1'
  },
  setupFilesAfterEnv: ['<rootDir>/test/setup.ts'],
  globals: {
    'ts-jest': {
      tsconfig: {
        jsx: 'react'
      }
    }
  }
}

deps:

"jest": "^26.6.3",
"jest-axe": "^4.1.0",
"react-test-renderer": "^17.0.1",
"ts-jest": "^26.4.4",
"typescript": "^4.1.2"

Hi,

I'd like to use toHaveNoViolations in a project to extend expect.

Is there a way to do this without requiring async/await for every render of my react components? I'd love for it to be transparent to the users of the extended expect.

Thanks for the lib, I love it!

I would like to demonstrate the package to my teammates, but I need a convincing example. The one on the doc using <img> tag with an alt attribute is already catched by eslint-plugin-jsx-a11y in the IDE.

Could you provide a React/HTML snippet that would trigger jest-axe but not jsx-a11y?

Could make the terminal colours better for the screenshot too...