Skip to content

Commit 76a8ac7

Browse files
mchamwmcode
authored andcommitted
feat(all): brought back skipFailures, fixed violations asserstion
skipFailures to enable only logging errors, updated readme
1 parent 76adb3a commit 76a8ac7

File tree

11 files changed

+111
-176
lines changed

11 files changed

+111
-176
lines changed

.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,5 @@ node_modules
33
dist
44
.vscode
55
.eslintcache
6+
.DS_Store
7+
*.log

README.md

Lines changed: 91 additions & 161 deletions
Original file line numberDiff line numberDiff line change
@@ -8,30 +8,67 @@ Test accessibility with [axe-core](https://github.com/dequelabs/axe-core) in [Cy
88
>
99
> **Reasons**: to upgrade dependencies (i.e. `Cypress ^7` & `axe-core ^4`) and try out some of the suggesions in [RFC 75](https://github.com/component-driven/cypress-axe/issues/75) 👀
1010
11-
## Installation
1211

13-
1. **Install `cypress-axe-core` from npm:**
12+
1. [Installation and Setup](#Installation-and-Setup)
13+
- [Typescript](###TypeScript)
14+
2. [Examples](#Examples)
15+
- [Simple](##Simple)
16+
- [Customised](##Customised)
17+
3. [Commands](#Commands)
18+
- [cy.injectAxe](##cy.injectAxe)
19+
- [cy.checkA11y](##cy.checkA11y)
20+
- [cy.configureCypressAxe](##cy.configureCypressAxe)
21+
- [cy.configureAxe](##cy.configureAxe)
1422

23+
24+
# Installation and Setup
25+
1. **Install** required packages
1526
```sh
1627
npm install --save-dev cypress-axe-core
1728
```
18-
19-
2. **Install peer dependencies:**
20-
29+
As its name implies, this package has two peer dependencies: [cypress](https://github.com/cypress-io) and [axe-core](https://github.com/dequelabs/axe-core)
2130
```sh
2231
npm install --save-dev cypress axe-core
2332
```
2433

25-
3. **Include the commands.** Update `cypress/support/index.js` file to include the cypress-axe commands by adding:
34+
2. **Include the commands.** Update `cypress/support/index.js` file to include the cypress-axe commands by adding:
2635

2736
```js
2837
import 'cypress-axe-core'
2938
```
3039

31-
4. **Add a task to log the messages to the terminal** when Cypress executes the spec files. [Example - configuring log task](https://docs.cypress.io/api/commands/task.html#Usage).
40+
3. **Enable logging results to terminal** by defining _logging_ tasks in cypress `plugins/index.js`.
41+
```js
42+
// cypress/plugins/index.js
43+
module.exports = (on, config) => {
44+
on('task', {
45+
log(message) {
46+
console.log(message);
47+
return null;
48+
},
49+
50+
table(message) {
51+
console.table(message);
52+
return null;
53+
},
54+
});
55+
};
56+
```
57+
> **NOTE**: You can control how results are displayed via [the `violationsCallback` config option](##cy.configureCypressAxe)
58+
59+
After following the steps above (_and defining cy.tasks_), violations will be displayed as follows:
60+
- Cypress output
61+
![Default Cypress output](cypressOutputSample.png)
62+
63+
- Terminal
64+
![Default terminal output](terminalOutputSample.png)
65+
66+
- Browser console
67+
![Default browser console output](browserOutputSample.png)
68+
3269

33-
### TypeScript
3470

71+
### TypeScript
3572
If you’re using TypeScript, add cypress-axe-core types to your Cypress’ `tsconfig.json` file:
3673

3774
```json
@@ -46,24 +83,29 @@ If you’re using TypeScript, add cypress-axe-core types to your Cypress’ `tsc
4683
}
4784
```
4885

49-
## Usage
86+
# Examples
5087

51-
### Simple check
88+
## Simple
5289

5390
```js
5491
it('passes axe', () => {
5592
cy.visit('/')
5693
cy.injectAxe()
5794
// ...
58-
cy.get('button').checkA11y() // check certain elements only
95+
cy.get('.my-button').checkA11y() // target certain elements
5996
// OR
6097
cy.checkA11y() // check the whole document
6198
})
6299
```
63100

64-
### Customized check
101+
## Customised
102+
103+
Leveraging ([Cypress commands](https://docs.cypress.io/api/cypress-api/custom-commands)) You can either:
104+
105+
- Overwrite `cy.checkA11y` _or_
106+
- Wrap it in your own custom command
65107

66-
You can provide `shouldFail` function to decide which rules should fail. For example, only assert against _serious_ & _critical_ violations but ignore _color-contrast_ rule
108+
then pass it a `shouldFailFn` function to decide which rules should fail. For example, only assert against _serious_ & _critical_ violations but ignore _color-contrast_ rule.
67109

68110
```js
69111
// cypress/support/commands.js
@@ -73,7 +115,7 @@ Cypress.Commands.add(
73115
(subject, options, label) => {
74116
return cy.checkA11y(
75117
{
76-
shouldFail: violations =>
118+
shouldFailFn: violations =>
77119
violations.filter(
78120
v =>
79121
v.id !== 'color-contrast' &&
@@ -95,9 +137,9 @@ it('passes custom axe tests', () => {
95137
})
96138
```
97139

98-
## Commands
140+
# Commands
99141

100-
### cy.injectAxe
142+
## cy.injectAxe
101143

102144
This will inject the `axe-core` runtime into the page under test. You must run this **after** a call to `cy.visit()` and before you run the `checkA11y` command.
103145

@@ -110,9 +152,40 @@ beforeEach(() => {
110152
})
111153
```
112154

113-
### cy.configureAxe
155+
## cy.checkA11y
114156

115-
#### Purpose
157+
When not chained to another element, it will run against the whole document. You can have it at the end of your test (after other interaction assertions) so it checks against all possible violations. It accepts the same (optional) config object that [`cy.configureCypressAxe`](##cy.configureCypressAxe) accepts
158+
159+
**Note**: if you have a toggle-able element i.e. a side menu, make sure it's on (shown) by the time `cy.checkA11y` is called, otherwise you might end up with some false-positive cases. Or, you can target those elements directly to make sure they're tested
160+
```js
161+
cy.get('#menu-button').click();
162+
cy.get('#side-menu-container').checkA11y()
163+
```
164+
165+
## cy.configureCypressAxe
166+
167+
Instead of wrapping or overwriting `cy.checkA11y`, you can configure it. It accepts the following:
168+
- `axeOptions` passed to axe-core.
169+
- `shouldFailFn` function that returns array of violations to check for.
170+
- `skipFailures` if true, it will log the violations but not assert against them.
171+
- `violationsCallback` reporter function that receives the result.
172+
173+
**The default** `violationsCallback` function assumes that `cy.task('log')` and `cy.task('table')` have been defined already during the [Installation & setup](##Installation-and-Setup). If you don't want to define those tasks, you can pass a function here to control how results are outputted.
174+
175+
```js
176+
cy.configureCypressAxe({
177+
axeOptions: [], // axe.RunOptions[]
178+
shouldFailFn: violations => violations,
179+
skipFailures: false,
180+
violationsCallback: ({
181+
filename: 'test.spec.ts', // spec filename
182+
results: [], // violations axe.Result[]
183+
label: 'my custom component', // if passed to checkA11y
184+
}) => void,
185+
})
186+
```
187+
188+
## cy.configureAxe
116189

117190
To configure the format of the data used by aXe. This can be used to add new rules, which must be registered with the library to execute.
118191

@@ -139,149 +212,6 @@ it('Has no detectable a11y violations on load (custom configuration)', () => {
139212
})
140213
```
141214

142-
### cy.checkA11y
143-
144-
This will run axe against the document at the point in which it is called. This means you can call this after interacting with your page and uncover accessibility issues introduced as a result of rendering in response to user actions.
145-
146-
#### Parameters on cy.checkA11y (axe.run)
147-
148-
##### context (optional)
149-
150-
Defines the scope of the analysis - the part of the DOM that you would like to analyze. This will typically be the document or a specific selector such as class name, ID, selector, etc.
151-
152-
##### options (optional)
153-
154-
Set of options passed into rules or checks, temporarily modifying them. This contrasts with axe.configure, which is more permanent.
155-
156-
The keys consist of [those accepted by `axe.run`'s options argument](https://www.deque.com/axe/documentation/api-documentation/#parameters-axerun) as well as a custom `includedImpacts` key.
157-
158-
The `includedImpacts` key is an array of strings that map to `impact` levels in violations. Specifying this array will only include violations where the impact matches one of the included values. Possible impact values are "minor", "moderate", "serious", or "critical".
159-
160-
Filtering based on impact in combination with the `skipFailures` argument allows you to introduce `cypress-axe` into tests for a legacy application without failing in CI before you have an opportunity to address accessibility issues. Ideally, you would steadily move towards stricter testing as you address issues.
161-
162-
##### violationsCallback (optional)
163-
164-
Allows you to define a callback that receives the violations for custom side-effects, such as adding custom output to the terminal.
165-
166-
**NOTE:** _This respects the `includedImpacts` filter and will only execute with violations that are included._
167-
168-
##### skipFailures (optional, defaults to false)
169-
170-
Disables assertions based on violations and only logs violations to the console output. This enabled you to see violations while allowing your tests to pass. This should be used as a temporary measure while you address accessibility violations.
171-
172-
Reference : https://github.com/component-driven/cypress-axe/issues/17
173-
174-
### Examples
175-
176-
#### Basic usage
177-
178-
```js
179-
// Basic usage
180-
it('Has no detectable a11y violations on load', () => {
181-
// Test the page at initial load
182-
cy.checkA11y()
183-
})
184-
185-
// Applying a context and run parameters
186-
it('Has no detectable a11y violations on load (with custom parameters)', () => {
187-
// Test the page at initial load (with context and options)
188-
cy.checkA11y('.example-class', {
189-
runOnly: {
190-
type: 'tag',
191-
values: ['wcag2a']
192-
}
193-
})
194-
})
195-
196-
it('Has no detectable a11y violations on load (filtering to only include critical impact violations)', () => {
197-
// Test on initial load, only report and assert for critical impact items
198-
cy.checkA11y(null, {
199-
includedImpacts: ['critical']
200-
})
201-
})
202-
203-
// Basic usage after interacting with the page
204-
it('Has no a11y violations after button click', () => {
205-
// Interact with the page, then check for a11y issues
206-
cy.get('button').click()
207-
cy.checkA11y()
208-
})
209-
210-
it('Only logs a11y violations while allowing the test to pass', () => {
211-
// Do not fail the test when there are accessibility failures
212-
cy.checkA11y(null, null, null, { skipFailures: true })
213-
})
214-
```
215-
216-
#### Using the violationCallback argument
217-
218-
The violation callback parameter accepts a function and allows you to add custom behavior when violations are found.
219-
220-
This example adds custom logging to the terminal running Cypress, using `cy.task` and the `violationCallback` argument for `cy.checkA11y`
221-
222-
##### In Cypress plugins file
223-
224-
This registers a `log` task as seen in the [Cypress docs for cy.task](https://docs.cypress.io/api/commands/task.html#Usage) as well as a `table` task for sending tabular data to the terminal.
225-
226-
```js
227-
module.exports = (on, config) => {
228-
on('task', {
229-
log(message) {
230-
console.log(message)
231-
232-
return null
233-
},
234-
table(message) {
235-
console.table(message)
236-
237-
return null
238-
}
239-
})
240-
}
241-
```
242-
243-
#### In your spec file
244-
245-
Then we create a function that uses our tasks and pass it as the `validationCallback` argument to `cy.checkA11y`
246-
247-
```js
248-
// Define at the top of the spec file or just import it
249-
function terminalLog(violations) {
250-
cy.task(
251-
'log',
252-
`${violations.length} accessibility violation${
253-
violations.length === 1 ? '' : 's'
254-
} ${violations.length === 1 ? 'was' : 'were'} detected`
255-
)
256-
// pluck specific keys to keep the table readable
257-
const violationData = violations.map(
258-
({ id, impact, description, nodes }) => ({
259-
id,
260-
impact,
261-
description,
262-
nodes: nodes.length
263-
})
264-
)
265-
266-
cy.task('table', violationData)
267-
}
268-
269-
// Then in your test...
270-
it('Logs violations to the terminal', () => {
271-
cy.checkA11y(null, null, terminalLog)
272-
})
273-
```
274-
275-
This custom logging behavior results in terminal output like this:
276-
277-
![Custom terminal logging with cy.task and validationCallback](terminal_output_example.png)
278-
279-
## Standard Output
280-
281-
When accessibility violations are detected, your test will fail and an entry titled "A11Y ERROR!" will be added to the command log for each type of violation found (they will be above the failed assertion). Clicking on those will reveal more specifics about the error in the DevTools console.
282-
283-
![Cypress and DevTools output for passing and failing axe-core audits](cmd_log.png)
284-
285215
## Authors
286216

287217
The project was created by [Andy Van Slaars](https://vanslaars.io/), and maintained by [Artem Sapegin](https://sapegin.me).

browserOutputSample.png

74.2 KB
Loading

cmd_log.png

-301 KB
Binary file not shown.

cypressOutputSample.png

36.3 KB
Loading

src/index.ts

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,9 @@ import type { CypressAxeOptions } from './types';
88

99
let defaultCypressAxeConfig = {
1010
axeOptions: {},
11-
shouldFail: (violations: axe.Result[]) => violations,
11+
shouldFailFn: (violations: axe.Result[]) => violations,
1212
violationsCb: consoleReporter,
13+
skipFailures: false,
1314
};
1415

1516
export const injectAxe = () => {
@@ -40,7 +41,7 @@ const checkA11y = (params: {
4041
label?: string;
4142
}) => {
4243
const { context, label } = params;
43-
const { axeOptions, shouldFail, violationsCb } = {
44+
const { axeOptions, shouldFailFn, violationsCb, skipFailures } = {
4445
...defaultCypressAxeConfig,
4546
...params.options,
4647
};
@@ -53,7 +54,7 @@ const checkA11y = (params: {
5354
.then(({ violations }) => violations);
5455
})
5556
.then((violations) => cy.wrap(violations, { log: false }))
56-
.then((violations) => shouldFail(violations))
57+
.then((violations) => shouldFailFn(violations))
5758
.then((failableViolations) => {
5859
if (failableViolations.length) {
5960
violationsCb({
@@ -62,9 +63,13 @@ const checkA11y = (params: {
6263
label,
6364
});
6465
}
65-
return new Promise(resolve => resolve(failableViolations));
66+
return cy.wrap(failableViolations, { log: false })
6667
})
67-
.then((failableViolations) => assertViolations(failableViolations as axe.Result[]));
68+
.then((failableViolations) => {
69+
if (!skipFailures) {
70+
assertViolations(failableViolations as axe.Result[])
71+
}
72+
});
6873
};
6974

7075
Cypress.Commands.add('injectAxe', injectAxe);

src/types.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,9 @@ declare global {
2222

2323
export interface CypressAxeOptions {
2424
axeOptions?: axe.RunOptions;
25-
shouldFail?: (violations: axe.Result[]) => axe.Result[];
25+
shouldFailFn?: (violations: axe.Result[]) => axe.Result[];
2626
violationsCallback?(): typeof consoleReporter;
27+
skipFailures?: boolean;
2728
}
2829

2930
export interface RunResults {

0 commit comments

Comments
 (0)