Back to home

Running axe-core with a Playwright script

Published: | Author: Russ Weakley

What this tutorial is about

In Running axe-core from the command line, you ran axe-core with a single command. However, the command line tool can't filter by severity, and it only checks a page when you tell it to.

In this tutorial, you'll write your own script, using Playwright to open a page and run axe-core. You'll have control over what gets tested, what gets shown, and when it runs. By the end, you'll have a check that shows only critical and serious issues, passes or fails, and runs again every time the page changes.

About Playwright

Playwright is a free tool that controls a web browser with code. It can open pages, click buttons and fill in forms, just like a person would. The @axe-core/playwright add-on lets you run axe-core on any page Playwright opens.

Of the three tutorials, this one is closest to how development teams use axe-core in practice. It can become part of their automated testing process, helping them find accessibility issues as they work and prevent new code from being released when tests fail.

Before you start

Complete Setting up Node.js first. You should have:

You won't need ChromeDriver in this tutorial, because Playwright uses its own browser.

You'll type the commands in Terminal (Mac) or Command Prompt (Windows).

How this tutorial works

You'll build one script, check.js, a little at a time. Each time you change it, you'll replace everything in the file with a new version, so you always start from a complete, working file. The new parts are marked with comments in CAPITALS, so you can see what's changed.

What's in this tutorial

Getting started

Building and running the check

Running the check automatically

Fixing the page

Optional extras

Getting started

Step 1. Open your project folder

Open Terminal (Mac) or Command Prompt (Windows), then move into the axe-learning folder you made in Setting up Node.js.

Mac:

cd Documents/axe-learning

Windows:

cd %USERPROFILE%\axe-learning

In plain terms: cd stands for "change directory", which means "go into this folder".

Watch out for:

Step 2. Add the test page

This is the same test page used in the earlier tutorials, with 7 deliberately planted accessibility errors. This time, you need your own copy, in your axe-learning folder. That's because later on, you'll fix the errors yourself and watch the results change.

Create an empty file, then open it:

Mac:

touch test-page.html
code test-page.html

Windows:

type nul > test-page.html
notepad test-page.html

Paste in the following code, then save the file (Command + S on Mac, Ctrl + S on Windows):

<!DOCTYPE html>
<!-- ERROR 1: no lang attribute on the html element -->
<html>
<head>
  <meta charset="utf-8">
  <title>axe test page</title>
  <style>
    body {
      margin: 0;
      padding: 2em;
      font-family: arial, helvetica, sans-serif;
    }

    button {
      padding: .5em 1em;
      border-radius: .3em;
      border: 1px solid red;
    }

    a:link {
      padding: .5em 1em;
      border-radius: .3em;
      border: 1px solid red;
    }

    .faint { color: #aaa; }
  </style>
</head>
<body>
  <main>
    <h1>axe test page</h1>

    <!-- ERROR 2: heading level skips from h1 to h3 -->
    <h3>A section</h3>

    <!-- ERROR 3: image with no alt attribute -->
    <p><img src="https://placehold.co/200x200"></p>


    <!-- ERROR 4: text input with no label -->
    <p><input type="text"></p>

    <!-- ERROR 5: button with no accessible name -->
    <p><button></button></p>

    <!-- ERROR 6: link with no text -->
    <p><a href="#"></a></p>

    <!-- ERROR 7: low contrast text (grey on white) -->
    <p class="faint">This text is hard to read.</p>
  </main>
</body>
</html>

To check it worked, list the files in your folder (ls on Mac, dir on Windows). You should see test-page.html.

Watch out for:

Step 3. Create the check script

Create a script that uses Playwright to open the test page, and the axe add-on to run axe-core on it. To start with, it just counts how many rules failed.

Mac:

touch check.js
code check.js

Windows:

type nul > check.js
notepad check.js

Paste in the following code, then save the file:

// Load the tools
const { chromium } = require('playwright');
const AxeBuilder = require('@axe-core/playwright').default;
const path = require('path');
const { pathToFileURL } = require('url');

(async () => {
  // Open an invisible browser, a fresh private window, and a new tab
  const browser = await chromium.launch();
  const context = await browser.newContext();
  const page = await context.newPage();

  // Load the test page from this folder
  await page.goto(pathToFileURL(path.join(__dirname, 'test-page.html')).href);

  // Inject axe, run it, and store the results
  const results = await new AxeBuilder({ page }).analyze();

  // Print how many rules failed
  console.log('Violations found:', results.violations.length);

  // Close the browser
  await browser.close();
})();

In plain terms, the script follows the same steps you used in the browser console:

  1. Open a browser and a tab.
  2. Load the test page.
  3. Inject axe-core, run it, and store the results.
  4. Print how many rules failed.
  5. Close the browser.

Don't worry about the (async () => { ... })(); wrapper. It's just what lets the script wait for each step to finish before moving on to the next.

Watch out for:

Step 4. Run the check script

Type:

node check.js

After a few seconds, you should see:

Violations found: 7

Watch out for:

Building and running the check

Step 5. Show only critical and serious

Change the script so it only shows critical and serious violations, with the details of each one. The command line tool can't do this, which is one of the main reasons to write your own script.

Open check.js in your editor. Select everything (Command + A on Mac, Ctrl + A on Windows), delete it, paste in this new version, then save the file:

// Load the tools
const { chromium } = require('playwright');
const AxeBuilder = require('@axe-core/playwright').default;
const path = require('path');
const { pathToFileURL } = require('url');

(async () => {
  // Open an invisible browser, a fresh private window, and a new tab
  const browser = await chromium.launch();
  const context = await browser.newContext();
  const page = await context.newPage();

  // Load the test page from this folder
  await page.goto(pathToFileURL(path.join(__dirname, 'test-page.html')).href);

  // Inject axe, run it, and store the results
  const results = await new AxeBuilder({ page }).analyze();

  // KEEP ONLY CRITICAL AND SERIOUS
  const important = results.violations.filter(
    v => v.impact === 'critical' || v.impact === 'serious'
  );

  // PRINT THE RESULTS
  console.log(`Critical and serious issues: ${important.length}`);
  important.forEach(v => {
    console.log(`\n[${v.impact}] ${v.id}: ${v.help}`);
    v.nodes.forEach(n => console.log(`  - ${n.html}`));
  });

  // Close the browser
  await browser.close();
})();

What's new:

Now run it:

node check.js

You should see Critical and serious issues: 6, followed by the details of each violation. The heading-order violation is no longer included because its severity is moderate.

Step 6. Pass or fail

In Make accessibility tests fail when issues are found, you used --exit to make the command line tool report a failure through its exit code. 0 means success, and anything else means failure. Development teams use this to stop a build or block a release when accessibility tests fail.

The catch was that --exit fails on any violation, even a minor one.

Writing your own script means you decide what counts as a failure. You can choose which issues should make a test fail, rather than treating every violation the same way.

In this step, you'll change the script so it only fails on critical and serious issues, displays a clear PASS or FAIL message, and sets the exit code to match.

Open check.js in your editor. Select everything (Command + A on Mac, Ctrl + A on Windows), delete it, paste in this new version, then save the file:

// Load the tools
const { chromium } = require('playwright');
const AxeBuilder = require('@axe-core/playwright').default;
const path = require('path');
const { pathToFileURL } = require('url');

(async () => {
  // Open an invisible browser, a fresh private window, and a new tab
  const browser = await chromium.launch();
  const context = await browser.newContext();
  const page = await context.newPage();

  // Load the test page from this folder
  await page.goto(pathToFileURL(path.join(__dirname, 'test-page.html')).href);

  // Inject axe, run it, and store the results
  const results = await new AxeBuilder({ page }).analyze();

  // Keep only critical and serious
  const important = results.violations.filter(
    v => v.impact === 'critical' || v.impact === 'serious'
  );

  // Print the results
  console.log(`Critical and serious issues: ${important.length}`);
  important.forEach(v => {
    console.log(`\n[${v.impact}] ${v.id}: ${v.help}`);
    v.nodes.forEach(n => console.log(`  - ${n.html}`));
  });

  // Close the browser
  await browser.close();

  // PASS OR FAIL
  if (important.length > 0) {
    console.log('\nFAIL: critical or serious issues found');
    process.exitCode = 1;
  } else {
    console.log('\nPASS: no critical or serious issues');
  }
})();

What's new:

Now run it:

node check.js

You should see the 6 issues, then FAIL: critical or serious issues found.

Step 7. Check the exit code

Your script now reports PASS or FAIL, but it also sets an exit code that other tools can use to tell whether the check succeeded or failed.

Run the script again:

node check.js

Then check the exit code:

Mac:

echo $?

Windows:

echo %errorlevel%

You should see 1, meaning the check failed. An exit code of 0 would mean the check passed.

Watch out for:

Running the check automatically

So far, you've run the check yourself, each time you wanted to test the page. Development teams don't usually work that way. Their checks run automatically whenever the code changes, often as part of a CI (continuous integration) pipeline.

In this section, you'll set up something similar on your own computer. The check will watch the test page, and run again every time you save a change.

Accessibility checks work best when nobody has to remember to run them. Running them automatically means issues are caught as soon as they're introduced.

Step 8. Run the check whenever the page changes

Node has a built-in watch mode. It runs your script, watches a file for changes, and runs the script again whenever that file is saved. This is similar to a developer working on a page, with checks running automatically as they make changes.

Type:

node --watch-path=test-page.html check.js

In plain terms: "run check.js, then watch test-page.html, and run it again whenever that file changes."

You'll see the same 6 issues and FAIL as before. This time you won't get your prompt back, because watch mode keeps running, waiting for changes.

To test it:

  1. Leave watch mode running.
  2. Open test-page.html in your editor.
  3. Change <html> to <html lang="en">, then save the file.
  4. Look back at the command line window.

The script should run again by itself, and now say Critical and serious issues: 5, with html-has-lang gone. Now undo the change and save again. The count should go back to 6.

To stop watch mode, press Control + C on Mac, or Ctrl + C on Windows.

Watch out for:

Step 9. Add check and watch shortcuts

Add two shortcuts to package.json so you don't have to type the full commands each time.

Open package.json in your editor:

Mac:

code package.json

Windows:

notepad package.json

Find the "scripts" part. If you haven't changed it, it looks like this:

  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },

Replace it with this, then save the file. It's the same on Mac and Windows:

  "scripts": {
    "check": "node check.js",
    "watch": "node --watch-path=test-page.html check.js"
  },

Now try the first shortcut:

npm run check

This runs the check once and stops, ending with FAIL. Then try the second one:

npm run watch

This starts watch mode, just like step 7. Press Control + C (Mac) or Ctrl + C (Windows) to stop it.

Watch out for:

Fixing the page

Step 10. Save a broken copy of the test page

You're about to fix the test page. Save a copy with all its errors first, so you can put them back later, such as before a demo.

Mac:

cp test-page.html test-page-broken.html

Windows:

copy test-page.html test-page-broken.html

To check it worked, list the files in your folder (ls on Mac, dir on Windows). You should see test-page-broken.html.

Step 11. Fix the page and reach PASS

Fix the six critical and serious problems one at a time, and watch the count drop until the check passes.

Start watch mode:

npm run watch

Open test-page.html in your editor. Make each of these changes, saving after each one, and watch the count go down in the command line window:

Problem Change this To this
No page language<html><html lang="en">
Image with no alt text<img src="https://placehold.co/200x200"><img src="https://placehold.co/200x200" alt="">
Input with no label<input type="text"><label>Name <input type="text"></label>
Button with no name<button></button><button>Send</button>
Link with no text<a href="#"></a><a href="#">Home</a>
Low contrast text.faint { color: #aaa; }.faint { color: #595959; }

After the last change, you should see Critical and serious issues: 0 and PASS: no critical or serious issues.

Press Control + C (Mac) or Ctrl + C (Windows) to stop watch mode.

Watch out for:

Step 12. Reset the test page

Put the broken version back, so the page has all its errors again.

Mac:

cp test-page-broken.html test-page.html

Windows:

copy /Y test-page-broken.html test-page.html

To check it worked, run npm run check. You should see 6 issues and FAIL again.

Watch out for:

Optional extras

These extras build on the script you created in Step 6. Each adds a new feature, so work through them in order.

Sort the most severe issues to the top

axe-core lists issues alphabetically, so the critical and serious ones are mixed together. Change the script so the critical ones come first.

Open check.js in your editor. Select everything (Command + A on Mac, Ctrl + A on Windows), delete it, paste in this new version, then save the file:

// Load the tools
const { chromium } = require('playwright');
const AxeBuilder = require('@axe-core/playwright').default;
const path = require('path');
const { pathToFileURL } = require('url');

(async () => {
  // Open an invisible browser, a fresh private window, and a new tab
  const browser = await chromium.launch();
  const context = await browser.newContext();
  const page = await context.newPage();

  // Load the test page from this folder
  await page.goto(pathToFileURL(path.join(__dirname, 'test-page.html')).href);

  // Inject axe, run it, and store the results
  const results = await new AxeBuilder({ page }).analyze();

  // Keep only critical and serious
  const important = results.violations.filter(
    v => v.impact === 'critical' || v.impact === 'serious'
  );

  // SORT CRITICAL ISSUES TO THE TOP
  const order = { critical: 0, serious: 1 };
  important.sort((a, b) => order[a.impact] - order[b.impact]);

  // Print the results
  console.log(`Critical and serious issues: ${important.length}`);
  important.forEach(v => {
    console.log(`\n[${v.impact}] ${v.id}: ${v.help}`);
    v.nodes.forEach(n => console.log(`  - ${n.html}`));
  });

  // Close the browser
  await browser.close();

  // Pass or fail
  if (important.length > 0) {
    console.log('\nFAIL: critical or serious issues found');
    process.exitCode = 1;
  } else {
    console.log('\nPASS: no critical or serious issues');
  }
})();

What's new:

The sorting code sits between the filter and the printing. This means the script first selects the critical and serious issues, then sorts them by severity before displaying the results.

Now run it:

npm run check

You should see the three critical issues (button-name, image-alt, label) first, then the three serious ones.

Write a simple HTML report

Change the script so it also creates a simple HTML report called report.html, listing the critical and serious issues found.

Open check.js in your editor. Select everything (Command + A on Mac, Ctrl + A on Windows), delete it, paste in this new version, then save the file:

// Load the tools
const { chromium } = require('playwright');
const AxeBuilder = require('@axe-core/playwright').default;
const path = require('path');
const { pathToFileURL } = require('url');
const fs = require('fs'); // NEW: Node's built-in file tools

(async () => {
  // Open an invisible browser, a fresh private window, and a new tab
  const browser = await chromium.launch();
  const context = await browser.newContext();
  const page = await context.newPage();

  // Load the test page from this folder
  await page.goto(pathToFileURL(path.join(__dirname, 'test-page.html')).href);

  // Inject axe, run it, and store the results
  const results = await new AxeBuilder({ page }).analyze();

  // Keep only critical and serious
  const important = results.violations.filter(
    v => v.impact === 'critical' || v.impact === 'serious'
  );

  // Sort critical issues to the top
  const order = { critical: 0, serious: 1 };
  important.sort((a, b) => order[a.impact] - order[b.impact]);

  // Print the results
  console.log(`Critical and serious issues: ${important.length}`);
  important.forEach(v => {
    console.log(`\n[${v.impact}] ${v.id}: ${v.help}`);
    v.nodes.forEach(n => console.log(`  - ${n.html}`));
  });

  // WRITE A SIMPLE HTML REPORT
  const escape = s => String(s ?? '')
    .replace(/&/g, '&amp;')
    .replace(/</g, '&lt;')
    .replace(/>/g, '&gt;');

  const items = important.map(v => `
    <li>
      <h2>[${v.impact}] ${escape(v.id)}</h2>
      <p>${escape(v.help)}</p>
      <ul>${v.nodes.map(n => `<li><code>${escape(n.html)}</code></li>`).join('')}</ul>
      <p><a href="${v.helpUrl}">More about ${escape(v.id)}</a></p>
    </li>`).join('');

  const report = `<!DOCTYPE html>
<html lang="en">
<head><meta charset="utf-8"><title>axe report</title></head>
<body>
<main>
<h1>axe report: ${important.length} critical and serious issues</h1>
<ol>${items}</ol>
</main>
</body>
</html>`;

  fs.writeFileSync(path.join(__dirname, 'report.html'), report);
  console.log('\nReport saved to report.html');

  // Close the browser
  await browser.close();

  // Pass or fail
  if (important.length > 0) {
    console.log('\nFAIL: critical or serious issues found');
    process.exitCode = 1;
  } else {
    console.log('\nPASS: no critical or serious issues');
  }
})();

What's new:

Now run the check, then open the report. Type each line separately, pressing Enter after each one:

Mac:

npm run check
open -a "Google Chrome" report.html

Windows:

npm run check
start chrome "%cd%\report.html"

You should see Report saved to report.html near the end of the output. The report lists the 6 issues, critical ones first.

Watch out for:

Next steps

In real projects, developers usually put checks like this inside Playwright's own test runner, alongside their other tests. These checks can then run automatically as developers work, or when new code is submitted for review.

axe-core can only find some accessibility issues. Playwright can also test things axe-core can't, such as keyboard access, focus and changes to a page as people use it. To see how, read Accessibility testing with Playwright.

This tutorial is part of a series on accessibility testing with axe-core and Playwright. The articles in this series are:

  1. axe-core, Playwright and Cucumber
  2. Running axe-core in the browser console
  3. Setting up Node.js
  4. Running axe-core from the command line
  5. Running axe-core with a Playwright script
  6. Accessibility testing with Playwright

Back to home