Conducting accessibility testing on complex widgets

Introduction

Because I am presenting from Australia, I’d like to acknowledge the Traditional Owners of the lands on which I’m presenting today, the Cammeraygal people.

I’d also like to pay my respects to Elders past and present, and to all First Nations people who may be present or watch a recording of this session.

‘Are the components accessible?’

Part of my role over the years has focussed on helping teams build and test a wide range of UI components.

The term ‘components’ (sometimes referred to as widgets) means any self-contained UI concept.

This could mean simple components such as:

• Links.
• Buttons.
• Form controls and their associated labels.

But most of my time was spent on complex components such as:

• Sortable tables.
• Accordions.
• Dropdown menus.
• Date pickers.

In the earlier days, I would conduct basic accessibility testing on each component, but there were some underlying problems in my approach.

My testing process lacked rigour, and I did not provide detailed documentation of my test results.

At one point, a manager asked me the following question:

Are all the components in the design system accessible? Can you guarantee it?

What is accessibility unit testing?

‘accessibility unit testing’ is a process where the smallest parts of a component are individually and independently evaluated.

The aim of accessibility unit testing is not to make broad, sweeping statements such as ‘this component is accessible’.

Instead, the aim is to test the accessibility of all aspects of the component.

This process provides teams with a detailed understanding about all possible accessibility issues associated with the component.

Accessibility unit testing is typically conducted when reviewing:

• An existing component within the system.
• A new component to be added to the system

Accessibility unit tests are often based on two concepts:

1. How all aspects of the component are presented to users - either on screen or via assistive technologies.

2. How users should be able to interact with the component to complete all relevant tasks.

These tests must include all possible happy and unhappy paths associated with the component.

A high-level task might be something like:

But numerous small interactions associated with this overall task should be tested.

And there are different methods that people use to achieve these interactions - mouse, keyboard, assistive technologies etc.

Each of these needs to be defined as a test.

For example, what tests would be needed to determine if the date picker overlay could be closed via keystrokes only?

These are two separate tests, because a date picker may allow users to close the the overlay using ESC, but send focus to the wrong location.

An unhappy path could be where the date-picker question is required, but the user does not enter a date value for the field.

What tests would be needed to determine if users could identify the field as required before deciding whether to enter data?

And what tests would be needed to determine if the field is accessible once it is in a state of error?

As you can see, every test must be written as a statement where the only outcome is either a pass or a fail.

Tests do not have to be tasks. A test may relate to how aspects of the component are presented to users.

Each test must be repeatable so that it can be re-tested at a later date.

These tests may need to be performed in complex combinations of different assistive technologies, browsers and operating systems.

The end result should be a document that defines the following:

• The test.
• The environment in which the task was performed.
• The result of the test within the environment.

Let’s look at an example of an autocomplete component from the last design system I worked on.

(I’m going to open a strange spreadsheet now).

Starting with people and their needs

How could you create a complex series of tests for any component?

You can start by looking at different groups of people and their needs.

This means you can focus on one user type and explore how they may need to interact with the component.

These groups of people could include:

1. People with some sort of cognitive impairment or neurodivergence.
2. People with some sort of colour vision deficiency.
3. People with reduced vision.
4. People with limited mobility.
5. People with limited or no sight.
6. People with limited or no hearing.

For each group, there are some high levels questions that can be asked for any component to help determine possible tests.

1. People with some sort of cognitive impairment or neurodivergence

1. Is the purpose of the component clear?
2. Is the component easy to use?
3. If additional instructions are required, are they clear?
4. If users make a mistake, can they quickly and easily recover?
5. Is there any complex language used associated with the component?
6. Are there any animations or movements that could distract the user?

2. People with some sort of colour vision deficiency

Does the component:

1. Have sufficient colour contrast?
2. Use alternative methods to display colour information?

3. People with reduced vision

Sometimes, these people may need to customise the user interface or magnify the display.

Is the component operable/understandable:

1. When the text is scaled to 400%?
2. When the overall layout is scaled to 400%?
3. When aspects of the component are magnified?
4. When displayed in low contrast?
5. When displayed in different viewport sizes?

4. People with limited mobility

Sometimes, these people may rely on keyboard interactions or voice recognition software.

1. Does the component need to receive focus?
2. Is focus managed into, within and out of the component?
3. Does focus order follow a meaningful sequence?
4. Can all actions be executed using keystrokes only?
5. Can all actions be executed using voice controls?
6. Are keystrokes intuitive for keyboard-only users?
7. If focus is relevant, are all visible focus states clearly defined?

5. People with limited or no sight

These people may rely on screen reader software.

1. Does the component have an accessible name?
2. Does the accessible name clearly define the component's purpose?
3. Does the component have a role?
4. Are all of the component's possible states defined?
5. If a value can be added within the component, is it exposed?
6. If the component has any dynamically added content, is this announced at the appropriate time?

Creating tests from each question

The next step is to break down each question into a series of individual tests.

In some cases, the question can almost become the test. For example:

Question:

Does the component have an accessible name?

Test 1:

Test 2:

In other cases, you may need multiple tests associated with a question. For example:

Question:

Can all actions be executed using keystrokes only?

This question may need a wide range of individual tests, depending on the complexity of the component.

Let’s dive a bit deeper into tests that could be used to determine the accessibility of a date picker.

For simplicity, we will focus on people with limited mobility.

How would someone who uses a keyboard-only need to interact with this component?

But which part of the date picker overlay should receive focus?

• The date picker overlay container?
• The current date?
• The close button?
• The previous or next month buttons?

You may need to do a lot of testing before deciding on the best solution.

Ideally, you should also test with real users to find out their preferences and expectations.

This is not a significant issue for keyboard users but becomes very important for screen reader users as they require additional context!

These are just some examples of unit tests that could be conducted using keyboard-only.

We could then use the same process for other uses, such as ‘People with limited or no sight’.

Screen readers must be given additional context for each interaction - name, role, states, values etc.

The names and roles must also clearly define the purpose of the element.

I always run keyboard-only tests before screen reader tests, as screen reader users need the same basic functionality with additional criteria.

After the tasks are finished?

We need to determine if each test can be conducted using a single browser or if multiple browsers are required.

We also need to determine if any assistive technologies are required for any tests and if these need to be conducted across multiple browser options.

We can then create a testing spreadsheet and begin testing!

Circling back to design and development

As mentioned earlier, this overall concept can also be used to help design and development new components:

Teams can ask themselves the following three questions:

Questions or discussion?