Key words: Accessibility, Product Design, Screen Reader
Project Overview
Among children younger than 18 years in the US, 6.8% of of them are diagnosed eye and vision condition; nearly 3% of them are blind or visually impaired.
Because of the technology limitation, they don’t have equal access to digital educational product.
All of our digital product do not support the use of screen readers for users who are blind and low vision. This initiative is focusing on screen reader implementation and design for our most widely used ELA (English Language Arts) products for grade 3-8 students: Building Reading Comprehension (BRC) and Building Sentence Comprehension (BSC), so we can provide equivalent digital experiences for students who are blind and low-vision.
My Role
I am the lead designer of this project starting in March 2021, and I am responsible for all aspects of the design features from conception to launch. I work with a senior product manager to lead the team of 9 developers and 4 QA engineers.
The POC (proof of concept) prototype using one lesson was completed in August 2021, and then the team started working toward applying the POC to the rest of 150+ grade 3-8 reading comprehension (RC) lessons. The official launch date for the Building Reading Comprehension (BRC) product was September 2022. The official launch date for the Building Sentence Comprehension (BSC) was March 2023.
Team & Timeline:
- Research: 03.2021 – 05.2021
- POC: 05.2021-08.2021
- Productization: 09.2021-03.2023
- BRC Release: 09.2022
- BSC Release: 03.2023
Methodology: User Research, Wireframing, Prototyping, Interaction Design, User Testing
Tools: Adobe XD, Javascript, VoiceOver/NVDA/JAWS, Braille Display
Initial Research
I work with stakeholders and product owner to define product strategy which supports the business objectives and goals, as well as creating high-level plan to focus on a specific market, target users and feature sets.
Business Value:
- Blind/low vision support remains the biggest gap in our product (over 80% of our “Does Not Support” categories in WCAG).
- This can help us increase the level of WCAG compliance, which is a legal requirement in many U.S states for digital educational products.
- Opportunity is there for Curriculum Associates to be a leader in the field for accessibility for educational products.
Overall Product Strategy:
By interviewing various educators who have experience with teaching students who are blind and have low vision, we decided to focus on the following:
- Students typically become proficient SR users by middle school (start to learn mid-late elementary). -> Focus: starting with grade 3 and up lessons
- Math domain often requires tactile manipulative for effective classroom learning; text-based reading content is typically easier to access via SR for students. -> Focus: targeting reading comprehension lessons
- Learning Braille display is important, it provides the direct connection to language with “seeing” words with one’s fingers. -> Focus: implementing refreshable braille display in addition to screen readers
- When possible, the same content is preferred but alternate content is acceptable. -> Focus: providing equitable experience to SR user
Phase One: Proof of Concept
Since there are so many unknowns in the development of assistive technologies, we decided to start with a proof of concept, it can help us demonstrate screen reader/braille implementation feasibility and verify this product has practical potential.
POC goals:
We are proposing an initial product strategy of providing screen reader (and refreshable braille display) support for the 3-8 Reading Comprehension (RC) domain – starting with a targeted POC (Proof of Concept). A POC can help us:
- Better understand the most effective documentation format/design process to support dev implementation & QA testing
- Learn more about UX of screen reader flow across domains (Including a variety of commonly-used activity types)
- Better estimate level of effort (design, development and testing) for screen reader implementation, for lesson teams moving forward for potential productization
Note: Team for the POC: 1 designers, 3 developers, 1.5 QA engineers.
WCAG Audit:
Before we get started, I did an extensive audit of the current product using WCAG success criteria and identified the areas that we are failing the compliance in terms of screen reader/braille experiences, our goal is to achieve level A & AA.
- Success Criterion 1.1.1 Non-text Content (Level A)
- Success Criterion 1.3.1 Info and Relationships (Level A)
- Success Criterion 1.3.2 Meaningful Sequence (Level A)
- Success Criterion 2.5.3 Label in Name (Level A)
- Success Criterion 4.1.2 Name, Role, Value (Level A)
- Success Criterion 4.1.3 Status Messages (Level AA)
Major Product Decisions:
1. Selection of POC
We selected a reading compression lesson “Jump into action” (Parts of Plays) for the POC, because:
- It is a 3rd-grade lesson, so we will potentially have more chances to test with students
- It uses a “play format”, which may be interesting for student feedback and is easy to read without not too much cognitive load
- It includes all three major render types and variations (maze, multiple-choice, graphic organizer) which will help us better understand the effort in relative complex interaction pattern
Opening screen
Lesson setting panel
Multiple choice render
Maze render
Graphic organizer render
“Find your clue” activity
2. Determine the screen reader + browser combination:
Based on the data from WebAIM, there are many combinations in use, we decide to focus on designing, developing and testing experiences for users using JAWS with Chrome, NVDA with Firefox and Chrome, and VoiceOver with Safari.
Based on the data from WebAIM, there are many combinations in use, we decide to focus on designing, developing and testing experiences for users using JAWS with Chrome, NVDA with Firefox and Chrome, and VoiceOver with Safari.
Creating SR Specs:
Since in the current design industry, there isn’t a standard accessible design documentation process or design spec format for screen reader, I came up with these 5 essential steps to achieve a good user experience for screen reader users.
- Define landmarks and headings for navigation
- Define content order, semantic roles and labels
- Add extra elements for UX improvement
- Define focus behavior and management
- Add status messages for visual only events
To get started, like the common wire framing process, we can mock up a rough sketch of the screen since we might not get this right the first time.
1. Define landmarks and headings for navigation
Since in the current design industry, there isn’t a standard accessible design documentation pBased on the data from WebAIM, the most common way to find information on the page is to navigate through the headings on the page, so we restructured the headings’ hierarchy and change the HTML tag in the code.
In addition, around half of the users use landmark roles for navigation sometimes and often, so we decided the screen into 4 major landmark regions (banner, passage region, activity region, footer) to provide easy navigation
2. Define content order, semantic roles and labels
The next step is to mark all the sr-relevant elements and the sequences of each element in a meaningful order and choose an ARIA Role for each element (button, radio, checkbox, link, switch, dialog, etc)
Then, provide a meaningful label and state for each element. For images, provide alt text for images.
3. Add extra elements for UX improvement
When it comes to repetitive content or places where users need to navigate back in the passage to locate certain content, we provide a skip link for users to easily jump to the relevant content to increase usability.
4. Define focus behavior and management
It is also important to indicate the SR cursor focus on the spec: where is the SR cursor focus when on load and when does the SR cursor focus move if users interact with certain elements.
5. Add status messages for visual only events
Status messages are frequently used to provide screen reader users information about visual-only events such as correct/incorrect icon showing, new content loaded onto the screen, existing elements’ state change, and any other screen changes to provide equivalent experiences for screen reader users.
Phase Two: Productization
The productization phase started in September 2021. We are applying the POC to the rest of the 150+ grade 3-8 reading comprehension (RC) lessons.
POC User Testing:
Before applying the patterns we implemented in the POC to the rest of 150+ reading compression lesson, I collaborated with the UXR (user research) team to conduct user testing with SR students. We have tested with 8 students in total.
Here are the major questions we are asking and observing in the testing session, and based on them, we have created extensive testing protocol:
- Can students successfully navigate between different sections? (banner, passage, activity, settings, etc.)
- Can students successfully read & interact with the passage?
- Can students successfully go through the three renders and “fine the clue” activity? (maze, multiple choice, graphic organizer)
- What are students’ preferred settings? (audio support on vs off, text unlocking on vs off, etc.)
Major Changes Based on Testing:
1. Navigation inside the passage seems to be difficult.
When students“accidentally” click certain keys and they are taken back to the top of the passage, there is no easy way to go back to where they were left. Based on students’ recommendation, we decided to embed invisible sr-only headings in the passage to divide the passage into different sections.
2.During the “Find the clue” activity, students seem to have problems difficulties locating the correct answer.
This is mainly because the answers choices sentence are located in the passage and are mixed up with normal passage sentences. In order to tackle this issue, we have embedded invisible sr-only text and status message to signal the boundaries of the answer choices area. Also, we decide to label the correct answer sentences after 30 seconds.
3. Students tend to prefer audio support off and text unlocking off
In terms of lesson setting preferences, students tend to prefer audio support off and text unlocking off, so there aren’t much extra audio buttons and “unlock” text buttons on the screen to distract students from other essential UI elements.
4. Levels of SR proficiency impact usability significantly.
While proficient users can navigate & interacting easily with our POC, less proficient users seem to have some difficulties at first. But the interaction patterns in the POC are consistent so they are quickly learnable behavior. In order to support this even more, we are in the process of creating guidance documentation for students and TVIs.
Phase Three: Productization
The productization phase started in September 2021. We are applying the POC to the rest of the 150+ grade 3-8 reading comprehension (RC) lessons.
Productization: Feature Breakdown
In order to estimate the scope of the project, I work with the product manager to analyze the different types of interactive components and their variations in the current product. Based on this, we break down different features so we can plan the project based on a timeline and estimate resources.
Team for the productization: 1 designers, 9 developers, 4 QA engineers.
Three additional renders are added to the scope:
Sentence Organizer Render
Sentence Modifier Render
Two-Part Questions Render
Creating Design Specs for All Features
I created detailed specs for each component and each render, and provided aria roles, states, labels for each element, as well as SR focus management and status messages. I also specified user flows in every scenario.
Design Highlight: Graphic Organizer (GO)
One of the renders, Graphic Organize is highly visual-dependent, it provides a visual approach for organizing and analyzing information drawn directly out of a text. This can include words, phrases, images, and concepts. In our current product, it has 15 different variations of layouts and some of them are highly complex, here are some examples:
Sentence Organizer Render
Sentence Modifier Render
Two-Part Questions Render
I decided to do the following to increase the usability for screen reader users:
- Provide easy to understand labels for UI elements
- Group related elements together and give them a label
- Attach a description of the layout at the beginning of the activity
Design Highlight: Drag and Drop Patterns
Drag and drop interaction is a common pattern in our digital educational product. However, it is not a common web pattern, so there is no extensive design guidelines and technical documentation for screen readers. In addition, drag and drop interactions can be challenging for screen reader users since it requires users to move elements from one location to another.
Sentence Organizer Render
Sentence Modifier Render
Multiple Targets —> Multiple Dropzones.
The KN/SR pattern is to have the users use the dropdown menu on the target to select a dropzone. This interactions pattern will be used in the current ELA product for the Sentence Organizer render and Sentence Modifier render.
Results & Summary
The BRC was release in Sepetmber, 2022 and BSC was release in March, 2023. This has made 150+ Grade 3-8 Reading Comprehesion lessons accessible for students who are blind or have low vision.
How does a full BRC ad BSC lesson look like?
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.
Results & Summary
This is a major milestone for ensuring access to our digital instruction content, allowing students who are blind or have low vision the ability to experience 3-8 Reading Comprehension lessons for the very first time. This project triggered a major shift in our product development circle: ensuring our focus on usability in accessibility, including centering input from end users of this assistive technology on our team from ideation, design, development and testing.
The release also marks the first time we will be able to claim WCAG 2.1 AA Compliance with documented exceptions, as our lack of screen reader support in any lessons previously blocked us from meeting some criteria.
Next Step & Additional Features
1. Optimize the refreshable braille display expierence
The output of the refreshable braille display is connected to the screen reader, we need to optimize the screen reader output for a refreshable Braille display to provide a smooth and efficient Braille reading experience, for example: test on common screen readers and Braille model combinations, provide guidelines for students to customize their screen reader and Braille device settings, and provide customized keybindings for specific functions to make navigation and interactions easier.
2. Provide audio descriptions for “moments of delight”
Another feature we are exploring is audio description: It involves providing a spoken narration of visual elements, actions, and non-verbal information on screen. In this product, we have many “moments of delight” transitional animations that create fun and surprising moments for students. We want to use audio description to provide an equivalent experience for users who are using a screen reader.
Lessons Learned
1. Retrofitting accessbility feature is NOT efficient.
This is the first screen reader retrofitting project ever. The biggest lesson we learned is that retrofitting is never the most efficient way to implement accessibility features. Modifying existing structures or products to meet accessibility standards can be costly and can be technically challenging, especially when dealing with older structures and if the original design is fundamentally incompatible. Even if the product team doesn’t have resource to implement certain accessbility feature right away, the product structure needs to be designed in a way that is flexible enough for “future-fitting”.
2. Always designing with accessibility in mind.
Another lesson we learned is the importance of designing with accessibility in mind. Our traditional approach is we have a strong tendency to focus on designing digital educational experiences for sighted users who use the mouse as an input device. However, pedagogy and interactions that help these users are not necessarily the best for users with different abilities and using different assistive technology (for example, complex drag-and-drop interactions can be fun and engaging for sighted mouse users, but can be cumbersome for keyboard navigation and screen reader users). In addition, accessible design often leads to a better overall user experience for all users.