Role: Lead UX Researcher
Methodology: Moderated Usability Studies
- The Challenge: When parents need to find service providers to support their children with disabilities, schools and governmental organizations typically hand them a list of providers to contact. Parents typically go down the list, calling one provider after another, getting little response, and having no clue which would be the most appropriate for their child.
- The Solution: We created a searchable directory of providers, designed based on our research about the types of filters and provider details most desired and needed by parents.
- The Impact: Parents surfaced important considerations we wouldn’t have recognized without their input, such as the importance of the provider having a waiting room. Our research allowed us to be certain that we were solving the problem instead of duplicating the problem.
📓 Context + Leadership Priorities
- Who: Inclusive is an organization that provides resources for caregivers of children with intellectual and developmental disabilities (I/DD).
- Target user: Caregiver of a child with I/DD seeking speech therapy services through the DOE (Department of Education) in New York City.
- Generative research had previously been completed, from which prototypes were developed
- Why: A usability study was needed to ensure that these products would make a good first impression with our users, and meet their needs.
- Fill the need-gap: There is presently no efficient and effective way to obtain services such as speech therapy for children with disabilities.
- Build trust: We want caregivers to trust Inclusive as a reliable source for disability resources.
- Timeline: 4 weeks to complete usability testing for 2 products, taking the products from mid-fi prototypes to hi-fi, and ready for development.
❓ Research Goals, Assumptions + Research Questions
- Objective: to assess the usability and desirability of two new products for Inclusive, to prepare them for development and launch:
- Guide to Services: A tool to help caregivers navigate the process of obtaining disability services for their child
- Resource Directory: A search tool for resources such as therapeutic providers and classes
- Having a search tool with up-to-date data on speech therapy providers would be useful to caregivers having difficulty selecting a speech therapist who accepts payment through the DOE (Department of Education)
- Caregivers would benefit from a navigational tool that guides them step-by-step through the process of obtaining speech therapy services through the DOE.
- Do users understand who Inclusive is and what services we offer?
- Is the language used, especially in regards to disability, both understandable and inoffensive?
Guide to Services:
- Are users able to navigate through the flow of obtaining speech therapy services for their child?
- Are the steps outlined are easy to understand and matching up with the lived experience of going through the process of obtaining services?
- Are users able to successfully complete the process of searching for and choosing a provider?
- Is there anything missing or confusing in the search filters or provider information cards/profiles?
🛠 Teams, Tools + Ownership
- My role: Team lead for the research team, leading four researchers
- Inclusive R&D is comprised of a number of pods. The pod I was working with included a project lead, project strategy team, content team, and design team, all of whom worked directly with the research team.
- Notion for documentation and collaboration
- Google Forms + Sheets for recruitment and research ops
- Canva for recruitment flyers
- Vowel and Figma for usability sessions
- Figma for collaboration with the design and content teams
- Kickoff documents and research plan document - to document the research and refer to it as a guide for our team and as a northern light for our overall goals.
- Creating the Usability Study Plan - to ensure that our research and discussion guide are in line with research goals.
- Evaluative 1:1 sessions with target caregivers - to answer our research questions and gain insight into target user perspectives.
- Analysis and synthesis - to organize the data gathered from 1:1 sessions and translate it into actionable insights
- Communication of insights (presentations, reports, meetings) - to ensure that key stakeholders have the information they need to develop a successful product
🪜 Methods and Process
- The research team was able to recruit our highly specific users through community connections we had developed during our generative research
- We developed a screener with questions to further refine our selection criteria in two ways:
- Socioeconomic diversity - Because the user journey of the caregiver in this situation is highly dependent on the resources available to them
- Age of child - Because parents of children age 5-8 have more recent experiences (better recall) and more relevant (the process evolves)
Completing the study
- To give both our discussion guide and prototypes a test run, we conducted 2 pilot usability sessions, based on which we made some last minute improvements
- We completed 5 target user sessions with rotating teams of two - an interviewer and a notetaker. We invited interested project members outside of research to sit in.
- Session notes - After each session, a team member would carefully review the video and transcript in order to refine the notes from the session, adding key quotes and themes. I reviewed all of the sessions and notes for accuracy.
- Thematic analysis + synthesis - After all sessions and notes were completed, I gathered the team for a working session in which we reviewed the notes and extracted themes, which we then organized by strength and potential impact.
Planning the Usability Study
- Cross-functional alignment - We met with design and content to agree on goals and negotiate the scope of the study. We didn’t have time to test every aspect of the new pages and products, so we had to prioritize.
- Study planning - I led the research team in creating a plan for the usability study including recruitment, session structure, questions-to-be-answered, discussion guide, and analysis/synthesis plan.
- Discussion guide - We carefully crafted open-ended interview questions based on our questions to be answered (above).
- Scheduling - With no research ops, we were also in charge of outreach, scheduling, re-scheduling, and re-scheduling again. These caregivers are busy!
🗒 Research Artifacts + Learnings
Themes from Usability Sessions
Some of the strongest themes from usability sessions are summarized below.
Guide to Resources
Sharing Findings with Stakeholders
We shared our study findings in a variety of ways, based on stakeholder needs.
🎯 Final Results + Impact
What we launched:
- Information about who Inclusive is and what they offer was abbreviated and moved above the fold
- Confusing language such as “I/DD resources” was removed and replaced
Guide to Services
- Due to overwhelming negative feedback, the overall organization and design of the guide is being reconsidered
- A revamped version of the guide will be tested before launching
- Filters were refined to reduce confusion over the naming of funding sources for services
- Designs and content were updated to reflect the greater understanding of user needs and usability issues gained through this study.
- Areas for future research were identified.
- Many of the findings will positively impact other Inclusive features currently in development.
The design, content, and research teams struggled with the timeline for prototype completion and testing. After completing the study, folks were divided on whether usability testing had happened too early or too late, which probably means that it happened right on time.
We had a near crisis just before testing began when leadership looked over the prototypes to be tested and declared them unfit for testing. While some last minute changes were negotiated with design and content, I also advocated for the value of testing a product that does not yet feel "perfect.”
Although the feedback we received on the Guide to Services was disappointing, I emphasized the importance of getting this feedback before the product went to market. We knew that we were trying to solve a big problem that no one else has successfully solved yet, so it's not surprising that we didn't get it right on the first try.
Though it can be tempting to focus only on what went wrong or wasn't good enough, it's important to also celebrate what's going well. The positive feedback we received on the Resource Directory motivated us to keep iterating on the Guide to Services before passing it to dev.