Usability

2024

CoPilot Heuristic Evaluation

Year

2024

Industry

Software Development

Methods

Heuristic Evaluation, Accessibility UXR

Timeline

2 weeks

An accessibility heuristic evaluation was done on prototypes of a CoPilot integration within a Human Resources app designed for Microsoft. It was important for designers and engineers to understand any usability and accessibility errors users may face early on in the application's development.

Goals

  • Outline a repeatable process to evaluate the accessibility of apps (in development or production) for current and future designs
  • Identify usability and/or accessibility compliance issues in a way that can be communicated clearly to PMs, designers, and engineers. 

Roles & Responsibilities

Stakeholders

This work was presented to a panel of stakeholders including designers, research directors, engineers, product owners, and department leads. A final share-out deck was provided and presented to a group of over 40 different stakeholders, to identify any challenges and user flow errors of the product, as synthesized by the research team.

Collaborators
  • I was brought into this project in the second half of testing. I was responsible for evaluating the prototypes for accessibility and inclusivity, according to usability principles and WCAG/ADA guidelines.
  • The lead UX researcher on this project was responsible for evaluating the product as a whole and will be leading usability reviews once the product is launched and tested with end- users. 

Process

Create an accessibility heuristic checklist to evaluate the usability and inclusivity of features on a new app design. 

The heuristic evaluation used took inspiration from various accessibility checklists and heuristic evaluations (Microsoft’s Tenets and Traps, Microsoft’s Accessibility Checklist, and Deque System’s Accessibility Heurisitc Evaluation) used for user interfaces. Components of the evaluation were made specific to the goals and requirements of the task and organization. 

Example of the heuristic evaluation done for HelpDesk and Copilot. Documents provided instructions, metrics, and a place for direct notes

Wireframes representing two different user flows were evaluated using the heuristic checklist to identify any gaps or “failures” with the UI and code. Any notes or key features requiring attention were directly brought to the design and engineer teams to implement changes.

Future usability testing will be done, along with another round of heuristic evaluations, to make sure the final product is inclusive and usable for its users.

Deliverables

A report and share out of the heuristic evaluation was given to key stakeholders within the department. Any necessary improvements were clearly reported and shared with designers and engineers for further development before app release. 

A template for the accessibility heuristic evaluation was kept for researchers to use on current and future products to improve the overall accessibility UXR process. During the share-out, there was a discussion with designers, engineers, and researchers to review the document so that its processes are intuitive and easy to repeat in future projects.

Outcomes

An accessibility heuristic evaluation was created to evaluate Microsoft digital products and ultimately this research informed changes to be made for a more usable and inclusive product.

Learnings & Takeaways

Because the prototype did not have the entire code built in and limited functionality, the full user experience and user flows could not be evaluated. I recommended that the evaluation would be done again upon completion of the application's design, to address any gaps in the findings and to identify any UI errors that may challenge a user's experience.

Because I would not be leading research and usability evaluations on the completed project, I tried to ensure that enough documentation/resources were available to properly analyze the usability and accessibility of the final product. In addition to checklists and templates, I created a resource deck outlining inclusive design practices and resources that outline accessible user research practices for reference.

Other projects

No items found.