​
Summary

The Usability Evaluation Plan outlines your strategy for evaluating the human factors and usability of your medical device. This plan defines both formative and summative usability testing approaches to ensure users can operate your device safely and effectively without committing use errors that could lead to harm.

​
Why is Usability Evaluation Plan important?

User errors are among the leading causes of medical device incidents and recalls. Regulators require manufacturers to systematically evaluate how real users interact with devices to identify potential use-related hazards before market release. The usability evaluation plan demonstrates your proactive approach to user safety by establishing testing methodologies that validate your user interface design. This planning prevents costly post-market issues and ensures your device meets regulatory requirements for human factors engineering under both FDA and MDR standards.

​
Regulatory Context

Under FDA Human Factors Guidance and 21 CFR Part 820.30 (Design Controls):

  • Human factors engineering required for medical devices per FDA guidance documents
  • Usability evaluation must be risk-based focusing on use-related hazards
  • Formative and summative testing required to validate user interface safety
  • Must identify critical tasks where user errors could cause significant harm
  • Testing results must inform design controls and risk management processes

Special attention required for:

  • High-risk devices requiring formal summative usability validation
  • Software medical devices with complex user interfaces
  • Integration with existing clinical workflows and environments
  • Documentation of use-related hazard scenarios in risk assessment

​
Guide

Your usability evaluation plan establishes a systematic approach to validate that users can operate your device safely and effectively.

​
Planning Framework

Begin by defining use-related hazard scenarios identified in your risk assessment that involve user interactions. These scenarios form the foundation for your testing strategy since they represent situations where user errors could lead to harm. Your plan should clearly distinguish between formative evaluation conducted during development and summative evaluation performed on the final device.

Document key definitions including critical tasks where user errors could cause significant harm, hazard-related use scenarios linking user actions to potential hazards, and user errors representing actions that differ from manufacturer intent. These definitions ensure consistent evaluation throughout your development process.

​
Formative Usability Evaluation

Plan formative evaluations as ongoing development activities rather than formal validation events. These evaluations help improve your user interface before final design freeze. Include expert reviews with medical professionals familiar with your user population, prototype testing with representative users, and design iteration based on feedback.

Formative evaluation can be conducted informally but should focus on identifying usability problems early when changes are still feasible. Plan to document key insights and design improvements resulting from formative work, as these demonstrate your systematic approach to human factors engineering.

​
Summative Usability Evaluation

Design summative evaluation as formal validation of your final user interface against hazard-related use scenarios. Plan to test all critical tasks with representative users under conditions that simulate intended use environments. Your summative evaluation must provide objective evidence that users can complete safety-critical tasks without committing use errors.

Plan for adequate sample sizes typically involving five or more representative users per user group, though complex devices may require larger samples. Ensure testing covers all user populations including primary users, secondary users, and any special populations identified in your intended use.

​
Testing Environment and Materials

Plan testing environments that simulate intended use conditions including lighting, noise levels, time pressures, and workflow interruptions typical of real-world usage. Provide users with actual labeling and instructions that will accompany your device, including any required training materials.

Consider training requirements as part of your evaluation plan. If user training is a risk control measure, plan for appropriate training delivery and time intervals that account for learning decay between training and testing.

​
Evaluation Criteria and Outcomes

Define clear acceptance criteria for successful task completion including time limits, accuracy requirements, and safety parameters. Plan to capture not only task failures but also use difficulties and close calls where users nearly commit errors but recover.

Your plan should specify how evaluation results will feed back into your risk management process including procedures for identifying new risks, evaluating risk control effectiveness, and updating design requirements based on findings.

​
Example

​
Scenario

You are developing a mobile ECG monitoring app that patients use at home to record heart rhythms and share data with their cardiologists. The app includes critical tasks like proper electrode placement guidance, rhythm recording, and emergency alert management. Your usability evaluation plan must ensure patients can safely operate the device without errors that could delay critical medical intervention.

​
Example Usability Evaluation Plan

ID: UEP-001-ECG-Monitor

Scope: This Usability Evaluation Plan outlines planned usability evaluation activities for the Mobile ECG Monitor System including patient-facing mobile application and clinical dashboard components.

Definitions:

  • Critical Task: A task where user error could delay emergency medical care or result in missed arrhythmia detection
  • Hazard-Related Use Scenario: Patient interaction scenarios that could lead to diagnostic delays or inappropriate clinical responses
  • Formative Evaluation: Development-phase testing to improve user interface design
  • Summative Evaluation: Final validation testing on production-equivalent device

Use-Related Hazard Scenarios: Hazard-related use scenarios identified in Risk Assessment include: patient failure to properly attach electrodes leading to poor signal quality (Risk ID: R-015), patient confusion about emergency symptoms resulting in delayed care-seeking (Risk ID: R-023), and clinician misinterpretation of patient-reported symptoms due to unclear data presentation (Risk ID: R-031).

Formative Usability Evaluation: During development, we conducted design reviews with cardiologists and emergency medicine physicians to gather feedback on clinical workflow integration. Patient advisory groups reviewed prototype interfaces focusing on clarity of instructions and symptom reporting mechanisms. Iterative testing with five patients identified confusion about electrode placement, leading to enhanced visual guidance features.

Key improvements from formative evaluation included simplified electrode placement animations, clearer emergency symptom descriptions, and streamlined data sharing workflows based on clinician feedback.

Summative Usability Evaluation: Summative evaluation will test the final device interface with 15 patients representing our intended user population: adults aged 45-80 with suspected cardiac arrhythmias and varying technology comfort levels. Testing will cover all hazard-related use scenarios through specific task scenarios.

Testing Tasks:

  • Task T-001: Initial device setup and account creation (Risk ID: R-015)
  • Task T-002: Proper electrode placement using app guidance (Risk ID: R-015)
  • Task T-003: Recording ECG during symptom episode (Risk ID: R-023)
  • Task T-004: Recognizing and responding to emergency alerts (Risk ID: R-023)
  • Task T-005: Sharing symptom diary with clinician (Risk ID: R-031)

Test Environment: Testing conducted in simulated home environment with typical distractions including television, family members, and time pressure scenarios. Patients receive standard instructions for use and 15-minute training session as specified in risk control measures.

Acceptance Criteria:

  • 100% of patients must successfully complete critical tasks T-003 and T-004 without errors
  • 95% of patients must achieve adequate signal quality (>80% clean signal) during 30-second recording
  • No patient should require more than two attempts to complete emergency response tasks
  • Use difficulties and close calls documented for risk assessment review

Results Integration: Evaluation results will be documented in the Usability Evaluation Report and integrated into risk management files. Any identified use errors will trigger risk assessment updates and potential design modifications. New risks identified during testing will be assessed and controlled according to ISO 14971 processes.

​
Q&A