SOP Statistical Methods

​

Summary

The Standard Operating Procedure (SOP) for Statistical Methods establishes guidelines for selecting, applying, and documenting statistical techniques throughout medical device development, verification, and validation activities. This SOP ensures consistent and appropriate use of statistical analysis to demonstrate product safety, effectiveness, and regulatory compliance while maintaining scientific rigor in data interpretation.

​

Why is SOP Statistical Methods important?

Statistical methods in medical device development exist because regulatory agencies require objective, quantifiable evidence that your device performs safely and effectively. Simply testing a few units and declaring success isn’t sufficient - you must demonstrate statistically that your results are reliable and representative of your entire production. Statistical methods help you determine the right sample sizes to detect meaningful differences, calculate confidence levels that satisfy regulatory requirements, and interpret test data in ways that support your safety and performance claims. This SOP is essential for your certification because it ensures your testing approach is scientifically sound and will withstand regulatory scrutiny, preventing costly re-testing or submission rejections.

​

Regulatory Context

​

Guide

​

Determining When Statistical Methods Are Required

Identify situations where statistical analysis is necessary rather than optional. Sample size calculations are mandatory when you’re making claims about device performance, safety, or reliability. Use statistical methods for design verification testing where you need to demonstrate that specifications are met consistently. Apply statistical analysis for process validation to prove manufacturing processes produce consistent results. Statistical methods are also required for clinical evaluations and post-market surveillance data analysis.

​

Sample Size Determination Methodology

Select appropriate sample sizes based on the severity of potential hazards and required confidence levels. For catastrophic risks that could cause death, use 95% confidence with 95% reliability requiring larger sample sizes. For negligible risks with minimal patient impact, 90% confidence with 90% reliability may be acceptable. Consider the type of data you’re collecting - attribute data (pass/fail) requires different sample sizes than variable data (measured values). Document your rationale for chosen confidence and reliability levels based on risk assessment outcomes.

​

Attribute vs Variable Data Analysis

Understand when to use attribute sampling versus variable data analysis. Attribute data involves counting successes and failures, such as devices that pass or fail acceptance criteria. Use zero acceptance sampling plans when failures are unacceptable, calculating sample sizes that demonstrate compliance with acceptable risk levels. Variable data involves measuring continuous parameters like blood pressure accuracy or response times. Apply tolerance interval analysis to ensure specified percentages of your production meet performance requirements within defined confidence levels.

​

Statistical Validation of Software Functions

For software medical devices, implement statistical validation approaches appropriate to your software safety classification. Deterministic functions that always produce identical outputs for identical inputs may require minimal statistical validation. Non-deterministic functions involving algorithms, machine learning, or user interfaces require robust statistical testing. Calculate sample sizes sufficient to detect clinically meaningful performance differences and establish statistical significance thresholds appropriate to your device’s risk profile.

​

Risk-Based Statistical Planning

Integrate statistical planning with your risk management process. Higher risk functions require more rigorous statistical validation with larger sample sizes and stricter acceptance criteria. Document how statistical parameters relate to residual risk acceptance. For each identified hazard, determine appropriate statistical confidence levels that ensure patient safety while remaining practically achievable. Link statistical validation plans to specific risk controls and verify their effectiveness through statistical analysis.

​

Documentation and Regulatory Compliance

Maintain comprehensive documentation of all statistical analyses including methodology selection rationale, calculation details, and interpretation of results. Create statistical analysis plans before conducting studies, specifying hypotheses, endpoints, and acceptance criteria. Document any deviations from planned analyses and provide scientific justification. Ensure statistical reports include sufficient detail for regulatory reviewers to understand and verify your conclusions.

​

Example

​

Scenario:

You’re developing a blood glucose monitoring app that must achieve ±15% accuracy compared to laboratory reference methods. Your risk assessment identifies incorrect glucose readings as a serious hazard requiring 95% confidence with 90% reliability. You plan attribute testing where readings within ±15% are considered “successes.” Using zero acceptance sampling, you need 29 paired measurements with zero failures outside the ±15% range. For variable data analysis, you collect continuous accuracy measurements and calculate that 95% of readings fall within ±12.3% of reference values with 95% confidence. Your statistical validation plan includes power analysis showing your sample size can detect clinically meaningful differences, pre-specified statistical tests for accuracy assessment, and documented procedures for handling outliers or missing data.

​

Example Statistical Analysis Plan:

Statistical Methods Plan for Blood Glucose Monitoring App

1. Objectives and Hypotheses

• Primary objective: Demonstrate ±15% accuracy compared to laboratory reference
• Null hypothesis: Device accuracy differs from reference by >15%
• Alternative hypothesis: Device accuracy is within ±15% of reference values

2. Sample Size Determination

• Risk level: Serious hazard (incorrect glucose readings)
• Required confidence: 95%, Required reliability: 90%
• Attribute sampling plan: 29 measurements with zero acceptance number
• Power analysis: 80% power to detect 10% accuracy difference

3. Statistical Methods

• Primary analysis: Proportion of readings within ±15% (attribute data)
• Secondary analysis: Mean absolute relative difference (variable data)
• Confidence intervals: 95% confidence intervals for accuracy metrics
• Tolerance intervals: 95% confidence that 90% of readings meet specification

4. Data Collection Procedures

• Paired measurements: App reading vs laboratory reference
• Measurement conditions: Fasting and postprandial glucose levels
• Reference method: Laboratory-grade glucose analyzer (±2% accuracy)
• Data recording: Automated data capture with manual verification

5. Acceptance Criteria

• Primary endpoint: ≥95% of readings within ±15% of reference (zero failures in 29 measurements)
• Secondary endpoint: Mean absolute relative difference <10%
• Statistical significance: p<0.05 for hypothesis testing
• Clinical significance: Differences must not affect treatment decisions

6. Statistical Analysis Software and Validation

• Statistical software: R version 4.3.0 with documented validation
• Analysis scripts: Version-controlled and independently reviewed
• Data integrity: Electronic source data with audit trails
• Quality control: 10% independent verification of calculations

​

Q&A