Also known as: Severity Rating System / Prioritization Matrix
evaluationmediumIntermediate
TL;DR
Analytical tool to classify, prioritize and assign criticality to usability problems found.
Strategic value
Focuses attention on problems that really matter. Avoids the frustration of delivering lists of problems without prioritization that paralyze developers and analysts.
The Severity Matrix is an analytical tool used after user research to classify, prioritize and assign a criticality level to usability problems found. It transforms an overwhelming list of errors into a structured action plan, evaluating problem impact against its frequency of occurrence.
What it is for
✓Classify and prioritize usability problems by impact and frequency
✓Transform extensive error lists into structured action plans
✓Justify to executives or developers why certain changes should be prioritized
✓Create a common language between UX and development to discuss problem severity
Research methods that feed it
Usability tests (Think Aloud)Heuristic evaluationsModerated remote testsError analysis in analytics
When to use it
✓After usability tests or heuristic evaluations with high volume of problems
✓When there are time or resource constraints preventing fixing everything before launch
✓To justify to executives or developers why certain changes should be prioritized
When NOT to use it
✗In very fast agile environments where documenting each problem with scales delays iteration
✗If technological cost to fix is minimal, better to fix immediately
✗When there is debate about the validity of severity scales among specialists
Required components
✓Issues list: description of identified usability problems
✓Impact/severity scale: problem magnitude for user (Low, Medium, High, Catastrophic)
✓Frequency scale (occurrence probability): percentage of users experiencing the problem
✓Criticality/priority score: result of combining impact and frequency (Level 1, 2, 3)
Optional components
○Business objectives impact: how much it affects commercial goals
○Implementation/technological costs: effort or time to fix
○Persistence: if it will repeatedly annoy or be overcome once learned
○Core task importance: how central is the affected task
○Workaround: if there's an alternative way to accomplish the task
How to create it step by step
1Identification and listing: Document all problems found during usability testing.
2Scale definition: Choose a consistent rating system (1-4, 1-5, or categorical).
3Independent evaluation and consensus: More than one specialist assigns ratings independently, then discusses differences.
4Criticality calculation: Use chosen method: Sum (Rubin), Multiplication, or Decision tree.
5Tier classification: Order from highest to lowest priority and assign action plan.
Tips for small teams
Use a simple 3-level scale (Critical, Important, Minor) instead of complex scales
If you're the only evaluator, document your reasoning for each rating
Include screenshots alongside each problem
Share the matrix with development before starting fixes to align priorities
Common mistakes
✗Subjectivity and lack of standardization: using own judgment without standard process creates conflicts
✗Evaluator Effect: tendency to rate problems you discovered as more severe
✗Reluctance to use low ratings: avoiding 'Low' category for fear development will ignore those problems
✗Treating it as exact science: believing ratings are irrefutable
Quality criteria
✓Consistency and transparency: same scale across all studies to allow historical comparisons
✓Common language: definitions of each level understandable by non-UX audience (developers, analysts, managers)
✓Actionability: transforms hundreds of data points into manageable, prioritized task list
Authority quotes
“Figuring out the criticality of a problem is a simple matter of adding the severity rating and the frequency rating.”
— Rubin & Chisnell
“Severity ratings help focus attention on the problems that really matter.”
— Handbook of Usability Testing
“By asking only three questions about any usability problem, we can classify its severity.”
— Usability Testing Essentials
Contextualized example
Context: Post-heuristic evaluation of e-learning platform with 47 identified problems.
Application: Using Criticality = Severity + Frequency (Rubin method): 8 problems classified Tier 1 (critical), 15 problems Tier 2 (important), 24 problems Tier 3 (minor). Tier 1 included: registration flow without email confirmation (Severity 4 + Frequency 4 = 8). Result: development prioritized all 8 Tier 1 problems for the current sprint.