Overview
Depending on the maturing of the orgainzaion, I like to have a list of possible testing methods to use.
I have created a list of testing methods so that I can refer to when a product owner asks me to test a new feature or a new design.
Expert Based Methods
Heuristic Evaluation
A systematic inspection of the user interface design against recognized usability principles (heuristics). Experts evaluate the interface independently and compare findings. Measures compliance with established design principles and identifies potential usability problems.
We had a classic case of accretion. Each addition made local sense. Globally, the dashboard had become a list of everything the product could do, rather than a tool for the things users actually needed.
Cognitive Walkthrough
Evaluators work through a series of tasks from a user's perspective, focusing on cognitive processes required to complete each task. Measures learnability and identifies areas where users might struggle with task completion. The method involves the tester, who is not a user, asking four simple questions about the way a specific user journey is conducted.
- Will the user try to achieve the right result?
- Will the user notice that the correct action is available?
- Will the user associate the correct action with the effect they're trying to achieve?
- After the action is performed, will the user see that progress is being made toward that goal?
add link to an example
"The dashboard had become a mirror of the org chart, not a tool for the user."
User-Based Methods
Usability Testing
Observation of real users completing specific tasks while thinking aloud. Measures ease of use, task completion rates, time on task, and user satisfaction. (usability testing).
- Moderated: Conducted with a facilitator present
- Unmoderated: Users complete tasks independently
- Remote: Conducted online
- In-person: Conducted in a lab or specific location
Eye Tracking
Tracks users' eye movements while they interact with the interface. Measures visual attention, scanning patterns, and areas of interest, helping understand what users actually look at and for how long. (eye tracking).
First Click Testing
Users indicate where they would click first to complete a given task. Measures intuitive navigation and information architecture effectiveness
Preference Testing
A quick research method used to determine which design option users prefer when presented with two or more variations. It primarily focuses on subjective appeal, aesthetics, and initial impressions. (preference testing).
Information Architecgture
Card Sorting
Users organize content items into groups that make sense to them.
- Open-sort: Users create their own categories
- Closed-sort: Users use predefined categories
Measures how users expect content to be organized and labeled.
Tree Testing
Users attempt to locate items in a stripped-down site structure. Measures the effectiveness of your site's navigation hierarchy without visual design influence.
Quantitative Methods
A/B Testing
Compare two versions of a page element to see which performs better. Measures specific metrics like conversion rates, click-through rates, or time on page.
Multivariate Testing
Tests multiple variables simultaneously to determine optimal combinations. Measures how different elements interact with each other to influence user behavior.
Heatmap Analysis
Visual representation of where users click, move, and scroll on your pages. Measures user engagement and interaction patterns across your website.
Web Analytics
Collection and analysis of website usage data, including:
- Traffic patterns
- User flows
- Page performance
- User behavior
- Conversion funnels
Measures various aspects of user behavior and website performance at scale.
Technical Methods
Performance Testing
Evaluation of website speed, responsiveness, and stability under various conditions.
- Load times
- Server response times
- Resource usage
- Scalability
- Stability under stress
Cross-Browser Testing/Device Testing
Testing website functionality and appearance across different browsers, devices, and screen sizes. Measures consistency of user experience across platforms.
Accessibility Testing
Evaluation of website usability for people with disabilities. Measures compliance with accessibility standards (WCAG) and actual usability for users with various disabilities.
Feedback-Based Methods
Surveys and Questionnaires
Structured collection of user feedback through various question types. Measures user satisfaction, preferences, and demographic information.
User Interviews
In-depth conversations with users about their experiences, needs, and pain points. Measures qualitative aspects of user experience and gathers detailed feedback.
Contextual Inquiry
Qualitative research method where researchers observe users performing tasks and interacting within their natural environment (e.g., their workplace, home) to gain a deep understanding of their behaviors, context, challenges, workflow, and motivations as they naturally occur. It often involves observation combined with targeted questions asked "in context."
Focus Groups
Moderated discussions with groups of users about their experiences and expectations. Measures collective user opinions and generates ideas for improvement.
I've included this method due to its frequent request but it carries several challenges that I describe below. Use it carefully.
- Groupthink: Participants may conform to dominant opinions or avoid expressing dissenting views.
- Conformity pressure: Individuals may align with group members—especially influential ones—even if they disagree.
- Dominant voices: More outspoken participants can overshadow quieter ones, skewing the discussion.
- Social desirability bias: Participants sometimes provide answers they think are acceptable rather than honest.
- Response bias: People may hold back criticism or soften their feedback to avoid judgment.
- Moderator influence: A facilitator's tone, body language, or leading questions can unintentionally shape responses.
- Sampling limitations: Small or non‑random participant groups may not be representative of the broader population.
- Limited generalizability: Insights cannot be reliably extrapolated to larger audiences due to sample constraints.
Cross-Browser Testing/Device Testing
Testing website functionality and appearance across different browsers, devices, and screen sizes. Measures consistency of user experience across platforms.
Accessibility Testing
Evaluation of website usability for people with disabilities. Measures compliance with accessibility standards (WCAG) and actual usability for users with various disabilities.