Most product research produces a deck that gets presented, agreed with, and then has no observable effect on what the team builds. That is not a research quality problem, it is usually a design problem: the study was set up in a way that could only confirm the existing plan. Research earns its cost when it is capable of returning an answer somebody did not want, and when it arrives early enough that acting on it is still cheap. This is a core part of experience design.
01Ask a question that could change a decision
The test of a research question is whether a specific decision would be made differently depending on the answer. What do users think of this design fails that test, because no answer to it changes anything in particular. Whether users can complete this task without assistance passes, because a no has an obvious consequence.
Before commissioning anything, write down the decision waiting on the result and what each possible finding would mean for it. If nothing changes under any outcome, the study is reassurance rather than research, and it should be recognised as such before it is funded.
02What people do and what people say diverge
Asked whether they would use a feature, people are agreeable. Asked what they want, they describe improvements to what they already know rather than what would serve them best. Neither response predicts behaviour reliably, and both are what unstructured interviews mostly collect.
Observation is more expensive and much more truthful. Watching someone attempt a real task in their own environment reveals the workaround they built, the field they always skip and the step where they consistently hesitate, none of which they would have thought to mention because it has stopped being remarkable to them.
| Question | Appropriate method |
|---|---|
| Can people complete this task? | Moderated usability test |
| Why is this step abandoned? | Contextual observation plus analytics |
| Which of two designs performs better? | A/B test with real traffic |
| What problems exist that we have not named? | Open contextual interviews |
| How widespread is this problem? | Survey, but only after qualitative work |
| Would people pay for this? | Behavioural signal, never a stated answer |
03A handful of participants is usually enough
Teams delay research because a statistically meaningful sample seems unaffordable. For usability work that concern is misplaced: a small number of participants surfaces the majority of the problems in a given flow, because serious usability faults affect nearly everyone who encounters them.
Statistical power matters for questions of magnitude, such as how much a change improves conversion, and those belong to quantitative methods. For identifying what is broken, five well-chosen participants observed carefully will change more decisions than a survey of a thousand.
- Recruit people who match the actual user, not colleagues or friendly customers.
- Include people who abandoned the product; they know things current users cannot tell you.
- Test the real task with their real data where privacy permits.
- Run a small round, fix what you find, then run another. Iteration beats sample size.
- Reserve large samples for measuring effects, not for finding problems.
04Involve the people who will act on it
A research finding delivered as a report to people who were not present is routinely disputed, reinterpreted or quietly ignored. The same finding witnessed directly is rarely argued with, because watching a user fail at something you designed is not an experience that survives much rationalisation.
Have engineers and product managers observe sessions live, even one each. It converts research from an external opinion into a shared observation, and it is by far the most reliable mechanism for making findings stick.
05Research late is just documentation
Testing a design the week before release can only produce cosmetic changes, because everything structural is now expensive to alter. The findings are real and the timing makes them unusable, which teaches the organisation that research does not deliver value.
Run it when the design is cheap to change: on sketches, prototypes or even a competitor's product standing in for the concept. The fidelity required to learn whether a workflow makes sense is far lower than teams assume, and low fidelity has the advantage that participants critique the idea rather than the visual polish.
- Test concepts before committing to a direction, when changing it costs a day.
- Test prototypes before build, when changing it costs a week.
- Test the built product to verify, not to discover.
- Low fidelity invites honest criticism; polished mockups suppress it.
- If the only available slot is pre-release, be clear that it is validation, not research.
06When to skip it
Research is not always the right spend. Where the problem is already well evidenced by support tickets and analytics, where the fix is obvious and cheap, or where the decision is genuinely reversible, building and measuring is faster and more informative than studying.
The discipline is knowing which situation you are in. Expensive, hard to reverse, or affecting a workflow people depend on daily: research first. Cheap, reversible and measurable in production: ship it and watch. Spending research budget on reversible decisions leaves none for the irreversible ones.
Topics
Priya Iyer
Staff Engineer · SyncTrix
Writes about the engineering decisions behind production systems - architecture, delivery and the trade-offs that only show up at scale.
Building something like this?
SyncTrix engineers AI, SaaS, platform and cloud systems for enterprises and high-growth teams. Tell us what you're shipping and we'll scope it with you.