Modern product development is increasingly moving away from surface-level data, as industry experts identify a widening "say-do gap" that often leads to costly business failures. Despite the proliferation of data analytics, many organizations continue to operate on what researchers call "big hunches," assuming they understand user needs while lacking the empirical evidence to support those assumptions. To address this deficiency, user experience (UX) professionals are championing a multi-layered approach to customer understanding that prioritizes behavioral observation over direct questioning. At the heart of this shift is a framework developed by researcher Hannah Shamji, which categorizes customer insights into four distinct levels: what users say, what they think or feel, what they do, and ultimately, why they do it.
This shift comes at a critical time for the digital economy. As subscription-based models dominate the market, understanding the root causes of customer churn has become a survival imperative. Recent industry reports suggest that customer turnover is rarely a result of a single, easily identifiable factor. Instead, churn is often categorized into voluntary and involuntary segments, with underlying motivations that users themselves may not fully articulate. By triangulating data across these four levels of understanding, companies can move beyond the "obvious reasons" for user behavior to uncover the messy, noisy reality of how people interact with technology.
The Chronological Evolution of UX Methodologies
The history of user experience research has undergone several distinct phases since the formalization of usability testing in the 1980s and 1990s. Initially, research was largely confined to "lab settings," where participants were asked to complete tasks in controlled environments. By the early 2000s, the "Speak-Aloud" protocol became the industry standard. This method required users to verbalize their thought processes in real-time as they navigated an interface.
However, contemporary analysis suggests that this method may be fundamentally flawed. Experts like Erika Hall have noted that asking a question directly is often the least effective way to obtain a truthful or actionable answer. Human beings are frequently unaware of their true motivations and are prone to applying their own context and interpretations to researcher inquiries. Furthermore, the act of speaking while solving a complex task creates a cognitive load that can obscure genuine emotional responses.
In the last decade, the field has transitioned toward "silent observation" and mixed-method research. This evolution reflects a growing realization that what people say and what they do are often contradictory. For example, a user might state in a survey that they require a complex product comparison table, yet behavioral data might show they never utilize such a feature, instead relying on brand familiarity or price-point shortcuts.
The Linguistic Nuance of Uncertainty: Possible vs. Probable
A significant challenge in customer research lies in the inherent ambiguity of human language. A study on Dutch verbal probability terms highlights how unreliable word choice can be in a research context. While there is general agreement on extreme terms like "always" or "never," there is a massive variance in how people interpret terms such as "possible," "maybe," "uncertain," or "likely."
Data indicates that when a user says a feature is "likely" to be used, their internal estimation of probability might range anywhere from 50% to 90%. Conversely, "possible" can represent anything from a 1% to a 40% probability depending on the individual’s personal lexicon. This linguistic instability makes survey data notoriously difficult to quantify. For organizations to gain a realistic view of customer needs, they must look past these verbal descriptors and analyze the frequency and duration of actual interactions.
Analyzing the Four Levels of Understanding
The framework popularized by Hannah Shamji requires researchers to investigate four nested layers of reality, each providing a different dimension of the truth:
- What They Say: This is the outermost layer, often captured through surveys, interviews, and Net Promoter Scores (NPS). While this data is the easiest to collect, it is also the most susceptible to bias, exaggeration, and social desirability bias.
- What They Think or Feel: This layer attempts to capture the internal state of the user. It moves beyond the spoken word to identify the sentiments—such as frustration, anxiety, or delight—that drive interaction.
- What They Do: Behavioral data is considered the "gold standard" of UX research. This includes heatmaps, click-stream data, session recordings, and task completion rates. Observation reveals the "friction points" where users lose time or abandon a process, regardless of what they claimed they liked in a focus group.
- Why They Do It: The innermost layer seeks the root cause. This level of understanding requires a deep dive into the user’s environment, goals, and long-term motivations. It is here that researchers identify "jobs to be done" rather than just "features to be built."
The Empathy Debate: Sympathy vs. Compassion in Design
As organizations strive to reach the deeper levels of understanding, the role of empathy has become a subject of intense debate within the design community. Sarah Gibbons of the Nielsen Norman Group has articulated a "Spectrum of Empathy" that moves from pity (feeling sorry for someone) to sympathy (feeling with someone), then to empathy (understanding the experience), and finally to compassion (the drive to take action).
However, some critics, such as Alin Buda, argue against the "empathy-first" approach. Buda posits that it is a flawed belief that one must emotionally absorb a user’s experience to build a great product. From this perspective, the designer’s job is not to "emote" but to diagnose and solve problems. This pragmatic view suggests that emotional responses should be treated as data signals rather than emotional burdens for the researcher.
Regardless of where one stands on the empathy spectrum, tools like the "Emotion Wheel" by Geoffrey Roberts are increasingly used to help users articulate their sentiments more precisely. By moving beyond binary terms like "good" or "bad," researchers can identify specific feelings—such as feeling "overwhelmed" versus "confused"—which lead to very different design solutions.
The Shift from Validation to Diagnosis
One of the most significant shifts in professional UX culture is the move away from the word "validation." In many corporate environments, "validating" a design has become synonymous with seeking a "rubber stamp" for existing assumptions. Industry leaders like Nikki Anderson advocate for replacing "validate" with terms such as "research," "investigate," "assess," or "evaluate."
This linguistic change reflects a deeper philosophical shift: the goal of user testing should not be to prove that a design works, but to diagnose where it fails. A diagnostic approach assumes that the design is a hypothesis that needs testing, rather than a solution that needs confirmation. This mindset allows researchers to identify risks, doubts, and potential harms that might be overlooked in a traditional validation process.
Broader Business Impact and Implications
The inability to distinguish between what users say and what they do has profound financial implications. According to various market studies, up to 80% of new product features are rarely or never used. This represents a massive waste of engineering and design resources. By implementing a more rigorous, multi-layered research strategy, companies can significantly reduce this "feature bloat" and focus on the core needs that drive retention.
To make these insights actionable, companies are adopting practical strategies to spread user-centric data across departments:
- Video Highlights: Short, 30-second clips of users struggling with a specific task can be more persuasive to stakeholders than a 50-page report.
- Monthly Insight Newsletters: Keeping the entire organization, from marketing to engineering, informed about ongoing user struggles ensures that "the human element" remains at the forefront of decision-making.
- Mirroring Techniques: During interviews, researchers use mirroring—repeating back what a user said—to encourage the user to provide more context and detail, often revealing hidden frustrations.
Conclusion
The evolution of customer understanding marks a transition from a surface-level "listening" culture to a deep "observational" culture. By recognizing that human behavior is complex, contradictory, and often driven by subconscious motivations, organizations can move beyond the limitations of surveys and focus groups. The future of product design lies in the ability to triangulate data across the four levels of understanding, shifting the focus from confirming hunches to diagnosing real-world behavior. Without this depth of research, product development remains a high-stakes gamble based on assumptions that rarely survive the reality of user interaction.
