Deep Analysis in UX Research: Ethnography as an Approach Not Method

The qualitative research methodologies of academia, including ethnographic practice in anthropology and other social sciences, is not the ethnography that is practiced in the private sector, despite the evangelizing efforts of industry thought leaders. In the private sector, where time and resources are constrained, we have retrofitted qualitative methods to emulate aspects of immersive participant-observation and the fieldwork canonical to ethnography.

The proposition put forward here is that ethnography in the private sector is no longer a specific research method but a goal-oriented approach that can be applied to many different kinds of qualitative research. As such, we can attribute varied research methods as “ethnographic” if and when they share the same underlying goal of identifying the intersection of context and environment with a person’s understanding, perception, and behavior.

Shifting our perspective on “ethnography” from a method towards that of an approach to research. Namely a focusing of data collection around relational and context building to support a user’s sayings and doings.

Ethnographic research, whatever the method of data collection applied, must include deep analysis. Analysis that works beyond pattern identification (repeated instances of the same observation, forming a “pattern”). The power of ethnographic analysis is in identifying logical associations and meanings inherent in the sayings and doings of people. It is about understanding the nexus of knowledge, belief/perception, culture, and power; specifically what facets of these elements come to bare on one’s interaction with said product or service to form their lived experience.

Tackling hypotheticals

The ethnographic approach of deep analysis lets you do something that no other UX research method is readily equipped to do — to assess hypotheticals. The “what if” question always on the tip of every product manager and marketing professional’s tongue when it comes to their users: “What if you could use X with Y?” A practiced qualitative researcher knows that a research participant’s response to such hypotheticals cannot be taken at face value. A true situation of what people say they would do as not being what they actually do in practice. The way to go about approaching any future-oriented hypotheticals — what if — is through contextualizing conditions of possible use within relevant underlying value systems, knowledge, perception, etc. that truly frame and form a user’s behavior.

With deep analysis you are armed with the relevant information to determine with a high degree of certainty if people actually will use X in hypothetical Y because you have enough of a framework to think the way they think, understand what factors or variables motivate their behavior, and the limits of their knowledge or information.

Example Application: New Game Console Controller Form Factor

(Note: this scenario has been fictionalized for purposes of confidentiality. It draws components from past research to demonstrate application of ethnographic approach and deep analysis)

A research request came in to assess the performance of the new interaction schema for a video game console controller. The product team wanted to know if gamers would prefer the new configuration over the current and if the game play experience would be satisfying. Like any user researcher, my alarm bells went off upon hearing product questions containing “if” and “will users.” In using the aforementioned ethnographic approach with deep analysis, I was able to arrive at the conditions under which the new interaction schema would be successful and much more.

We set out to perform qualitative research, involving broad recruitment of user segments, including: long term console users, newer console users, diverging demographics, heavy and light gamers, and a cross section of game-genre’s engaged in. A series of unmoderated and loosely scripted in-depth interviews were performed, assessing past and current gaming experience, monumental or influential games in the individual’s life, game play and genre preferences, and must-haves (what is valuable to you). In our analysis the following was uncovered:

1. Inherited mental models introduced friction. Particularly acute in moments of heightened player stress/emotion, users found themselves resorting to past, and firmly ingrained, mental models for engaging action buttons in game play. If/when a game’s interaction settings are “not normal” participants evoked a negative emotional register: frustration, disliking, dismissing, and even abandoning the game. Conversely, games that had “intuitive” game design, that is, in line with their existing mental models, were spoken of highly. Embellishing descriptors were applied such as “tactile” and “having gravity” to describe satisfying game play experiences.
2. There are already a host of innovative designer controllers available to use even for the game console in question. Motivation to purchase one of these designer controllers were usually for niche applications. For example, purchasing “retro controllers” was important for avid retro gamers that wanted to simulate the environment the game was “intended” to be played in.
3. Game studios have ownership of game control settings. Study participants spoke not of the controller’s setup and capabilities (interaction schema), but of the game’s settings, providing examples of games with good and bad gameplay settings. Without buy-in and adoption of the new interaction schema from game studios at time of launch, the new controller will not gain the needed threshold to stand in as primary system controller.
4. Among the user populations evaluated, one segment in particular had an above average engagement with exclusively “new” video games. In qualitative research it came to light that this group included Video Game Live-streamers(on Twitch and Youtube), vocal members of gaming communities, and gamers who valued being on the cutting edge. For these players, using and experiencing new technology, such as our new form factor, was important for their personal (and sometimes professional) identity.

The resulting impact of this research was to recognize that the conditions of possibility that would enable success for the new controller, did not amount to any one individual’s preference. Rather, it encompassed much more — Instinctual motor-reflexes built over countless years of game-play; institutional backing and support from game studios; incremental adoption in new games by the latter user-segment that values being on the cutting edge; and lastly, building reflex level training of new mental models.

All of these variables were closely connected. One might have the desire to use the new interaction schema, but no institutional support without game design adopting it. Competing motivations could come to a head in moments of high stress, resulting in poor game performance despite one’s desire for authentic, cutting edge, game-play experience.

Recognizing how these different variables compete or compound, research insights led to the development of a multifaceted solution — the building out of a tutorial game to help familiarize users with the new schematics and allow them repetitious practice in embodied learning; the intentional development of a “Retro Gamers” controller series, supporting authentic game-play; building out marketing division to foster relationships and rapport with game studios to ensure adoption and incorporation of new controller schematics; and lastly, introducing a plugin for porting older games into new controller schematics to ease cumbersome game play and cognitive load of setting’s customization.