Why is there no easy way to measure user experience?
Whilst out walking the dog the other day, I was listening to an episode of The Infinite Monkey Cage podcast titled “The Human Brain”. The guests on this episode were David Eagleman, Professor of Neuroscience at Stanford, Gina Rippon, Professor of Cognitive Neuroimaging at Aston University in Birmingham, and Conan O’Brien of late-night American TV fame.
You may ask, why am I writing about this when the work I do is centred so heavily around UX and design? It is because the way in which human beings perceive and interact is so intrinsic to how we go about building our products if we want them to be successful.
The part that struck me the most in this episode was the notion that there is no specific location for a given task within the brain. For example, what the brain does when we drive a car or watch a movie is not contained to a single area within the brain; the processes are distributed.
This quote from Professor Gina Rippon in this podcast episode describes how we used to try to organise things within the brain:
Early on with brain imaging, people were still trying to do the Phrenology thing and putting the brain into nice little boxes and saying that’s where justice is, or maternal instinct.
It is now understood that this is not how the brain works when performing any kind of task. Trying to pinpoint a location of happiness, frustration, or other emotion in the brain does not deliver an easily conveyable answer. There is no one specific thing we can measure in all human beings that will give us a definitive answer for this.
Professor David Eagleman likened this way in which the brain functions to the economy of a city:
If you look at a city and ask ‘Where is the economy of the city?’, it’s not any one place, it’s an emergent property of the functioning of everyone interacting there.
This is where the problem of measuring user experience becomes pertinent.
Businesses run on numbers. Decisions are predominantly based on a return on investment; If X amount is spent on creating a given solution, then Y needs to be the outcome for it to be given the green light.
The problem we have with Design and User Experience is to be able to give solid and actionable numbers. There are a collection of measurements that we can use that businesses are familiar and more comfortable with, such as Net Promoter Score (NPS), Average Order Value, Conversion Rates, Customer Satisfaction Score, System Usability Scale, the list goes on and on.
Just as there is no single way to determine where happiness lives in the brain, there is no single way to determine the experience of a user when using your product.
Many organisations and individuals are always striving to create a single metric that enables us to measure the user experience on a given product, and they all become a distillation of a wide breadth of many inputs down to a single metric.
This begins to create a problem where we are simplifying the complex nature of human experience – not from a single human being, but hundreds, thousands, or even millions of people – down into a single point of data.
I’ve had many conversations when trying to justify gathering more insights into the behavioural and attitudinal insights of the users, as often the findings don’t deliver a specific set of data which can easily inform decisions on what to do next.
User Experience is a messy discipline. It’s full of trial and error, full of moments of discovery and new understanding, and is always changing based on what you uncover when researching and testing with the users of your product.
Rather than attempting to distil the vast amount of data that can be collected by an iterative approach to user experience design, we should be looking to find the emergent properties from our research and use those as our guide to creating a better product.
Back to the podcast episode and Professor David Eagleman for a great explanation of emergent properties of the brain:
The notion of emergent properties […] is a well-accepted notion because, if you look at a neuron, you don’t have anything about your identity or consciousness in a neuron, it’s something that you have to put together 86 billion neurons to get out of it.
That is how our brains work. It is how our own experiences are pieced together; not through a single area of our brain that deals with a specific thing, but a huge collection of neurons firing that creates an emergent experience.
Essentially, the experience of someone using our product is not measurable only by statistics; it is a sum that is greater than its parts. It is millions of things coming together to form something unique within an individual. It is something that only they can recall, and can only share with others in the ways in which they have learned to communicate.
I doubt that we will ever gain a complete understanding of how user experience can be properly measured in one single person, let alone everybody who would use something we have designed for them.
So how do we go about convincing stakeholders in a business that the work we do is of value?
Treat user experience as an emergent property
Use all of the data and research you can to form a larger picture of the experiences your users have in a more collective fashion. A higher level of understanding is not the equivalent to a homogenised conclusion. Use things like user personas, the jobs to be done framework, and other similar tools to build a better picture and a wider understanding of the user experience.
Denounce silver bullet metrics
There is not one single way in which to measure a users experience effectively. Many of the metrics we use in an attempt to do this are often argued as misguided when used to inform stakeholders on the performance of user experience. Champion the fact that there is no one way to measure something that is as inherently complex as the human experience.
Convey multiple measurements as conclusions
Take all of the metrics, research, and data you can gather from your iterative design phases, launches, and ongoing use, and pull them together to make informed conclusions. These conclusions are what need to be communicated to stakeholders, and if you need to dive into the details of how and why you came to these conclusions, you’ll be able to justify everything with the data from your findings.
Things change. Constantly. You will not likely be able to conduct research and gather insight from every single one of your users for every change you make in your offering. The only way to be certain that you are moving in the right direction, and that you have the backing of your stakeholders is to be in constant communication with them. When things don’t go as expected, communication of the facts is key to prevent continuing down the wrong path and providing the information that decision-makers need to make the right choice.
There is no easy way to measure the user experience as the human brain and its associated perception and experiences are so complex. After all, that’s what we’re trying to understand to build a better product; how individuals experience the things we design and build for them so we can aim to provide them with a better experience in future.