The Premium Metric
The Premium Metric was created to measure if a brand or product feels like a premium experience to customers/users. Developed using over a dozen studies with 500+ individual test takers in multiple markets, the Premium Metric is a tool to provide simple and immediate feedback on the perceived premium-ness of an interface, feature or product.
As a product designer it’s not uncommon to hear stakeholders voice their grand visions for their upcoming offerings in jargon and manager speak. They have ideas of how the customer will describe it and what qualities it will take to turn potential buys into dollar signs.
“Make it modern” they say, or “make it just like Brand X they’re premium.”
Ok, but what makes it premium? What even is premium?
How do you measure a qualitative attribute besides customer verbatims? Is it possible to compare something so subjective in a way that stakeholders can understand?
Premium can be measured with a qualitative methodology, similar to the System Usability Scale, by understanding how users perceive and think of premium. If the commonalities of premium can be understood across a data set then they can be quantified and compared.
Exploration and Metric Creation
An initial pool of 20 participants that fit the profile of a core or adjacent luxury vehicle customer, participated in a 90-minute usability study in September, 2017. An additional group was tested from a pool of participants in the United Kingdom. As a part of this initial study, a card sort was completed by each participant to understand how a customer defines the qualities of ‘premium’.
Participants were asked the define the word ‘premium’ and to characterize the qualities thereof. After providing a definition, the participants were presented with 8 individual IVI home screens. Participants were unaware of the screens’ origins and all branding had been removed from the images. The participants were asked to order the screens from most to least premium based on their individual definitions and talk-aloud while performing this task. Following a usability test of a prototype, participants were asked if the system they had interacted with met the premium criteria they had established.
High Level Findings
Initial high level findings were common and consistent across respondents.
- Premium is simple, not basic.
- Premium puts top priority on driving by surfacing timely and meaningful information.
- Premium offers lots of options, but isn’t busy.
- Premium thoughtfully balances appearance, ergonomics, usability and design elements.
- Premium includes graphics tailored to enhance the information presented.
- Premium is not common.
- Premium is the highest level of value. The exchange of value for cost is positive.
- Premium exemplifies high quality.
- Premium presents content that is informative and robust.
- Premium makes features easy to access and available “at my fingertips.”
“Premium means quality. It will be of a good quality and that could be in aesthetics, that could be the quality of the thought, the amount of thought in the investment they put into that.”
Further studies were completed and from over 1000 responses there were 26 qualities of premium that stood out. These 26 qualities were recognized by the participants while expressing their personal definition of premium and evaluating the prototype. Patterns emerged throughout the process that developed semantic differentials for the qualities, i.e., each item is represented by two terms with opposite meanings.
A premium product would represent as many, if not all, of the positive differentials as possible.
The 26 Qualities of Premium
Makes you feel good
Top of the Line
“When I think of that word I think of something that's usually offered like a little bit better — an item or object or service that you get that is a little bit above something that people normally get for a type of thing.”
GOALS OF THE METRIC
Once the qualities of ‘premium’ are established, a metric should be defined to accomplish these goals:
- Provide a simple and efficient way to measure and interpret the premium-ness of an interface, product, or feature.
- Establish a premium baseline for products to meet or exceed. Score a feature, interface, or product based on the agreed definition and qualities of premium.
- Potential to create a second metric that evaluates ‘premium’ service.
- Determine whether a feature or product meets the expectations of something that is premium.
- Provide signal on any qualities that are lacking or considered ‘failing’.
- Compare iterations of a feature or product to track progress with respect to premium design/experience over time.
The metric provides a quick score to determine the adjacency of premium.
Through ongoing studies the score established as a premium threshold is approximately 78/100.
As more data is collected the threshold is being refined.
In addition to the completing the metric, each participant is asked a binary yes or no question as to whether the product or interface evaluated meets their personal definition of premium. This allows the threshold to be evaluated and the overall score provided matched to the user definition.
The below graph shows the results for an interface study completed across three markets, comparing the market results to the overall scores of all data sets.
"Premium means elegant, It means top performing, it means, just um... intuitive, something your needs are met in advance rather than something you have to muddle through. It's already there for you. It's more intuitive to most people."
The score is also broken down into “hot-spots” that allow the researchers and designers to understand where the product or interface performed best and worst.
These hotspot graphs allow for actionable changes to be made to previously nebulous feedback about aesthetics, tone, brand, and expectations.
Two different studies shown before demonstrate the hot-spot results provided.
So what’s the big deal?
Previously qualitative data such as whether the right qualities of a product or interface evoked the right tone or brand voice were hard to communicate or understand. It’s very difficult to base competitive qualitative data against a baseline when you don’t understand what you’re looking for.
The Premium Metric removes some of the ambiguity of “I don’t like it, but I don’t know why” or “I love it because it’s so luxurious” in product design – which is especially useful when designing interfaces to match physical products.
I created this metric because I was sick of hearing stakeholders and designers use premium as some buzzword that they could just toss around without proof.
You can’t just say something is premium. Saying it doesn’t make it true.
With the premium metric designers and researchers can take actionable feedback away from a qualitative user study and integrate directly into UI designs and user flows, targeting the right aesthetic qualities and letting things within the product that work remain exceptional.
This metric is about enhancement to usability and understanding qualitative feedback at a core level. It also makes communicating why something is or isn’t premium back to a stakeholder much easier.