The Premium Metric

As a product designer it’s not uncommon to hear stakeholders voice their grand visions for their upcoming offerings in jargon and manager speak. They have ideas of how the customer will describe it and what qualities it will take to turn potential buys into dollar signs.

“Make it modern” they say, or “make it just like Brand X they’re premium.”

How do you measure a qualitative attribute besides customer verbatims? Is it possible to compare something so subjective in a way that stakeholders can understand?

Premium can be measured with a qualitative methodology, similar to the System Usability Scale, by understanding how users perceive and think of premium. If the commonalities of premium can be understood across a data set then they can be quantified and compared.

TEST SETUP

An initial pool of 20 participants that fit the profile of a core or adjacent luxury vehicle customer, participated in a 90-minute usability study in September, 2017. An additional group was tested from a pool of participants in the United Kingdom. As a part of this initial study, a card sort was completed by each participant to understand how a customer defines the qualities of ‘premium’.

METHODOLOGY

Participants were asked the define the word ‘premium’ and to characterize the qualities thereof. After providing a definition, the participants were presented with 8 individual IVI home screens. Participants were unaware of the screens’ origins and all branding had been removed from the images. The participants were asked to order the screens from most to least premium based on their individual definitions and talk-aloud while performing this task. Following a usability test of a prototype, participants were asked if the system they had interacted with met the premium criteria they had established.

Initial high level findings were common and consistent across respondents.

1. Premium is simple, not basic.
2. Premium puts top priority on driving by surfacing timely and meaningful information.
3. Premium offers lots of options, but isn’t busy.
4. Premium thoughtfully balances appearance, ergonomics, usability and design elements.
5. Premium includes graphics tailored to enhance the information presented.
6.  Premium is not common.
7. Premium is the highest level of value. The exchange of value for cost is positive.
8. Premium exemplifies high quality.
9. Premium presents content that is informative and robust.
10. Premium makes features easy to access and available “at my fingertips.”

CONTINUED UNDERSTANDING

Further studies were completed and from over 1000 responses there were 26 qualities of premium that stood out. These 26 qualities were recognized by the participants while expressing their personal definition of premium and evaluating the prototype. Patterns emerged throughout the process that developed semantic differentials for the qualities, i.e., each item is represented by two terms with opposite meanings.

A premium product would represent as many, if not all, of the positive differentials as possible.

GOALS OF THE METRIC

Once the qualities of ‘premium’ are established, a metric should be defined to accomplish these goals:

• Provide a simple and efficient way to measure and interpret the premium-ness of an interface, product, or feature.
• Establish a premium baseline for products to meet or exceed. Score a feature, interface, or product based on the agreed definition and qualities of premium.
• Potential to create a second metric that evaluates ‘premium’ service.
• Determine whether a feature or product meets the expectations of something that is premium.
• Provide signal on any qualities that are lacking or considered ‘failing’.
• Compare iterations of a feature or product to track progress with respect to premium design/experience over time.

The metric provides a quick score to determine the adjacency of premium.

Through ongoing studies the score established as a premium threshold is approximately 78/100.

As more data is collected the threshold is being refined.

In addition to the completing the metric, each participant is asked a binary yes or no question as to whether the product or interface evaluated meets their personal definition of premium. This allows the threshold to be evaluated and the overall score provided matched to the user definition.

The below graph shows the results for an interface study completed across three markets, comparing the market results to the overall scores of all data sets.

The score is also broken down into “hot-spots” that allow the researchers and designers to understand where the product or interface performed best and worst.

These hotspot graphs allow for actionable changes to be made to previously nebulous feedback about aesthetics, tone, brand, and expectations.

Two different studies shown before demonstrate the hot-spot results provided.

Previously qualitative data such as whether the right qualities of a product or interface evoked the right tone or brand voice were hard to communicate or understand. It’s very difficult to base competitive qualitative data against a baseline when you don’t understand what you’re looking for.

The Premium Metric removes some of the ambiguity of “I don’t like it, but I don’t know why” or “I love it because it’s so luxurious” in product design – which is especially useful when designing interfaces to match physical products.

I created this metric because I was sick of hearing stakeholders and designers use premium as some buzzword that they could just toss around without proof.

You can’t just say something is premium. Saying it doesn’t make it true.

With the premium metric designers and researchers can take actionable feedback away from a qualitative user study and integrate directly into UI designs and user flows, targeting the right aesthetic qualities and letting things within the product that work remain exceptional.

This metric is about enhancement to usability and understanding qualitative feedback at a core level. It also makes communicating why something is or isn’t premium back to a stakeholder much easier.