Research methods
Moderated Usability Study, Interviews, Customer Journey Mapping
​
Industry
E-commerce
​
Role
Lead UX Researcher​
​
Year
2023
The journey of buying a designer bag
As a fashion lover at heart, I was very excited to work with one of my favourite luxury fashion brands. This case study outlines a large moderated usability study, covering the full online shopping journey across 4 key markets. The goal of the study was to understand how the complete redesign of the website impacted its usability, as well as brand recognition.
Research summary
A well-known luxury fashion brand needed input on the usability of their website after an impactful redesign. My team and I observed 21 participants across 4 key markets as they purchased an item of their choice on the website. They were asked to think out loud, and to answer a series of follow-up questions afterwards. We uncovered a variety of usability issues, as well as technical issues on the website. Some of these led to so much uncertainty or frustration that customers would have left the website. The outcomes were so valuable to the brand, that they decided to conduct the same study annually.
Research question
​
The brand had redesigned its website, without having insight in the impact of the new design on the shopping experience or brand recognition. I was able to help them answer the following research questions:
​
-
What tools do customers use during the shopping process, and how?
-
Is the website's brand identity intact after the redesign?
-
How can we make the website more inspiring?
A international sample ​of fashion lovers
We used a sample of 21 participants across the brand's 4 key markets across Europe and the USA We also made an even split between existing customers and potential customers (fashion enthusiasts that had not purchased anything from the brand before).
​
The existing customers were recruited by the client. I was personally responsible for the recruitment of the potential customers. I used a screener to reach targets in terms of demographics, affinity with luxury fashion and brand familiarity.
How to observe natural shopping behavior?
One of the most important aims of the study was to observe participants' natural shopping behaviour (as closely as possible in a moderated setting). This means that we needed participants to make a purchase. However, the product range of the client is expensive (think €400 for a handbag) - making it unlikely that they would just agree buy something for the sake of the study. This introduced the first challenge: How do we make participants place an order?
​​
We considered dummy orders and free products. However, in both of these approaches consumer behaviour is heavily impacted by the fact that participants don't spend money. After all, people will put more thought into expensive purchases when they spend their hard-earned money on them. In a study where observing natural shopping behaviour is the main objective, this is too big of a risk.
​
Therefore, we opted for offering a significant discount, in consultation with the client. The discount code could be used to purchase an item of their choice. The risk here is that there might be participants that won't want to place an order. We were happy to see that the approach proved to be successful: 86% of participants used the discount code for a purchase during the interview, and there were no returns. This indicates that people were serious about their purchase.
Option 1 | Dummy orders
Participants don't have to spend money on the test
Can lead to indifference about the purchase
The shopping process becomes unnatural
Option 2 | Free orders
Participants don't have to spend money on the test
Decision is more important, because they get it
Can heavily influence cunsumer behavior (after all, it's easier to spend someone else's money)
Option 3 | Discount code (winner)
Participants will make meaningful decisions
Therefore, it's closer to a natural shopping experience
Not all participants will want to spend money on the test and place an order
Check list ​
​
â–¢ Read product description
â–¢ Used the search functionality
â–¢ Used the filters
â–¢ Used the size chart
â–¢ Looked at delivery options
​
If these tasks are not naturally performed during Use Case 1, we will ask testers to perform additional tasks
Use cases and questions​
During the task, the moderator tracked if the pre-determined core functionalities of the website (see image) were spontaneously used. If not, additional tasks were given to the participant after the shopping task was completed. This way, we ensured that we would gather feedback about all functionalities, without breaking the participant's natural flow, and without framing them in any way during their task.
​
The script for this study included in-depth questions about their experience with all core functionalities, as well as their overall experience with the website.
For example:
-
"What are your thoughts on the checkout process?"
-
"What was the most difficult part of your experience today?"
We also went into detail about more brand-related topics, such as brand image and recognition. For example:
-
"Based on your knowledge of [brand], do you feel like the website represents the brand well? Why or why not?"
-
"How would you describe the type of person that shops at [brand]?"
Planning​
We started with 2 pilot sessions. This is a good way to test if the flow of a moderated usability test is as you envisioned, and if any unexpected problems come up. In this case, I noticed that the sessions were a little too short for participants to decide on an item to purchase. In consultation with the client, I decided to extend the duration of the sessions from 60 to 90 minutes.
​
My team and I completed the rest of the sessions over a course of 4 weeks. All test sessions were moderated in the participants' native languages.
Results
30+ hours of interview recordings hold a lot of interesting insights. A detailed report was created, with a summary of results about each of the pre-defined core pages, the overall experience, and the follow-up questions. In addition, I delivered a summary video consisting of interview snippets, showing the most critical usability problems.
​
01
Missing product information causes potential customers not to place an order
​​
Many customers were unable to find the product information they needed in order to make an informed decision. The most important example of this is the missing dimensions of bags (the brands most popular product category!). This insight proved to be valuable in increasing conversion and reducing returns.
​
​
02
Product finding tools cause frustration
​​
Filtering and searching are the two most popular methods for finding desired products. Unfortunately, both tools regularly gave irrelevant output, causing frustration. This insight validated an existing assumption within the client team, making it's priority on the roadmap increase.
​
​
Quotes​
"I need to know how big the bag is. I am not sure if my laptop will fit inside."
​
"There are no model photos, so I am looking for the measurements. That way I can compare them with a bag I already own."
Quotes
​
"I know they sell leather jackets, but when I typ it in the search bar I don't see any."
​
"I filtered on coats, why do I get blazers?"
​
​
​
Customer Journey Map
​
Based on the Moderated Usability Test in combination with the in-depth interview questions, a Customer Journey Map was created. Below is a simplified version, that was used as part of the management summary.
In an ideal world... ​
​
​
01
Working in different languages
​
Analyzing interview data in four different languages proofed to be challenging. Each researcher had to translate their interview notes to English before I could even start coding. As it would not be possible for me to quickly rewatch parts of the recordings, the notes had to be extra detailed. If I could redo this study, I would work with a translation service to subtitle the recordings. I have done this with many multi-lingual studies afterwards.
​
02
Decision-making time​
​
We got from some of the participants is that 90 minutes is not enough time to decide on a large purchases like a designer bag. If I were to do a similar study again, I would like to make the moderated usability test part of a larger diary study. It would be interesting if participants would report on their considerations and different website visits over the span of multiple days or weeks.
Curious what I can do for your project?