Discovering fundamental User Needs for a Last Mile Logistics Business

Context

Shell Mobility Systems (Beezr) released an MVP application for the Service Operatives of their Vehicle Maintenance and Last Mile Logistics Business. Feedback, wish lists and ideas to improve the application start flooding in, which coupled with high attrition rate of Service Operatives overwhelms the business and tech teams.

The Challenge

How might we sustainably win with our Service Operatives solving for the right fundamental problems without incurring a lot of new costs for our MVP?

The Solution

Together we changed the way we envisioned the role of the Service Operative application of Shell Mobility Systems, shifting the mindset from a service appointment data-collection app to a single, streamlined app spanning all operational needs, adding value via a simplified operative workflows and new service-oriented features.

The research informed the strategic decisions taken by the business, prioritising features in product roadmap that address the core user needs. Using the insights from the research, we established a criteria for the business to either justify staying on course or make the right adjustments to certain key features.

Year

2021

Project Duration

3 months

Role

Research Lead

Teammates

Swathi Vishwanath

Neha Vaswani

The Impact

Through the research we discovered the on-field realities of the Service Operatives and misalignment of the designed conceptual model (MVP) and the users’ mental models.

The insights led to…

~40% workflow efficiency increase with minimal development costs

We streamlined the workflow of 2 core features and proposed the addition of a simplified consumables invoicing process, semi-automating regularly performed tasks

Potentially adding £1.5 million in 1 year to the business bottom line

Identified the need for creating and assigning a new skill for Service Operatives in the system to formalize the operative tasks that were currently not invoiced.

Improved the understanding and value of UX Research

Set up a regular research process as a part of the sprint and release planning. Baselined the current user experience and identified key to measure the impact of future work

The Process

Plan

Hypotheses

Conduct

Synthesis

Outcomes

Plan

Defining the Research Objectives

Designing the Research Study

The business had approached to conduct Usability Tests on the Version 1 of the MVP in order to identify the users' pain points. However, on understanding the background and business context, I suggested a generative research approach would bring more value to the business in the current scenario, especially as no prior in-depth user research had been conducted with the Service Operatives user group.

To enhance credibility and validity of the research and maximize its impact, I chose to triangulate research using the following methods.

01 Business and Stakeholder (Operations) workshop to…

understand the Shell Mobility Systems current strategy and product vision
understand the state of the operations in the 3 markets
scope the research

02 In-depth Interviews with Users (Service Operatives) to…

understand their challenges, motivations and needs
uncover any major and common usability issues in the product
identify the high-stress points/zones in the user journey

03 Day-in-the-Life Logs* to…

observe users as they perform tasks with the user interface and identify barriers
uncover differences in way of working across markets
understand the on-field realities
understand user behaviours on the App

*Screen recordings + Diary study style logs for a single day

Considering these methods, the overall research plan and timeline looked like this:

Hypotheses

Hypothesis

Business And Stakeholder (Operations) Workshops

The workshop informed my understanding of the Business’s strategy, the current state Operative experience, as well as aspirations for the future state. I also utilized the workshop to prioritize high priority segments of the user journey that were critical for business and would be the focus of the research.

We shortlisted 3 key sections within the Service Operative experience using the following parameters:

Shortlisted sections-

Generate Hypothesis

Identify the assumptions and biases in the current design that knowingly/ unknowingly influenced the design decisions. One example hypothesis from the research:

Participant Recruitment

Conduct

Day in the life Logs

I first selected UserTesting.com to conduct the logs. However, this was unsuccessful as the screen-recordings were too long (~1 hour) and failed to upload to the UserTesting platform.

I quickly adapted the research method to an everyday life tool- WhatsApp. In a small workshop, I assisted the operatives to enable ‘Show Touches’ on their phones and shared the instructions for creating their logs.

The Operatives were to:

Screen recorded every instance of using the App in a single day.
Make audio/video/text logs of what went well, and what did not go well throughout their workday.

In-depth user interviews:

60 minutes semi-structured interviews.

For the interviews, I requested Dummy Work Orders to be created in the app. The dummy work orders helped to obtain detailed, step-by-step feedback for the 3 Key features.

Synthesis

Lean Interview Analysis:

During the interviewing process, I designed Interview Debrief Kits- a daily email to the entire team containing:

An interview summary pdf containing the User Profile (anonymized), Key Highlights, and User Quotes.
Schedule for the next interview

The Debrief Kits were shared with a Disclaimer that the contents are raw data. Insights will only be shared post analysis.

The Debrief Kits allowed an increased transparency and reliability on the research process.

Observations to findings:

Fragmenting each interview into the micro sections (Key highlights, Pain Points, Surprises, Behaviours) to acquire in-depth comprehension of individual mental models.

Day-in-the-life log analysis to identify workflow challenges

Combined analysis of day-in-the-life logs and dummy work tasks from the interview to uncover usability issues in the app.

Journey maps for each country to identify differences and overlaps

Findings to Insights:

Collecting the micro sections and connecting them in conjunction to acquire overarching patterns.

Applying the WHAT - WHY - SO WHAT framework to extract insights from the identified patterns.

Insights were then categorised into strategic and tactical.

Tactical insights were turned into PBIs and tagged to the relevant teams - UX, Mobile, Architecture etc.
Strategic insights were presented to stakeholders in cross-functional team workshops.

Outcomes

Prioritizing Strategic and tactical recommendations

Insights were categorised into strategic and tactical quick wins.

Tactical insights were turned into PBIs and tagged to the relevant teams - UX, Mobile, Architecture etc.
Strategic insights were presented to stakeholders in cross-functional team workshops.

These were then prioritized using the Desirability- Viability- Feasibility framework.

Deliverables and Outcomes

Shifting the mindset and vision for the Beezr App from a service performance data-collection app to a single, streamlined app spanning all operational needs

5 key Strategic insights uncovered addressing which will enable on-field operations to work more efficiently and drive additional revenue to the business.

43 usability and technical issues identified and converted into PBIs to be addressed by the UX and Development teams. These were then grouped and prioritized in the product backlog.

Set up a framework to regularly conduct research as a part of the sprint and release planning, to continuously learn from end users and minimize re-work.

Baselined the current experience and set up relevant metrics to measure impact of all future work.

Deliverables were an insights and recommendation report and user journey maps baselining the current user experience with respect to the 3 Key feature.