User Experience Researcher
Car Guard
Mixed Methods | End-to-End Research
Purposes
Exploratory, Validitory, Generative, Evaluative
Data Types
Qualitative, Quantitative
Evidence
Attitudinal Traces, Behavioral Traces, Contextual Conditions
Methods
Surveys, Interviews, Focus Groups, Heuristic Evaluations, Usability Testing, Benchmarking, Analytics
Instruments
UserTesting.com, Qualtrics, Excel, Adobe Creative Suite
01 | Project Overview
Car Guard was a business initiative sponsored by the president of the organization with the intention to extend home security and automation capabilities around automobiles
The device and supporting app was developed by a third-party company with whom leadership contracted. The directive for our internal Product Innovation Teams was to integrate and expand the existing device and its supporting app with our home security and automation system. Our engineers had to coordinate with the contractor to implement all software and firmware needs.
01.1 | The Teams
Having established research at Vivint, I knew which teams would be impacted throughout the entire development lifecycle - designing, engineering, selling, installing, and supporting.
I met with the teams early and based on their expressed needs, I was able to create a more proactive, strategic research plan utilizing qualitative and quantitative methods.
02 | Problem: There was none
That is to say, there was no supporting, empirical evidence that gave this product a meaningful purpose in the lives of our customers. It was the proverbial solution looking for a problem.
While it seemed like a logical alignment with the business, its value was fundamentally driven by a vision inspired by technological capabilities.
A Cascade of Challenges 4
The lack of user-centered thinking in the product’s inception coupled with the constraints of an existing physical and digital product set off a series of challenges that the various teams were facing in succeeding the business directive.
The Disruption
It’s not uncommon for leadership to hand down product and feature directives to development teams.
What made this particularly disruptive was that it conflicted with the culture of product innovation the teams had spent years cultivating. A culture of human-centered thinking that drove innovation.
03 | Research Goals & Objectives
The biggest goal for research was providing the teams with essential insights needed on demand from start through launch in an accelerated timeline. To meet this challenge, the research strategy commanded a committed resource to deliver on:
​
-
Strategic, forward-thinking planning
-
Reactive adaptiveness
​​
Each research initiative had its own respective purpose defined in line with team needs, but it was imperative that all initiatives be coordinated within the greater goal of the project.
​This required a change in our operational function within the organization. Historically, as a team of one, I had to position myself as a horizontal service to support multiple verticals; however, at this time, the team had grown enough to support a temporary embedded approach. This gave me the ability to support the Car Guard team fully while providing the research team evidence to validate or challenge an embedded approach.
04 | Research-Method Strategy
The biggest goal for research was providing the teams with essential insights needed on demand from start through launch in an accelerated timeline. To meet this challenge, the research strategy commanded a committed resource to deliver on:
​
-
Strategic, forward-thinking planning
-
Reactive adaptiveness
​​
Each research initiative had its own respective purpose defined in line with team needs, but it was imperative that all initiatives be coordinated within the greater goal of the project.
05 | Analysis and Synthesis
Following are the all the research initiatives that were included in the Car Guard Project.
Heuristic / Expert Review
Purposes
Evaluative: Formative
Data Types
Qualitative
Evidence
Contextual Conditions
Methods
Heuristic Evaluation, Expert Review
Instruments
Adobe Creative Suite
Research began immediately with a series of mapping exercises to detail the existing software’s capabilities and the physical device’s setup experience. I was able to take these maps and conduct a series of usability evaluations and deliver a report on multiple issues needing to be addressed.
Composite Workflow Map
Expert Review: Product Installation
Outcome:
The teams had a centralized and hi-level view of the UI/UX of the third-party app in which to understand and coordinate the integration efforts immediately.
-
The lead UX Designer immediately began scoping the integration work across the entirety of the existing app and allocated junior UX Design resourcing according to their ability.
-
The PM and UX allocated internal Engineering resourcing to the appropriate areas of the app and coordinated with external engineers to begin coding the integration into the Vivint app.
-
The PM made the decision to simplify the installation process for the customer by using Supply Chain resourcing to install SIM cards into all devices before distribution, eliminating tedious work for installers as well as development work designing installation UI/UX.
Contextual Inquiries
Purposes
Exploratory
Data Types
Qualitative
Evidence
Attitudinal Traces, Behavioral Traces, Contextual Conditions
Methods
Interviews
Instruments
Video and Audio Equipment, Zoom
Concurrent to the reviews, I recruited for and scheduled interviews with consumers to give the team directional insight on product and feature value.
Outcome:
The project now had directional evidence to support efforts to augment and expand the device's capabilities with the greater ecosystem in Vivint's portfolio.
-
The team felt confident to move forward with supporting evidence.
Brainstorming Workshop
Following the readout of interview insights, I scheduled a series of ideation sessions to build a larger portfolio of features than what was initially given to the team considering our system’s capabilities. I took those features and conducted a Kano feature-prioritization survey to qualify and inform prioritization.​​​​​​​​​​​​​
Outcome:
The team now had a comprehensive list of features that I could take and begin to validate for prioritization. I used the Kano method in the subsequent research initiative to provide both quantifiable and generative insights.
Feature Prioritization
Purposes
Evaluative Summative
Data Types
Quantitative, Qualitative
Evidence
Attitudinal Traces
Methods
Surveys
Instruments
UserTesting.com, Qualtrics, Excel, Microsoft Office,
Adobe Creative Suite
The Kano Method is a survey tool I introduced at Vivint to give teams insights to product decision making.
The methodology is inherently a quantitative method, given it is a survey; however, by launching it through a platform like UserTesting or DSCOUT, I was able to capture qualitative data explaining the reasoning behind responses.
Outcome:
The project now had directional evidence to support efforts to augment and expand the device's capabilities with the greater ecosystem in Vivint's portfolio.
-
The team felt confident to move forward with supporting evidence.
Product Pricing | Market Research
Purposes
Evaluative Summative
Data Types
Quantitative
Evidence
Attitudinal Traces
Methods
Surveys, Validation
Instruments
Qualtrics, Excel, Microsoft Office, Adobe Creative Suite
During product development, the Product Manager approached me with a need for data to defend pricing model projections.
The PM and his supervisor were in disagreement with leadership on a pricing structure for the product and a monthly service fee. The team felt leadership’s expectations would price the product out of our market space. I employed a consumer-facing pricing survey backed with a competitive analysis to equip the team with the insights for their defense.
Service Pricing
Competitor Pricing
Method Validation
Outcome:
The project now had directional evidence to support efforts to augment and expand the device's capabilities with the greater ecosystem in Vivint's portfolio.
-
The team felt confident to move forward with supporting evidence.
Product Issues
Purposes
Evaluative Formative
Data Types
Qualitative, Quantitative
Evidence
Attitudinal Traces, Behavioral Traces, Contextual Conditions
Methods
Interviews, Surveys
Instruments
Video and Audio Equipment, Zoom
As the product moved into Beta deployment, my research shifted to provide teams with detailed insights to optimize the product experience.
I worked with a panel of customers and had them send their stories along with photos and videos to provide details to their experiences, especially the bad ones that would give the teams a thorough understanding of what was happening.
One issue in particular emerged as a threat, not only to the viability of the product, but to the business from a legal perspective.
The device was reportedly interfering with drivers - the initial feedback was that it was hitting their legs while driving. I interviewed multiple customers to explore the issue further and had others send in photos and videos documenting the circumstance.
Outcome:
The team had a more complete view of the problem beyond what the survey alone revealed.
-
The PM found a new, single solution that would solve two of the issues the customers were facing, reducing the per-item cost.
Product Performance
Purposes
Evaluative: Summative
Data Types
Quantitative
Evidence
Attitudinal Traces, Behavioral Traces, Contextual Conditions
Methods
Surveys, Benchmarking
Instruments
Qualtrics, Excel, Adobe Creative Suite
Once the Beta was complete, the product went into a Pilot and then full circulation for summer sales.
The research then shifted into a Summative Evaluation, much like a report card to report on whether or not the product “made the grade”. The report included overall experience evaluations, like NPS and Top Task Analysis as well as feature specific ones.
NET Promoter Score
Accuracy
Top-Task Analysis
Ease of Use
Outcome:
The team now had a series of measurements and benchmarks from which to report on and compare to in future efforts.
-
The team used the performance review to prioritize work along with issue resolution and new feature development.
06 | Conclusions
The product launched as scheduled for the National Inside Sales and Direct to Home Sales teams to distribute.
​
Feedback channels were set in place to begin tracking success metrics and to identify and address issues immediately.
​
Discovery research for the next iteration could begin; informing product decisions and keeping teams moving.
07 | Retrospective
A case for embedded research
This project proved the case for the efficacy of embedded research. Although I returned to a more horizontal function of research due to limited resources, we had a vision and evidence for expanding the research team's headcount. In the following years, we were able to increasing staffing to explore an embedded research program again.
​
A case for more research
The Product Organization and the teams outside - Marketing, Sales, Support, Supply Chain, Field Services - became increasingly aware of the capabilities within research, increasing the demand for it and supporting the need for additional budgeting.
​
​
​
​