The awesome team at Product School gave me the opportunity to talk about Product Retention. Check it out!
At the start of the last quarter (Jan-Mar) I set a goal for our team to speak to 1 customer per week. Our aim was to use these interviews to generate meaningful opportunities for our product roadmap.
We had recently launched a few features that didn’t meet our expectations. We spent a lot of time making sure the features worked as intended and were technically sound, but that wasn’t enough. The results showed that we hadn’t moved the needle and we were left scratching our heads.
While reading some of Teresa Torres’ posts on Product Discovery, I realized we had fallen into a trap of focusing on solutions rather than problems. A common theme with these features was that we hadn’t validated them with customers. Maybe we saw others in our industry with similar features or we just assumed there was a clear value proposition for our customers. We didn’t intend for this – we were short on User Research help at the time and allowed ourselves to fall into bad habits. None of us had any formal training in User Research and still had our normal job-duties to manage, but we were determined to change how we were working.
By the end of the quarter we had spoken to 15 customers, came up with 2 main opportunities for product improvement, impressed our stakeholders, built compassion for our customers and learned a lot along the way. Here’s how we did it.
One of the most challenging things for us was finding people to interview. Our products had tens of thousands of weekly users, but an email address or account wasn’t required so the vast majority of our users were ‘anonymous’ to us. One of the things we did to establish a relationship with our customers was to send a survey in our App. While the answers to the questions were useful to us, what we really wanted was the opportunity to talk to these customers. The final question in the survey asked whether the customer was willing to do an interview and, if they said yes, their email address.
Customer Service was another channel we used to find interviewees. A former User Researcher on our team had set up an email account that we used to handle customer correspondence and we used this for recruiting. I put this email in our App Store release notes and encouraged customers to contact us with questions. While the volume wasn’t huge, it was steady and those who wrote in were clearly engaged – and sometimes angry 🙁 If we successfully solved a customer’s issue we would follow up and ask if they were interested in speaking with us further.
The yield of actual interviews from customer service channels was higher than the one from completed surveys. I imagine this was the case because the customer reached out to us in the former scenario and was pre-disposed to speaking with us.
Getting the Interview
We were happy that we now had a list of customers to email, but this was just the beginning. We found that actually scheduling the interview was another challenge. Here are some of our learnings.
Meeting-scheduling software is a must! We sent out maybe 2-3 invite requests without using a scheduling tool and it was so cumbersome that we signed up for a tool right away. Typing in our available times into an email or asking customers for times that worked for them just wasn’t ideal. As we were drafting these emails my colleague turned to me and said ‘this is too hard, no one will sign up.’ She was right.
There are many of these tools on the market, but we chose Calendly. There’s a free option that syncs with the major Mail and Calendar providers and that’s all we needed to get started. This worked – we now appeared more official and customers actually booked time with us!
Incentives will help you book interviews. We weren’t able to offer gift cards for participation due to legal reasons so we had to be a bit more resourceful. We had some company swag around the office and we added a line about ‘giving you a small gift as a token of our appreciation’ to our interview requests – our bookings markedly increased when we did this. These gifts were mugs or pairs of gloves – nothing special, but customers really appreciated receiving them. So don’t worry about finding the perfect gift.
The Right Tools for the Interview
A good set-up with all the right people tools is critical for a successful interview. Here are some items that will help you get up and running with your interviews.
Have a teammate conduct the interview with you. Only one of you needs to moderate, but a teammate is great for taking notes and picking up some of the nuance you may miss as you moderate. They can also suggest follow-up questions that you may not have thought of and remind you to stop taking and just listen…as I am often guilty of 🙂
To help with transcription, we used the stock Voice Memos App on iOS to record the interviews. Just make sure to ask permission before you record.
It seems obvious, but a working phone/conference line is a must. We used our company conference lines and the customers had no problem dialing in and entering the PIN. On a few occasions we used Zoom, which always works and has a free tier.
Asking the Right Questions
The goal of our interviews was to produce generative research – we wanted to define the opportunities before pursuing the solutions. We were interested in finding out how our Product fit into our customers’ lives, why they used it and what we could do to make it better. There were three types of questions that we thought really helped us understand the customer.
1 – Questions to build Context
Open ended questions were key to learn about context. One of the specific questions that helped us was:
‘Walk us through the last time you used the App – how was the experience?”
By asking questions like this we got the ‘Why behind the What.’
So instead of asking ‘Where and When do you watch?’ and getting something like: ‘I watch in my bedroom’…..
We got something richer: ‘I missed the episode on Live TV and wanted to catch up on my Apple TV, but my kids were watching something. I decided to go into the bedroom instead and fold clothes as I watched on my phone.
With this answer alone we learned what triggered them to watch, which device they preferred and what they did as they watched. This information is extremely valuable as we design. For example, maybe ‘multi-tasking’ becomes an opportunity we explore further and consider something like enabling audio when the App is backgrounded on mobile.
2 – Questions to Improve the Product
These are fairly straightforward, but remember to keep it open-ended.
‘What could be improved in the current App experience?”
An important point here is not to lead. We get a lot of App Store reviews about the number of advertisements we show, but we’re not going to ask ‘Do you like Ads?’ NOBODY likes ads.
By asking a more open-ended question ‘How do you feel about the Ad experience?’ – we heard things like ‘I don’t love ads, but I understand why you have them…maybe you could show them all at the start of the video rather than throughout.’
This type of questioning also empowers customers. Instead of pushing your agenda, you are giving them the opportunity to drive the conversation. They LOVED this – so many many customers thanked us for allowing their voice to be heard!
3 – Questions to build Empathy
Asking a customer more about themselves is a great way to build empathy. One we liked to ask was:
‘Tell us a little more about yourself. What do you do for a living? Where did you grow up? Any hobbies?’
At first I felt a bit awkward asking this, but a former User Researcher colleague swore by this question and we all respected her expertise. She was right – customers loved answering this! While it may not have presented us with product opportunities, it brought us closer to our customers. We learned that one had just retired and enjoyed watching our programming to pass the time; another was working night shifts so they could save money to go to culinary school; and another had our programming on in the background while they were wood-working to relieve stress.
Getting outside of your head and learning who you are building for has an amazing motivational effect; one that’s just different than hitting OKRs or a launch date.
For us, this exercise was totally worth it. We prioritized potential Product opportunities with input from Interviews (and other sources) and were able to convince our leadership that we were on the right track. And personally, it was gratifying to develop a relationship with individuals that use our Products regularly.
Interviews are one of MANY inputs to the Product Development process, but one that I feel is easy to ignore. Products, especially digital ones, produce so much data that is readily available to analyze. Interviews and the information from them, on the other hand, aren’t produced automatically. It’s hard work and the process isn’t particularly scalable, especially if you don’t have someone dedicated to it.
You can get pretty far by just looking at the data and forming hypotheses around them. But, like we were, you may find yourself in a place where you find yourself just chasing after numbers without a real, human problem to solve. We found that speaking to our customers gave us direction.
I hope I’ve shown you that it is possible to do regular customer interviews, even if you don’t have much experience. And like anything else, you can and will improve with practice.
Best of luck and please reach out with any questions!
Special thanks to my colleagues Mikaela and Natalie for partnering with me on this initiative!
We’re headed down the stretch run of the NBA Regular Season so I’m writing about one of my favorite things on the internet, LeagueFits. LeagueFits chronicles the NBA’s best-dressed as they walk down their ‘runway’ -the tunnel between the stadium parking lot and the locker room. I love LeagueFits because it gives us an intimate view into a player’s personality. On-court demeanor and playing style communicate a lot about creativity, but fashion sense says even more. Like Lebron says, they’re more than an athlete. Seeing another side of our favorite athletes only serves to increase our fandom. I know I’ll never be able to play like these guys, but LeagueFits gives me the sense that I could maybe, just maybe dress like them 🙂
But how did the NBA get the fashion bug and why is this Instagram account such a big deal to me?
At the start of 2005/6 NBA Season, then-commissioner, David Stern instituted a dress-code for players participating in Team or League Activities. The goal was to improve the image of the League, which had been damaged by events such as the infamous ‘Malice at the Palace.’ There was some initial resistance to the rule, but it pushed the Players to elevate their style and 10+ years in the NBA is as much about fashion as it is basketball. The elevated fashion may have come about regardless – I don’t think anyone looked good in the early 2000s – but many think the dress code was the big difference. In a 2014 interview Dwyane Wade commented:
“Then it became a competition amongst guys and now you really got into it more and you started to really understand the clothes you put on your body, the materials you’re starting to wear, so then you become even more of a fan of it.”
If you follow the NBA at all, you’ve definitely noticed this. It’s commonplace to see LeBron James, Russell Westbrook and James Harden wearing Gucci, Givenchy, Saint Laurent on GQ covers and out on the town.
LeagueFits is the brainchild of its parent brand, SLAM Magazine. One of the things I find so interesting about this is that LeagueFits was born on Instagram and not the SLAM website. It’s Instagram Native Media and a it’s a great pairing. A photo is the perfect way to capture a Fit and Instagram and NBA share so many qualities: culture-creating, style-craving, youth-laden.
The fact that it started on Instagram and not SLAM itself is consistent with the growing centralization of the Internet. While I generally regard these walled-gardens as problematic, I don’t think LeagueFits could have taken off without something like Instagram. I’m into fashion and the NBA, but I really don’t know if I’d go to SLAM online or even a dedicated Tumblr (pre-acquisition) to consume this content. So as much as we deride centralization and harken back to the days of the early internet, things like LeagueFits may not have attracted many eyeballs (yes, I realize that I’m talking about an Instagram account of really rich guys wearing really expensive clothing).
But indulge me for bit and let’s see where this can go. Instagram seems to be headed the direction of e-commerce platform and a brand like LeagueFits would thrive in such a space. Shoppable Tags already exist on Instagram and a curator like LeagueFits could do significant business through affiliate marketing. It will take more than a ‘buy-button,’ but that’s where the brands like LeagueFits come in. They’ve created a following by curating an experience – a fun and aspirational one at that. Their followers trust them and trust Instagram enough to purchase something through their feed. If done in an authentic way, Instagram can help creators monetize their posts without alienating their fans. One of my favorite Twitter follows, Jeff Morris Jr., offers a compelling vision for this:
Maybe the ‘marketplace’ is a standalone App or separate section of the App, leaving posts in the Timeline less cluttered. It will be interesting to see how FB/IG compensates the creators/curators (sellers) given their history of pulling out the rug from underneath them.
“The opportunity to build a major commerce platform that satisfies a different set of consumer needs than Amazon is particularly exciting. We believe that consumers’ desire for fun, experience, connection, and community remains strong despite the growing access to speed and convenience.”
A creator like LeagueFits could absolutely thrive in this model.
Anyway, can’t finish this post without highlighting some of my favorite Fits! So here we go:
Trying something new and will ‘review’ a book for this post. I recently read ‘The Design of Everyday Things‘ by Don Norman and really enjoyed it. My intent in reading was to level up my design skills and gain empathy for how designers think about problems and solutions. In reading the book I realized it contains many lessons for Product Managers and I’ll detail the ones I found most valuable here.
The Psychopathology of Everyday Things
‘Great designers produce pleasurable experiences’ is how Norman opens this chapter. Here he sets the tone that design isn’t just how something looks, but how people remember their interaction. If it’s positive, there’s a good chance they’ll come back to use your product. And here we’re introduced to the fundamental principles of design that will enable these pleasurable experiences (some examples from the book, some my own).
- Discoverability -figuring out what actions are possible
- Results from the next five principles
- Affordances – a relationship between properties of an object and the capabilities of the actor that determine how the object can be used
- Example: A chair ‘affords’ (‘is for’) support and affords sitting
- Signifiers -ensuring discoverability of how to use/what to do
- Example: think of a carousel of photos in an iOS app with the small indicator dots (page controls)
- Constraints – guide actions and ease interpretation; can be physical, logical, semantic, cultural
- Example: The door to your home has the physical constraint of only allowing keys to be entered in a certain orientation
- Mappings -relationship between controls and their actions
- Example: an automobile seat control that is the same shape as the seat – push up to move the seat up, push back to move the seat back, etc.
- Feedback – continuous information about the results of actions performed
- Example: a loading indicator on a web page/app after you click a buy button and then a confirmation dialog
- Conceptual Model – providing the information needed to project understanding and a feeling of control; can often be inferred from the device itself
- Example: a pair of scissors; the holes are there to put something into- their size indicate it can only be fingers; the blades are sharp and clearly not for holding
Keep these in mind next time you sit down for a design review with the team. Instead of asking to make the button bigger, consider how these principles are used or not used. And maybe suggest improvements to make them stronger.
The Psychology of Everyday Actions
This chapter is concerned with how People are able to figure out what to do with technology and what happens when things go wrong. Norman provides the Seven Stages of Action as a framework for what’s going through a user’s head as they interact with a product. He also groups these actions by whether they help answer questions of execution (feedforward, #1-4) or whether they hep answer questions of what has happened (feedback, #5-7) They are:
- What do I want to accomplish?
- What are the alternative action sequences?
- What action can I do now?
- How do I do it?
- What happened?
- What does it mean?
- Is this okay? Have I accomplished my goal?
He places the burden on the designer to make sure the product provides the information necessary to answer each question. This ties back nicely with the fundamental principles as each of them can be used to answer a user’s question as they progress through the Seven Stages of Action.
We’ve all been in work situations where we attribute poor reviews to ‘user-error’ or say designs are good enough because ‘people will just figure it out.’ We do this to absolve ourselves of the responsibility and this drives Norman nuts. He says these ‘errors’ are more often than not a result of poor design and should really be called ‘system error.’ Human errors will occur and designers should anticipate them; using the fundamental principles is key in minimizing these errors. And when errors do occur, because they will, the errors should be as cost-free as possible.
Human Error? No, Bad Design
Norman furthers the error minimization argument by discussing the relationship between Humans and Technology. Humans are versatile and creative while Technology is rigid and precise; their collaboration can be a beautiful thing. He illustrates this with a Human + Calculator collaboration – humans figure out what’s worth solving and the calculator can precisely produce the solution. This reminds be a bit of the ‘Steve Jobs bicycle quote,’ where he calls the computer a bicycle for our mind. It heightens human performance to levels never known (Norman actually worked for Apple).
Difficulty occurs, however, when we don’t think of humans and technology as collaborators. Norman says this that this ends up requiring humans to behave in machine-like fashion by doing things like monitoring machines or repeatedly performing actions that require great precision. And when humans fail at these tasks, they are blamed for them. Errors will occur, but design should anticipate these and make them as inconsequential as possible. His principles for dealing with errors are:
- The knowledge required to operate technology should be knowledge in the world – this is NOT the knowledge acquired by extensive teaching or practice; think of the iPhone touch-screen keyboard. When introduced, people were already accustomed to the location of letters on a physical keyboard so this didn’t require specialized knowledge
- Use the power of natural and artificial constraints to ‘force’ humans into performing the correct action
- Make things visible for both execution and evaluation – make it apparent what actions can be performed (feedforward) and clearly communicate results of what happened (feedback)
I read the Revised and Expanded Edition so I don’t think this chapter was covered in the original, but I’m glad Norman added it. He discusses best-practices such as defining problems before designing solutions, double-diamond model of design and human-centered design process. But he then quickly turns around and says that real world contexts don’t allow for any of this! Norman gives an anecdote of how a designer told him there are only two scenarios in which new products are actually introduced at their company: 1) adding features to match competition and 2) adding features driven by some new technology
I think many of us working on Product Teams can relate. Norman says that market-driven pressures and an engineering-driven company drive us toward more features, complexity and confusion. And even companies that do try to solve for customer needs run into problems like insufficient time or insufficient money that thwart their efforts.
So what can one do when faced with this reality? Norman writes that the only way to avoid the crunch that prevents the ability to do good up-front design research is to separate that process from the product team (I assume he means separate from development cycles). He argues to always have researchers out in the field studying potential customers and products and then when an initiative is green-lit, designers already have recommendations. I interpret this idea from Norman as Product Discovery, which has been written about in great detail by Marty Cagan and Teresa Torres. It’s a continuous, non-linear process that requires you to engage with your customers and understand their problems and present solutions in a quick and cheap manner. Product Discovery can be an antidote to the market pressures and feature factories because you’re constantly validating with customers and know what will stick and what won’t. You then use the results of this discovery to prioritize the biggest opportunities and push back on requests that might just be noise (always easier said than done).
Design In the World of Business
Norman starts this chapter by talking about something I’ve constantly grappled with in my Product career – ‘featuritis.’ Featuritis is the continual addition of features, even if the product is already successful and useful to customers. Norman writes ‘..it is rare the organization that is content to let a good product stay untouched.’ There is always the urge to keep building in any type of business environment, even if there’s no clear customer value in doing so. I think a lot of this comes from an engineering mindset where people just want to keep building and use the salaries paid to developers to justify this.
What can one do? A couple things I’ve tried to do to mitigate this are 1) remove features that aren’t performing if you are going to add more and 2) use learnings in Product Discovery to at least focus on the right problems to address if adding new features. Norman offers some help by citing Harvard Professor Yougme Moon’s book, Different, where she argues to double-down on strengths instead of matching feature by feature or ‘competition-driven design.’ If you are to build, build on what you are good at and ignore the irrelevant weaknesses. Norman writes that this requires support and understanding from your Leadership, but in my experience you cannot just wait for that to happen. I believe Product Teams need to ‘manage up’ and educate Leadership on these issues. Knowing your customers and your product metrics will help in telling your story.
The Moral Obligations of Design
Morality of design is a hot issue as 2018 draws to a close. Norman laments the use of design to promote superfluous consumption of objects, he writes ‘we are surrounded by objects of desire, not objects of use.’ I wonder what he’d say about products that ask for our constant attention and use our personal data in return for providing a feeling of connectedness among other things? He offers no easy solutions and I don’t expect any. He’s simply laying out the reality that readers of his book will face as they try to put his lessons into practice.
Thinking About Design
Norman concludes that the complexity and difficulty of Product Development are what makes it so rewarding. Yes, designers need to make things that satisfy people’s needs, but design in only successful if the final product is successful. This means having to work within engineering constraints so the design can be created on time; this means working with marketing to make sure you also design something that people actually want to buy; this means providing assistance to customers when they become confused about the product. Product Development is complex and it requires technical, business and personal skills to be able to work with so many parties and satisfy their requirements. Norman says there will be peaks and valleys throughout the process, but great products overcome the low and end up high.
And Norman reminds us to fight for and appreciate great design! Designers should fight for better usability and consumers should tell manufacturers about products with poor usability. Think about battles fought to make sure your favorite products shipped with good design; think kindly of the people that made sure it happened.
Last week on Twitter I read that Union Square Ventures was leading a round of funding for a company called Scroll. I’d never heard of Scroll (it hasn’t officially launched), but after reading USV’s announcement I can’t stop thinking about them. Scroll will sell ad-free, subscription-based access to some of the biggest News Publishers in the U.S. (USA Today, The Verge, Vox, and more) for $4.99 a month.
What excites me about Scroll is that it’s NOT an App or Aggregator, but a ‘Facilitator.’ When you subscribe to Scroll you go to the participating publisher sites and read without the ads. Scroll believes its would-be customers are already using ad-blockers, and by sharing subscription revenue they are introducing a new monetization opportunity.
I love this model. For consumers, it harnesses the power of the open web and how they actually use it. So no walled gardens (and the associated misinformation and filter-bubbles) and a transaction model that actually values their time and privacy. For publishers, it eases the tension in allowing an aggregator to distribute your content. As I’ve written previously, the aggregator and the publisher often share the same business model and monetize users in the same way – through their attention. But this is where Scrolls turns that upside down. They aren’t trying to keep customers on their platform (there isn’t one!), they’re incentivizing customers to use the publisher’s platform. And then actually paying the publisher for that usage.
So much of what makes a successful product comes down to timing. Given the state of the internet, the timing seems perfect for Scroll. Can’t wait until launch.