UserOnboard EXPOSED!

Update 003: The Biggest Problem with (Most) Onboarding Experiences

I've been studying user onboarding for the last five years, and doing so has led me to observe a pattern repeating time & time & time again:

When teams find themselves focusing on an onboarding design project, they often actually wind up focusing too much on the design of the onboarding itself.

That might sound counterintuitive, but there's a lot to consider before you launch into an onboarding design project, and I often (very, very often) see teams skipping right past those considerations and heading right into the "UI touchpoints" part instead.

For example, the most frequent onboarding questions I see are usually along these lines:

  • How can we get more people to explore our product?
  • Do tooltip tours or intro videos lead to better engagement?
  • How many emails should we send during our free trial?

And very often, I try my best to steer people toward asking questions more like these instead:

  • What are the different reasons that drive people to sign up for our app?
  • Which early user actions most strongly correlate with long-term retention?
  • What has to happen for people to feel like they've made progress in their lives outside of our product?

In other words, I very often see people jumping straight into "getting users to do stuff" phase without first evaluating what "the stuff we get users to do" should even be in the first place.

Why It's a Big Problem

Approaching onboarding in this way is, at best, a wasteful use of resources.

From the user's perspective, it's a drag to have to "explore features" by playing tooltip tour whack-a-mole and then not even winding up anywhere meaningful at the end of it.

And from the business' perspective... well, let's just say that there are more effective retention strategies than getting people to do a bunch of things that they largely find relatively meaningless.

As John Wooden (the all-time winningest coach in basketball history and probably a super chill homeboy) was known to say...

Screen Shot 2018-09-10 at 7.48.22 PM.png

In that same way, you want to make sure you're focusing on helping users make progress in something that they actually value, rather than just clicking around because you told them to.

Why do I bring all of this up now?

Because that's exactly what I've been working on lately with UserOnboard's own onboarding.

Avoiding This Big Problem

In the last two updates, I shared how I plan on using "phoenix states" and lifecycle emails to act as personalized guardrails for guiding people toward the progress that they're seeking.

However, a crucial ingredient has been missing from both of these strategies thus far: clarity on what that intended progress actually is.

If I want to create an onboarding experience that actually drives engagement, I first need to figure out what's causing UserOnboard to be relevant in people's lives, so I can craft an experience to match that sense of relevance.

Or, to put it more simply, I need to figure out what it is that people ACTUALLY care about when they're caring about UserOnboard.

To get to the bottom of it, I put on my Sherlock Holmes hat and conducted an investigation by -- gasp! -- asking people.

Going Straight to the Source

I began my detective work by first picking a good time for popping the question.

I could have put an anonymous questionnaire up on the UserOnboard site, but I wanted to survey people who were demonstrating a little more urgency in their interest in the topic than simply being a fly-by visitor.

I also wanted to avoid blasting the entire UserOnboard list with a general survey, though, because some people have been following along for years and years (holla!) and the voices that I wanted to hear the most were from people who are caring about onboarding RIGHT THIS MINUTE.

That left "new subscribers" as a highly compelling segment to investigate: by signing up for the email list, they were sending a strong indication that onboarding was not only a top-of-mind concern of theirs, but a current one.

I wouldn't have to ask them to speculate as to why they MIGHT care about onboarding (like with the anonymous site traffic) or to remember why they HAD cared about onboarding (like someone who signed up years ago because they were working on a project and then just stuck around because they liked the content).

By selecting solely for new signups, I could just ask them what they were caring about RIGHT NOW.

And so, I did.

Asking Questions, for Fun and Profit

Now that I knew who I wanted to ask (and even more importantly, when I wanted to ask them), I could turn my attention to figuring out how I wanted to do the asking.

Since this was my first attempt at getting some high-level insights, I wanted to aim for volume over depth. This meant I could hold off on more time-intensive research formats like phone calls (for the time being), and do simple email outreach instead.

As a hardcore introvert, this was totally cool with me.

Taking some inspiration from the work of my friend Val Geisler (who really understands conversion-oriented emails and whose services I highly recommend), I landed on the following for the email copy:

Screen Shot 2018-09-10 at 7.36.52 PM.png

I liked the personal feel of the copy and I considered sending each message manually from my personal inbox, but I knew I'd forget or get distracted if I tried that route.

Instead, I coded up a robot to do it for me by auto-sending it to each subscriber 2 weekdays after they joined the list.

All in all, I sent a little over 2,000 "new subscriber question" emails and received a little over 200 responses in return.

Then came the REALLY fun part.

The REALLY Fun Part

With all the responses in hand, I could finally dig into investigating the main thing I wanted to know: the different reasons that people could have for caring about user onboarding. If I knew those really well, the "design" part would largely take care of itself.

I pasted each response into one big spreadsheet column and then sifted through each row looking for patterns. Whenever one would appear, I'd create a new column just for that pattern and move the matching responses from the big, uncategorized column into it.

I repeated this process until the "uncategorized" column was empty, leaving twelve remaining categories to further consider and play with. After giving each some thought, I wound up consolidating a few redundant ones and wound up landing on six major "causes" for caring about UserOnboard:

  • I'm Overhauling the Entire Product's UX
    ex: "We're completely redesigning our product and need to give the redesign its own onboarding experience"
  • I'm Launching a New Product
    ex: "We're about to bring a new product out of Beta and need to make self-serve signups possible (and perform well!)"
  • I'm Filling a New Role
    ex: "I just got promoted to a PM position and now I'm in charge of onboarding, so I'm brushing up on the topic"
  • I'm Fighting to Get a Company to Value UX
    ex: "The company I just moved to doesn't care about UX and I'm sick of seeing new signups flame out over and over"
  • I'm Striving to Hit Conversion Goals
    ex: "We just started tracking our retention numbers and what we're seeing isn't pretty. We hope onboarding can help with that"
  • I'm Gaining UX skills
    ex: "I'm a developer and don't really care about onboarding per se, but learning about onboarding helps me talk with designers"

I feel pretty good about how these shook out, and the categories align pretty closely with a lot of conversations I've been having with people over the years.

Each of them pass the "eye test", so to speak, so I'm finally ready to move on to the next step!

Ok, NOW Can I Build the Onboarding?

If I'm feeling pretty good about things, does mean that I'm done with user research and can move on to design now?

Nope!

These are still just high-level ideas of what people MIGHT be caring about, and in their current form, none of them would make for a very solid foundation on which to build a robust onboarding experience.

I still need to:

A) Validate that these are, in fact, actual patterns of motivation

B) Learn all the ins and outs of each of the ones that I validate

Only THEN will I be ready to build the "getting people to do stuff" part.

In the interim, though, I have a healthy base of hypotheses to work from, and each step from here will aim to solidify that position even further.

In fact, the next phase of validation is already underway on the site right now! *dun dun dun*

Stay tuned for the next update, and please share your thoughts in the comments below!