How Customer.io Redesigned Their Onboarding

I recently had the pleasure of sitting down with Richard Hawkes, who is a product designer at Customer.io.

They just completed a big redesign of their onboarding process, and all the juicy details are available here for your listening pleasure!





The original article: Oh, the Features You'll Ship

Their whole collection of design thoughts: Customer.io Design


The Conversation

Samuel Hulick:

So as far as the conversation that I would like to have today is basically I read through your Medium post regarding the rundown of the project -- or projects, somewhere in between maybe -- that you've been working on. And the thing that stood out to me was I was just nodding my head the entire time and just saying “man, do I wish that people would approach onboarding like this more often.” And then I realized “Hey dum-dum, maybe you can have something to do with that. Why don't, why don't we try to, to get the, get the story out a little bit more.” So basically step-by-step walking through the process that you took so that other people can understand what parts might work for them, what parts might not work so well for them and, and just move forward from there. Does that sound good on your end?

Richard Hawkes:

Yeah, it sounds great. And that's, I would say kind of the reason why I wrote the post was I was kind of surprised at what we accomplished and I was like, yeah, we, people should probably work on this. Like, I feel like sometimes onboarding can be put in that box of it's just a checklist or it's just some tool tips and it's not it's so much more so hopefully my goal of the post is hopefully people would see it and be like, Oh, look at all these sort of like other things they shipped. Wow. I wonder if we worked on our onboarding, what might we discover?

Samuel Hulick:

Absolutely. And one thing that stood out to me, what you said just now is that you were, you yourself were surprised with what you were able to accomplish. Could you just give me a little overview there as far as what you set out to do and, and how you surprised yourself?

Richard Hawkes:

Yeah. So I was coming into this having sort of read user onboarding, tear downs, and sort of watch some of those talks by Kathy Sierra. I believe growth.design does some posts. And so I felt like I, I knew of improvements we can make. As far as what you mentioned about like efficiency you know, like what tasks could we bring sort of in line when you first enter the app? We have a lot of stuff where we kind of like link you out to deliverability settings to set up email as a channel when it's like, wait, maybe we could bring some of that stuff in line. So we're not directing people elsewhere. Like let's help them sort of find flow. So I, I thought we had a, there was some surface level stuff that was like, Oh, like we could do this, we could do that.

Richard Hawkes:

That would be helpful. And then in the process of doing it, and I would say, this is probably a, a credit to how we approached it as far as having a squad, which for us is a team of a product manager, a product designer, front end engineer, and a backend engineer, and sort of having a squad focused on an outcome. So we were trying to decrease the time it took for someone to send a message and then doing so you detach yourself from, Oh, let's make this revised version of the setup guide with these, you know, three to five things we know we can make more clear or better and you start to be like, Oh wow, there's a lot of other places where people sort of get off track and what can we do to help them there. And that's where I would say some of the stuff I was surprised by is, you know, we're making shit making improvements to the billing checkout flow. What happens when a trial expires? Yeah. A bunch of those things that, you know, we didn't, obviously when we set out, like we should improve our onboarding. We're not like, Oh, let's start at the checkout flow. That's not, not nearly what, the first thing that comes to mind.

Samuel Hulick:

It's funny because a lot of people feel kind of intuitively like their onboarding could really be improved and they have a sense of like, Oh yeah. I mean, there are these top three things come to mind, but then when you really start digging into it, you, you realize that you had no idea all the other 18 things that also were breaking down.

Richard Hawkes:

Oh, totally. And I mean, even in my head, there's still a list of things we could make better. But it's but yeah, it's, you know, you get in there and it's just like, you start to realize what people actually do and yeah, they just, there's so many other little things to improve and onboarding is one of those things that everyone goes to you can release a new feature and maybe you get now a certain percent of adoption for a feature, but for customers, you know, everyone does have to go through onboarding someone that purchased the product, had to go through the checkout flow. So what's that checkout flow easy. Was it difficult? You know you know, you start to bring in stuff like the peak end rule and it's like, was the checkout experience one of the worst parts? And like now all of a sudden they have a negative impression of it.

Samuel Hulick:

Yeah. Well, and especially right when they're like putting a ring on the whole relationship, that's, you know, that's not, that's not, when you want, you don't want your wedding to go poorly so to speak.

Richard Hawkes:

Yeah, no. Yeah. Especially there, it's just like, yeah. One of those surprising moments for us was the people that trial sort of was deactivated just cause it ran out of time and they actually like wanted to pay us. They're like, Oh no, we just sort of ran out of time, but we actually want to continue. And it's, they had to reach out to us to do all that. And it's like, man, they actually just want to give us a credit card and start using it. So let's let them

Samuel Hulick:

Yeah. Almost like you're playing hard to get in that sense. So, so for the people, for the people listening, let's see, maybe I should be a, a more articulate host here. So you started this project, was it about a year ago? Was it, there was a point where your company decided to place more of an emphasis on self-serve and a S a improving the experience for people who didn't want to go the sales approach? Is that effectively correct?

Richard Hawkes:

Yeah. So that, that happens right around like last fall, I'd say yeah, we sort of had this shift where sales was giving equal attention to all people that signed up for the product and, you know, speaking of efficiency you know, we could probably make it more beneficial for sales to focus on certain customers that actually wanted to talk to sales. And there's always a subset of customers that don't where B2B product for other people, building products and a lot of people like to be self-served and like to figure it out themselves. And so how can we better support those people? And then thus free up sales to focus on the people that do want that human touch.

Samuel Hulick:

And, and there was a point where it sounded like you had a company offsite or something where you put together a presentation, like an internal pitch for why you should be focusing on onboarding. Is that correct?

Richard Hawkes:

Yeah, sort of. So twice a year we have so customer we're a fully distributed company and twice a year we have retreats. And in them you, you can sort of, if you want to just give a presentation about any topic, it doesn't have to be work related. Actually in the same, I actually presented after someone that talked about bog bodies. So if you ever want to look that up just be aware but they really can be about anything, but I having sort of a passion for onboarding was like, Oh, this is my chance to sort of one for myself, force myself to summarize sort of all this sort of research and passive learning I've done and try to articulate it in a presentation. And two, if we have a, we're gonna work on onboarding, it'd be like, Oh yeah, clearly we should do that.

Richard Hawkes:

Like this person, like already has a good, fundamental understanding of it. So that was actually last spring, the spring of 2019. And then this sort of shift in sales came about in the fall of 2019. And that's where as a company, we sort of aligned around it and sort of, if we're going to have that shift in sales, it's like, let's put a squad on this and really make sure that if all of a sudden sales isn't going to talk to some of these people that they're not, you know, left out on their own and they actually have a path to be successful on their own

Samuel Hulick:

Boy that's music to my ears. And so this was just to make sure I'm getting the chronology correct. You had presented your summary of where you were coming from with onboarding, and then it was four to six months after that, that you actually were able to start in earnest. Is that correct? Or was it the other way around? Yeah,

Richard Hawkes:

No, that's correct. Yeah.

Samuel Hulick:

Well, I guess if that makes sense. And now that I say it out loud, it sounds kind of dumb if it would be the other way around, but fair enough. Well so as far as the, the main focus of the project, when you did start in earnest one thing that really stood out to me that you mentioned is that you were focusing on a result rather than focusing on onboarding as a feature. Like we just have to ship this particular scope of, of predefined solutions. Could you touch on a little bit, like what went into your decision to focus on the result and then specifically why you chose the, the time to first send as the result to focus on?

Richard Hawkes:

Yeah, so sort of the result or outcome focus is just something we do on our product team. Okay. So our product team, we have multiple squads. And so right now I'm now on a squad working on something else and sort of, we still have an outcome tied to that. So that's just sort of our internal product process where the squad themselves have sort of the freedom to figure out how can we achieve this result or sort of improve this outcome.

Samuel Hulick:

So you're results oriented by default were regardless of the nature of the project. Yep. Ah, gotcha. Okay.

Richard Hawkes:

Yeah. So it's, it's awesome as a product designer, cause it sort of gives you that freedom to sort of, you know, figure out the problem and be like, Oh, like there are a million ways we can approach this. What is sort of that a nice mix of, you know, actually figuring out a smaller scope project to get there or something like that. So you're not spinning your wheels forever, but you can take multiple routes to get there.

Samuel Hulick:

Yeah. And I imagine also multiple attempts, it's, there's a, a certain arrogance in, in the software industry of thinking that the first thing that you ship is going to work exactly as you intended it to, which is a humbling thing to pay attention to if you do.

Richard Hawkes:

Yeah. So, I mean, we're going to like, I'll just jump ahead. But yeah, so that's a, I mean, especially with this product our free trial is, is 30 days. So one thing that we've done in the past year or so is that we'll do follow up reviews on things we ship. So we'll go back, look at some analytics just look at some metrics and how things have performed, but with a free trial being 30 days, and we're trying to decrease the time it takes to send a message in your trial, then you kind of need a full, at least cohort of a trial. I mean, at a minimum to really analyze this. So it actually gave us the freedom to, okay, we shipped that great. We still need about like another one to two months to really know how, you know, if this was successful or not, which also sort of freed us up to work on these other things that are still towards that outcome, even though they're not tied to the initial project

Samuel Hulick:

While you're waiting for the cohort to bake, so to speak. Yeah, exactly. Gotcha. And then, so as far as the specific result that you did focus on decreasing time to sending the first message, how was that something, how did that stand out to you as being the main metric to want to move?

Richard Hawkes:

So, yeah, so a couple of things went into that. For us there are a complicated series of steps to send a message. A customer is a marketing automation platform where you can set up sort of workflows to send emails, push SMS, and more. And so sort of to do the basic example of sending an email you're going to have to come in, you're going to have to integrate your data source to actually, you know, pull in these people and pull in events that they're performing in their app. Then you're going to have to sort of create a workflow. That would be, you know, what emails are going to trigger. You're going to have to go then set up your email so that we can actually send emails on your behalf. And then you're gonna have to go actually write the message content, and then you can hit send.

Richard Hawkes:

So that's a lot of steps. And they're also not as straightforward cause we'll have people that want to join that are maybe they're the marketer that's responsible for writing the content or maybe they're the engineer that's responsible for integrating the data, or maybe they're a marketer who's technically savvy or a developer. Who's a solo founder and they're actually responsible for everything. So the are, so we really wanted to, if we're going to remove any human touch, we wanted to make sure that they could still complete those steps without having to interact with someone. Whereas at the current state it was pretty sort of, I guess, not as straightforward. So decreasing the time to send means that we'd probably have to make those steps more easily attainable in order for them to actually send a message and then also to sort of lower that time. It actually took to send.

Samuel Hulick:

Yeah. And so as far as the time to send itself, was that a metric that you had been tracking leading up to the project or was that something that you went back and sort of figured out what your baseline was? So you could determine whether you had improved upon that or not?

Richard Hawkes:

Yeah, so we implemented a mixed panel for some analytics. I want to say maybe it was a month or two before this, so that's, so the baseline was sort of just a month or two of data. I believe that's right. And so that's what we sort of went off of is, you know, what people actually, and it was one of those things where it's kind of surprising about you starting to see how many people actually were able to do this within a trial. And you, you understand why some people can't like, you know, one thing that's big with customer IO is there's this sort of connection. Internally that someone, a marketer might need to get developer resources to integrate their customer account, but that developer is working on actual product features. So how do you get their time? So you can see why that person might not make progress for two weeks because they have to, you know, maybe submit a ticket to get in sort of the developer's queue and something like that. So

Samuel Hulick:

There, there are internal bottlenecks.

Richard Hawkes:

Yes. Yeah, exactly. So, yeah, there's a lot of reasons why someone might not send a message in a free trial, but that just sort of gave us a window. So if we were decreasing that we just knew that we were sort of at least Hasting that process gotcha.

Samuel Hulick:

Time to first end. Was that something that you were looking at with your baseline in terms of days or, I mean, I imagine that's not something when you're talking about two week bottlenecks that people have to face internally. You're probably not measuring that in hours right away.

Richard Hawkes:

Yeah. So this, yeah, definitely, definitely in days. And also some of it is a little bit if, cause some people might not, so it might be more than 30 days. But yeah, definitely in days.

Samuel Hulick:

Gotcha. Okay. So you're just basically saying the clock starts running when somebody creates their account probably. And then you want to see, how can we shave that down from eight days on average to six days on average or something along those lines. Correct. And

Richard Hawkes:

And we see an interesting mix of, there are some people that like, and I think this is speaks to the solo founder. That's responsible for everything. Some people do it within a day. Some people can do that. Now if you have to configure your DNS settings, that can just take a day or two to propagate on the internet. But but some people will do it in less than a few days and that's awesome, but other people sort of drag on. And so we also have looked at now that we sort of better metrics since we launched, we also look at what people do within their first session, because that's also, I would say an area for improvement for us going forward is we know people need to complete these steps, but we also needed to figure out, you know, are they, what are they able to do right away?

Richard Hawkes:

For example, if you're the marketer and you're responsible for creating con message content and you don't know how to integrate your data source, like why am I seeing that as the first step? I'm probably not gonna be able to complete it. I don't know how to maybe I'll reach out to that person. They'll do it for me, but I probably can't do that. So are they creating a campaign first and then they'll go sort of backwards to then get the data integrated after the fact. So we started to look at sort of those steps individually to see what was completed more often within sort of our first session.

Samuel Hulick:

Yeah, that makes a lot of sense because I mean, one of the names of the game, so to speak with onboarding in my opinion, is figuring out what it is that people are trying to do and then orienting and aligning your product experience around helping those people do that. And to your point, if you're working with a multidisciplinary team and you're not sure which discipline that particular team member is trying to make progress with that can really leave you shooting in the dark.

Richard Hawkes:

Totally. And that's, I think that's probably our biggest opportunity going forward. Is going back to the, sort of the decreasing time to send is, you know, what is sort of the long term, I guess, value of customer. Like when do people really realize like this is the product for them. And I think where we Excel is our customers that want to sort of push the boundaries on the workflows they create and sort of the customization there. Just yesterday I was talking to a customer that has a simple one off email transactional message, but it was actually four because he was branching off of the language the person spoke. And then also they have settings in their app if they prefer to receive that in push or email. So essentially a single transactional message was actually for, and, you know, as a practice owner, that's music to my ears.

Richard Hawkes:

It's like, if I'm that user of that product, Hey, I'm actually going to get a message how I want to get it. That's awesome. So I would say probably that when people realize the value of customer, it's probably when they're creating that workflow and they're like, Oh cool. So I can create this crazy workflow that I've sort of dreamed up. And that's sort of the people we Excel with is those that like us for our flexibility. And then on top of that is then they get into the composing like, okay, this is how I use all this data. I have to, you know, go beyond the whole hi dot first name thing. But actually you know, make sure you're sending on their sort of preferred channel or, you know, the, the, the content itself is maybe dynamically generated and, and things like that.

Richard Hawkes:

So that's where I think we're maybe getting to going forward is seeing that, Oh, the majority of people, the first thing they do is create a campaign. So maybe, you know, that's something to look at going forward is, should we make that the first step or should that maybe just be a point of sort of like the first run experience of like, Hey, play around with this workflow. And then once you feel like, Oh, this is something I really want now here is those five steps to get you set up and you can start building your own workflow.

Samuel Hulick:

Yeah. Gotcha. And so maybe if we could take it, take it a step back here, as far as just working through the chronology of how you wound up getting to the realization that you have, it sounds like you, you went through a lot of different experiences along the way, starting with, I think just creating a new setup guide was that like the initial scope, so to speak of the, of the project when you started out and then it, and then it kind of changed over time.

Richard Hawkes:

Yeah. So we had an existing setup guide that again had some of those easy wins, referenced email first and then had some other channels later, but we're more of a multichannel product. So it's bringing those other channels, like push and SMS up to, up to the standard of email and sort of giving the option to pick which channels you want to set up. So there are some steps we could tweak there and that's sort of, we knew where we wanted to start as far as here's the path when you're ready. If that's now, if that's in two weeks, here's the path. And if you do this, you're ready to go. And so that's where we wanted to start. Cause even if we increased sort of motivation on the front end to be like, got people really excited, if they had to go to our old setup guide or something, they'd still have to reach out to us because it was still a little complicated to get set up. So we kind of had to fix that problem for us.

Samuel Hulick:

And when you say P a setup guide, do you mean like a, like a PDF that people could use in parallel to using the product? Or is that like an in product guide or sort of a form? Did that take,

Richard Hawkes:

Yeah, so it's a in product guide, that's sort of a, an interactive checklist, if you will. There are four steps or five steps first one's already created when you log in cause it's creating your account, which I'm sure you might appreciate start someone with a little wind. Yeah. Yeah. And then sort of each step we try to let people create stuff in line. So they, you know, pick which channels they want to set up and start to enter in their sort of like Twilio credentials, if they're going to do SMS or something like that. Or sort of, whereas before the setup guide was essentially linking out to the settings, so you'd have to go into, and that example, you have to go into your SMS settings, find the Twilio and just sort of go to the settings to add your sort of credentials, whereas this was bringing that stuff in line. So in theory, you could just sit here on the set of guidance, sort of get through it in a one place.

Samuel Hulick:

Gotcha. So the, the setup guide was kind of like a ride along widget that was like a, kind of a, to do list checklist sort of a sort of a thing.

Richard Hawkes:

Yup, exactly. Yeah. So we didn't really we didn't use tool tips to poke people around the different pages, but sort of everything was just on that sort of one page. Gotcha. and was actually on a sort of on top of our, what would be the dashboard that sort of is where you would have, you know, you started to get your metrics and stuff like that. Once you send a message. So we've kind of pulled it out from its own page and put it on the dashboard so that if you sort of scroll past it, you would see what your dashboard could look like in the future. Sort of to try to provide a little bit of incentive for them to be like, Oh, okay. So I'll start to see like my sentence metrics and all that stuff once I complete this.

Samuel Hulick:

Yeah. But also not that you're just dumping people into a completely empty dashboard with a bunch of you don't have any data messages and things like that.

Richard Hawkes:

Yes. Yeah, exactly. Yeah, we've all, we've all seen those. Right.

Samuel Hulick:

And so that was also something that you, that you worked on as far as empty States and other parts of the app as well.

Richard Hawkes:

Yeah. So I would say after that we then move to the signup flow and that's sort of, because we wanted for the set up guide, we want to sort of have that time to sort of learn like when needed some cohorts to go through it and start to recognize what was going on. So the signup flow was, I would say the one, the second of the two known things we wanted to do. And that's where I actually gave sort of an internal presentation at a company meeting that was similar to a tear down where I had our existing signup flow. And I have little sticky notes that sort of popped up and was like, Hey, here we have email confirmation that directs people to the inbox. Is there any way we can sort of, you know, do that after the fact? And so people don't have to leave from this flow. And so the second step was improving that signup flow and sort of that's where we started to ask people like, what's your primary messaging goal and sort of there, and also get a couple of details, but company, but just try to make that form or flow as sort of streamlined and efficient as possible.

Samuel Hulick:

Yeah. That's a great point that you bring up as far as people leaving the app. I mean, when you look at a path to value in, in the case of yours, which is complex of getting people all the way to sending their first email, it's really interesting to look at what that process looks like and what the, what your exit and entry points, or I guess, exit and reentry points would be where people have to go off and change their DNS settings or confirm their email address and things like that. And try to think of how you can not keep people in your app too, to sequester or trap them there, but how you can ease their, their path as much as possible by providing assistance within the context of what they're already trying to do within your own product.

Richard Hawkes:

Yeah, totally. So that was one where we sort of looked at the data and on sort of our, I guess you're the old flow each step sort of had a drop off rate of about 15%. So it's about 85% conversion. And there was an email confirmation step that was dropped off another 10%. So it was like 25% drop off. And it's all because, you know, they're going to their inbox and now they're distracted who knows how long it takes for that email to get there. And so when we sort of did the new flow, we all steps where now in the nineties in the last two were like 98 and 99%. Cause they're kind of just like, Oh one click and I'm done. And the email confirmation was happening after the fact and looking at some of the, some of the accounts that didn't confirm their emails, it was kind of obvious that it was maybe some less serious Gmail accounts or something like that, where it was sort of people just, if people wanted to use the product, they went and confirmed, their email is kind of the results that were happening.

Richard Hawkes:

So it was good to get everyone in the product that wanted to, and then the serious people really sort of took the extra step to go confirm their email.

Samuel Hulick:

Yeah, that's interesting. I mean, it's, first of all, congratulations on getting up to 98 and 99% conversion rates. That's, that's outstanding for one thing. But then specifically, as far as the email confirmation step is concerned, are you saying that you, you used to require it very early in the process and then you were able to delay the, the pay walls, not the right word, but what you, you were just able to like block people from doing particular things if they hadn't convert confirmed their address later, or how did you address the timing of that?

Richard Hawkes:

Yeah, so we sent the email while they're in the middle of the signup process, but when they got into the app, there's a sort of a banner at the top that says, confirm your email or resend if you need to, but where it's sort of we're the key point where it blocked functionality was if you wanted to actually send an email. So now you do have to set up DNS settings and all that, as I mentioned before, but you also have the ability to send a test email. And so that's where one where you can send to your, just to yourself, but we do require email confirmation in order to do that. Other than that, you can sort of navigate around the app and sort of start creating a workflow if you want. That's fine. You just, you won't be able to send a physical message.

Samuel Hulick:

Yeah. And so you were saying just a minute ago that the new setup guide and the empty States based off of your own kind of a in house tear down that you had made were the two knowns that you had going into this. So something like sending a test message to yourself. Was that something that you found only after diving in and really paying attention to what people are trying to do? Or was that also kind of on your radar at the beginning as well?

Richard Hawkes:

It was, I think it was something we knew that people wanted to do. And that's, I'd also get to with the empty States as well as it's something that like I knew our, you know, maybe our empty States were great. They just said no data or something like that. But I think once the squad was starting to focus on this outcome, you really started to figure out how to prioritize those things. So it's, Hey, everyone is getting to this step of trying to create their first message and naturally they want to send a test version of it to themselves. And so that problem just really comes to light. Once you sort of done those other pieces of work and it's like, okay, this, like everyone is getting stopped here. Like, this is the reason why they're not being able to complete that step. And the setup guide is because they're trying to send a test and they can't, and they haven't, you know, configured their email settings yet either. So that's not gonna work either. So that's where it's like, okay, let's prioritize this and sort of shipping improvement there to let them send the test message to themselves.

Samuel Hulick:

Yeah. And did you see an effect on your, your, your user base when you made that change? Like, did people really start adopting that or was it only under particular conditions?

Richard Hawkes:

Oh no. People still love to do that today. I think the big thing for us was just letting people do it earlier. So once they confirmed their email versus before you had to verify your whole account and get everything set up, whereas now we sort of made it a little bit easier to do an early on and yes, everyone, it allowed, I would say it allowed when we talk about sort of the different types of people that use our product, it allowed someone to continue to make progress in the app. Even though they weren't going to be responsible for integrating the data, but they did need to set up all the messages. So they were now able to continue to create messages and sort of be confident in them so that when they did get those developer resources to integrate their data, they were sort of ready to go.

Samuel Hulick:

Gotcha. Gotcha. And that sort of is a convenient segue to another thing that you mentioned as far as creating segments, where once you had, maybe you've built up your confidence by sending a first test message or a couple test messages to yourself at some, you still have to send them to other people, but creating the segments that decide who the recipients are, was a bit of a challenge too.

Richard Hawkes:

Yeah. So that's again, have to give credit to a full story. So we, we use full story to watch sessions. And that's not something where people were sort of explicitly writing to us saying, Hey, like, how come I, like, I'm trying to, like, no one really mentioned, like, I'm trying to set up this campaign, but I can't create a segment. The issue with that is that to create a segment, they had to navigate to a different page in the app. But once you watch a couple session, recordings of people get to the campaign page, they just immediately go there. They're on this page where they're trying to set up this trigger and how like it's going to trigger off this event. And I want sort of, or the segment I want these people to receive. It's like, they're like, wait, where you just, you see the cursor starting to drift away.

Richard Hawkes:

And you're like, Oh no, they don't, they don't know what to do. And then, then you're like, you just sit there. You're like, Oh, that's right. Go to the segments page off to the left. Oh, wait, first note, save the campaign that you were in the process of making now go to the, go to the second page, make the segment, go back to the campaign page and now continue. And so we were like, Oh, like, let's just let them create the segment in line right there on that trigger step. And then it will save as a segment. So it's, that's definitely more useful for the people that haven't set up any segments, but again, if you are new to the product, we pre-populate it with a couple examples segments, but you're going to want to come in there and create your own.

Richard Hawkes:

So it's one of those immediate things that you watch and you're like, Oh, that's yep. I can. Yup. I okay. Yep. I see why that's frustrating. Yeah, totally. It's not something that someone's going to like reach out to support about. But you kinda just have to, like with that squad focused on it, you're, you're there and you're watching sessions regularly to see sort of where people are getting tripped up. Whereas maybe beforehand, if you just sort of shipped the project itself and sort of moved on, you might wait to get like enough support tickets related to something to decide to go fix it. Whereas this is like, Oh, okay. Like this is stopping people right now.

Samuel Hulick:

It's interesting. It's it reminds, I have a little bit of a programming background originally, and it's kind of a similar process to just debugging code that you've written where you think everything should work. And it looks like at your work to you in your head to compiles, but then it doesn't really compile on the screen. And so in a similar way, like it might not be totally obvious from the outside that you need to help people over that little hump of getting their, their segments added. But then when you start paying attention to it, it is pretty straightforward observation to arrive at.

Richard Hawkes:

Oh yeah, totally. And that's something that we actually continued to do is so we, once, I mean, this is sort of a little bit off topic, but once a week we have what we call game film sessions, where we watch FullStory sessions and it's an hour long meeting and it's the open invite to anyone in the company if they want to watch and sort of you come in and just sort of pick any random sessions you want to watch. And then we sort of discuss them as a team. And see if we need to like, you know, plus one, any get hub issues or sort of if there's anything maybe like another squads working on, that's like, Hey, I found this, this might be helpful to you. So that's definitely a good source of learnings outside of, you know, just customer support tickets or something like that.

Samuel Hulick:

Wow. That's a really interesting company process. So you're saying you, it's a, it's a week long or not, it's not week long, excuse me. Weekly hour long watching session that you, that that's like invite a optional kind of a thing.

Richard Hawkes:

Yeah. And so we, yeah, so we typically get yeah, so I think anywhere from like five to 10 regular people, and so and you can kind of watch whatever you want. So what usually what happens is if you're on a squad and I, we just released a sort of a little feature, you naturally just want be like, okay, well, are people using it? Or you want to hop over there. And now that I'm talking to you the session is actually tomorrow morning and I'm going to go back and look up, look at the set of guide and just watch sessions there to see how, how new users are finding the product, not just set up guide, but new users in general to see how that goes.

Samuel Hulick:

That's awesome. Yeah, because one thing that I have found is even if you have a full story account or something similar to that, and you're able to glean these kinds of insights communicating them verbally within your organization is really different than having people just sit down and watch, like you were saying with that segment step where you see that cursor drift away and it's like, Oh no, it's a very different sort of emotional experience.

Richard Hawkes:

It is. It's very humbling as a designer to be like, yeah, it's right there. And then you watch something like, Oh no, they're never going to find it. And it's, and then you realize, and I'm sure you like reflect and realize you do this and other products, but you're like, that person just spent half an hour trying to do this task that like should have taken them a minute. And you're like, Oh, no, like let's, let's make that

Samuel Hulick:

Better. Let's make it better. Indeed. So, okay. So the next thing you, you focused on the new setup guide, fixing your empty States, sending the test message to yourself, creating segments in line. And then the next thing that, the big thing that you mentioned was looking at the end of the trial experience. Was there anything in between those two or was that pretty much the next conclusion that you arrived at?

Richard Hawkes:

Yeah. So this is where I would say

Samuel Hulick:

It wasn't

Richard Hawkes:

Like chronologically at this point, it was kind of just a mix of how things got organized and just sort of what you can maybe do to figure out these things. So this next section is sort of about just what people may be wrote in or again, where we could sort of also see stuff like, again the configuring, your emails settings, that's another one you, you watch people do it successfully in the setup guide. And we still have, are the email settings section of the app that are sort of the old way. And it's like, Oh, let's bring that over because people were successful with that. But yeah, a lot of the other things in that section where people were actually writing in about it and it's like, okay, like, yes, we can fix those. Like it's always good to always get to take a look at those

Samuel Hulick:

A whole grab bag of, of usability improvements and things along those lines. Yeah, yeah.

Richard Hawkes:

These ones, then they're also smaller things that, again, I think you may not have, you may not prioritize as like, we need to go fix the checkout flow, but you start to realize, well, we're here. Like, well, the reason they haven't sent a message yet is because they haven't paid and it's like, Oh, well, they're getting stopped here. And then you watch some sessions with some, maybe like unhelpful billing errors that like it's not to their fault. And it's like, Oh, well, yes, let's, let's go fix this or say like an incorrect discount being applied or not incorrect, but it was not obvious that the discount was being applied and it's like, yeah, why that person is going to obviously reach out and make sure that they're getting the discount before they purchase and before they can send a message. So let's what sort of get them checked out, get them set up and then they can send a message

Samuel Hulick:

Providing the relevant information at the right time.

Richard Hawkes:

Yep, exactly. So, and like, yeah, those sort of little things, they just they're in every product they're there. And sometimes it's just this project, like sort of illuminated that like, yes, we set up set out to improve the setup guide, but all these little things that like, yeah. A couple of tickets here and there, but they really do add up collectively it's okay.

Samuel Hulick:

It takes a village to raise a child kind of vibes.

Richard Hawkes:

Totally. And it's also, it's that sort of frustrating, humbling experience of watching someone struggle once. And yeah, you're like, okay, let's fix this. Like, cause the fix is usually pretty straightforward as to compare it to maybe some other projects or like a novel new feature where you have a bunch of exploration, but some of these are like, well, let's just make sure the discount is being visibly shown well on this sort of itemized list. Whereas before it wasn't as good.

Samuel Hulick:

Totally. Yeah. It's not like you have to create, you're not like the feature Oracle who's coming up with the new market leading feature to be added to the product from scratch. It's really just a process of going back and looking at things that you might have overlooked because they weren't really intuitive at the time that you were creating it.

Richard Hawkes:

Yeah, exactly. And so yeah, it also just, it feels good to sort of try to raise that bar in all areas of the product. I feel like that's just a thing with sort of product development in general. Sometimes there are areas of apps that can kind of get, you know, maintenance isn't kept up with and sort of, it's easy to chase the shiny new feature where sure.

Samuel Hulick:

Well, and especially to your, to your point earlier, as far as that the, the, the parts that you want to improve are also the areas where experiencing it by definition, because that's the first thing people experience, that's where their first impression comes from.

Richard Hawkes:

Totally. Yeah. So, I mean, that's like, I would challenge all isms this to, you know, go look at that stuff. Like when was the last time you, I mean, hopefully you were, maybe your company will reimburse you if you go pay for your own product and kind of like set up a sort of a fake account and go through the purchase flow and just check it out and see what it's like, can you make it through it?

Samuel Hulick:

Right. That's like an obstacle course almost. Yeah. And so, so speaking of which there was one part that, that it sounded like maybe you were kind of surprised once you started looking into it, but the, you have a 30 day trial is what you said, correct. Yep. And so, and then at the end of those 30 days, the person, regardless of how much progress they had made or how close they might be to, to actually making the sale, it would just cut them off and say your trial is over and good luck basically.

Richard Hawkes:

Yeah. Kind of admittedly, it's not a great experience. But we fixed it. But yeah, so you kind of would log on and it would be, or you wouldn't, you try to log out, log in and there'd be like, Oh, your account, your trial was ended. Like, please reach out to support to, you know, take the next steps. But when people did reach out, they were, a lot of them were just saying, Hey, like I like, yeah, I'm still interested. I want to pay like, I'm ready. So that was really the surprising pieces. It wasn't like, you know, you know, sometimes you think, well, they have 30 days. Like, you know, like if they're not gonna do it in 30 days, they're probably never going to do it, but it's like, we just give them the ability to sort of self reactivate the account and then go through the checkout flow and yeah. And 19% of those 77 per cent of people that viewed the cancel state, they reacted to their account. And then 19% of those actually paid for subscription. So yeah. So newfound money.

Samuel Hulick:

Yeah, absolutely. And I did the math and 19% of 77% is 15%. And so basically of, of everyone who hits the 30 day Mark in your trial, you've just essentially turned on 50, 15% improvement of your, of your revenue numbers essentially by, by pulling them in rather than turning those people into support tickets or something like that instead, which is basically just overhead.

Richard Hawkes:

Yes. Correct. Where yeah. Maybe some of them would continue, but yeah. It involves reaching out to someone. Then you've got to sort of go back and forth and maybe get someone internally to reactivate the account sort of temporarily and yeah. Kind of just mitigate all of that.

Samuel Hulick:

Yeah. So, I mean, that's not exactly couch cushion money, having 15% of everyone who hits the, the trial cancellation point or expiration point, I guess, is a better term for me to use here to, to convert, to paid automatically. Like that's, that's gotta be pretty meaningful to the business. Totally. And I would say that's

Richard Hawkes:

Where some of the biggest success of, or if you had asked me, like what made this sort of project successful is that we were aligned with the business. So like that's sales change, you know, we were aligned with sales and marketing and how they were sort of talking to customers. So we actually, I would say it helped us see growth both in these self-service trials, but like also in this other segment that did want to talk to sales. So having that alignment sort of internally with this sort of approach, I mean, it just helped everyone and everyone was on board.

Samuel Hulick:

Yeah. And it's wonderful to see how it can affect different parts of your own organization. Like taking more of a, taking some pressure off of customer support. Not only means that they're just not as, as burdened as they normally would be, but also that they can then take that energy and provide better support to people who really need it or really want to be engaging. Like you're saying.

Richard Hawkes:

Yeah. And I would like our support team is amazing. And some of the, like one of the reasons I love working at customer IO is that our customers are pretty sophisticated in what they ask. They'll come to us with some crazy campaign idea of some workflow they want to make happen. And like, those are the really cool tickets to see when you see someone on our technical support team respond to that sort of like a five paragraph answer. That's like details. Like you have three options, like this is what we'd recommend. And it's like, you want them to be able to focus on those. You don't want them focused on yeah. Something as silly as like, okay. Yeah. I'll reactivate your account. Okay. All right. Yeah. You want them helping the customers.

Samuel Hulick:

Yeah, exactly. Yup. That makes a ton of sense. And then one other, the really big thing that I saw that you mentioned as far as your process was concerned was also providing better campaign templates. So if the people are getting in and they are having a good trial experience and are just trying to get a better understanding of what kind of a higher level capabilities they can, they can take on with, with customer IO in their life. It sounds like you were, you were focused on painting a clear picture of that and by offering up premade examples, is that roughly correct?

Richard Hawkes:

Yeah. So we actually had existing examples, but they predated to a time when we didn't have any branching logic in our, in our sort of workflow builder. So you may come to the customer REO expecting, Oh, they have this, like they have branches, we can do some AB testing with that. You know, you can have these multiple splits and then you see these examples and they're all linear and it's like, wait, am I in the right place? So, well, one, we just wanted to update those examples to better illustrate sort of what you can actually do with the product. But two, we also sort of shifted how we sort of surface them. So they exist in like sort of the example in the list of campaigns where you can click into each, individually, but the setup guide we wanted to actually let you preview them and actually sort of look at them at a glance.

Richard Hawkes:

And so that's where you can kind of paint the picture of what that use case is, which sort of for further reinforces, like what the product is good for. So for example, there's an onboarding campaign right there, and you can sort of see that there's a trigger if they're in signed up and you only want to filter it. So it goes to, cause this is only for free customers. And then the goal of the campaign is to get them to be paying customers. So you can kind of see, Oh, this is an onboarding campaign just for the free customers that upgrade them to paid. And likewise, with the other campaigns, you can kind of take a look and see those at a glance and be like, okay, so this is kind of, what's possible. And now maybe I'll go start from scratch. It's, you know, it, we're not expecting people to use these, to, you know, create their own campaign and start filling in the content. That's, you know, every company is different in what they're trying to achieve with their workflow, but mainly just sort of provide sort of an inspiration to see what is possible without having to click into all the steps and be like, what is this branch? What is it like, what are these other items I can do? But instead they're just all there visually.

Samuel Hulick:

Yeah. So instead of them having to go through and piece that together, just by reverse engineering, the intention of your interface, they can just have the puzzle pieces put together for them. Yeah, exactly. And was it wasn't some, Oh, go ahead. Excuse me.

Richard Hawkes:

And we sort of did the same thing with some of our email templates too. So that's another case where when you're creating an email, you have multiple composer options, whether it's a, a drag and drop interface or rich text interface, or just like you want to code your email and drag and drop specifically, it's one of those things where you're uncertain of, okay, I have, I have my own email today. Like, can I recreate that? And so we added some templates for the drag and drop editor to say, Hey, look, yeah, you can do that. Like, here's this one with a full width image or here's one with a sort of, you know, three little, a three row and a one row split or something like that. So again, it's like, Oh, I do feel confident in picking the drag and drop editor because they have this template. That's kind of what I want, but like, I'm going to put it in my own content. And I know that it's at least possible.

Samuel Hulick:

Yeah. You're giving people that, that proof of concept without making them have to go in and manually assemble it themselves. Yeah, exactly. Gotcha. And so when you were saying also about the people coming in, who might be looking for branching capability, but that they were only seeing linear templates instead, was that also something where there was like a mismatch between what your marketing and advertising was selling versus what people were experiencing? Like were people expecting branching coming into it, possibly based off of how you were positioning your product, but then that wasn't really being fulfilled within the product?

Richard Hawkes:

Yeah. So I think this goes to I which I think can also happen in sometimes onboarding in general is you need to have that alignment across the product and marketing. So this was a matter of the product was fully capable of doing branching. So it's all fine to talk about it in sales and marketing, but we just had outdated examples that were from before a time when we were sort of released a branching as a feature that the examples just didn't have branches. So someone could easily be misled and be like, Oh wait, do they have a branch in, like, I see this, all their examples are just a straight line when it totally is possible, but it's like, it's just making sure you go over all those boxes of, okay, we are going to make improvements to the onboarding here. Like what else? Like, you know, what else is there? Like, you know, for example, like the email onboarding flow, like we worked with marketing on that as well, to make sure that was updated

Samuel Hulick:

Your own emails, the, the emails that you would send to your, your own signups, not the emails that you would have your sign up send to other people, correct? Yes.

Richard Hawkes:

Yeah. They were very they were very meta and there was an idea to make them even more meta which is sort of, which is fun to talk about. But they're meta in the sense that we, the emails, they each have like a screenshot of like one of those screenshots that talks, or one of the emails that talks about, you know, building your own campaign, it shows a screenshot of the campaign that they're in which you know, so they would actually like see that function. Cause the campaign, the email onboarding campaign is built in customer L internally. So it's like, Hey, look, here's the, here's like what our campaign looks like for you. And you're in it. It was like a URL. It was like a, you are here on like one of those sort of like information maps. Yeah,

Samuel Hulick:

Yeah. That is pretty meta.

Richard Hawkes:

Yeah. The alternative, the other metal option was to which we sort of talked about and actually gets a little tricky when you think about it is what if the person's own onboarding campaign for themselves was in their account. So they could actually edit it and like tweak it themselves and start to like, see how sort of the content, how the data is pulled and stuff like that. But it gets a little tricky with, well, who's sending the email, you know, you have to know again, verify this with this email settings. And so we had to sort of abandon that idea, but it was, it was a fun thought exercise.

Samuel Hulick:

Yeah, for sure. So with all of these different changes that you made regarding your setup guide, empty States, sending a text message to yourself, creating segments in line and have trial experience. So on and so forth, these were all things that you went in, either with the intention of fixing for better onboarding or discovered while you were in the mix of it. But, but all of this, it was, was with the primary focus of reducing the time to sending the first message. So I have to ask, I don't want to put you on the hot seat, but did, did, were you successful in bringing that time down?

Richard Hawkes:

Yes we were. So yeah, I don't have exact numbers on that, but I, yes, we did improve that. I will say that, you know, increasing the signup flow saw bigger success. So I mean, which is great, that's like sort of top of the funnel, if you will, like getting more people actually into the product. So that probably saw a higher degree of success than reducing the time to send, but yes we did. And I would say even better than sort of that is, we've sort of continued to see business growth from this sort of self service segment of our customers as well.

Samuel Hulick:

Interesting. Yeah. So when you say that it's succeeded on the time to sending the first message reduction effort, but it also the more top of funnel things just succeeded, I guess, in a more straightforward business way. Like, would that just be in terms of conversions and revenue or how are you, how are you determining that your efforts are working so to speak?

Richard Hawkes:

Yeah. So what would the final flow that, that was just a matter of, yeah, it's sort of a funnel of getting more people in the product, but as far as the business results in general, it's actually that sort of split the split of how we sort of look at revenue is we kind of look at those two sort of segments and they sort of continuing to grow since we released this work. And it's something we've sort of seen since the start of the year. So is it all because of this? Who knows? But it's definitely it's definitely a very nice sort of metric to see there was, you know, it seems, it seems like there was a spike and it's sustained. And so we like to think that it was a direct result of this, but yeah, I mean, that's, that's a harder problem and that's always a hard problem is trying to tie that revenue number back to, you know, that onboarding cohort and really try to like tie those two together. I mean, it's like a Holy grail that there's so many things that can impact it. It's hard to measure, but

Samuel Hulick:

Especially as you were saying, if you're, if you have a 30 month or 30 month, goodness gracious, 30 day trial period, then that's just going to your cohorts are just going to be so long that you might need to wait four or five or six months before you can really say that you've got the statistical significance in place to say, yeah, there's definitely an uptake here.

Richard Hawkes:

Yeah, totally. So that's yeah. So that's where it's, you know, it's hard to, it's hard to tell, but the early signs looked good seen as sort of the last of this work was sort of wrapped up. I don't know, three or so three or four months ago. Just took me that long to write a blog post about it. I know that feeling yeah. Wanting to do it and it was like, okay, I just need to, I just need to sit down and do it.

Samuel Hulick:

And, and it's also something where when you do focus on the flow, you don't necessarily have to see a direct correlation between those numbers and revenue. Although of course that's nice as well, but just mathematically speaking. Well, the math is on your side as far as, as far as that's concerned, because even in a, in a really simplified example, like let's say that you had a an email confirmation step as the very first thing that you had people do after creating their account. And you were losing 15% of the people who were doing that. And you can bring that down to only losing one or 2% of the people who are doing that by delaying that step or whatever. Then all of those people who would have been taken off the board at step number two are going to cascade down to the steps that follow that. So it's, it's kind of a numbers game in that regard as well.

Richard Hawkes:

Yeah, totally. So that's where it's like, yeah, yeah. If everything else is sort of going up, it can't be bad. Right.

Samuel Hulick:

Gotcha. And then, so you said you, you wrapped up this project about three months ago. I'm always interested in onboarding projects in general, but especially ones that are focused on, on attaining a particular result, like, like a decrease in time to sending the first message. How did you determine that you were done and that it was time to sort of tidy up the loose ends of that project and move on to a different project?

Richard Hawkes:

Yeah, so I would say that's where it came to just a discussion of what else, what else can we do? Which of course there's always more things we can do. But to what degree is that starting to maybe optimize for really small gains, where as a company, there are sort of other, a shift in strategy that we want to pursue. And at what point should the squad go focus on a new feature? So it was just a discussion. And we felt that it was like the things that we can continue to work on for onboarding would have sort of minimal gains and we'd be sort of optimizing like a little red or blue buttons type of level versus we could say, okay, it's time to go work on this new feature. And that's sort of what we're working on now, which will, you know, as much sort of impactful or a much more, a larger impact that we know of.

Samuel Hulick:

Gotcha. So you went in with a, with a couple ideas of what the low hanging fruit was, and then by in the process of addressing those found a bunch of other low hanging fruit, and then when you address those, you're like, okay, this is that there's only mid hanging and upward fruit from here, so to speak.

Richard Hawkes:

Yeah, totally. I'd say the big thing is, and it's also a little bit more of an unknown, but it's okay. So the people have the steps to complete this. If they want to, like, people can get set up if they're motivated, but what can we do in that sort of first run experience to make sure they are motivated and they do know it, and that's where it gets a little more hard to sort of quantify, or it's not as straightforward cause yeah, fruit's a little bit higher up the tree. Gotcha.

Samuel Hulick:

And so where does onboarding stand within the company from here? Is it something that you're continuing to keep an eye on or something that you plan to revisit in the future or somewhere in between?

Richard Hawkes:

So yeah, so I would say somewhere maybe somewhere in between I like we have a sort of metrics set up and sort of, I had a curiosity of, we'll go look at that. Sort of the point I made earlier now I'm motivated to make sure that in our sort of game film session tomorrow I go spend an hour looking at new user sessions to see how that's going. And if there's any glaring issues luckily someone internally actually just went through this last week just cause they hadn't done it in awhile. And they're like, Oh wow. That was really smooth. So it's, yes, it's an internal employee. That's familiar with our product, but it's also, you know, it's like, okay. Yeah, it's, it's, it's in a good place right now. But I'm sure there'll be some, some opportunities coming up where I'll be like, okay, here's a, a thing that we can do and I'll write a proposal for it. And maybe even be able to quantify what I think will actually do. And that will sort of maybe get it easier to get prioritized if that, if that's what we want to do.

Samuel Hulick:

Yeah. It does sound like you're that you, you you were able to seize the opportunity when it presented itself, as far as having that company-wide alignment around trying to bring the self-served numbers up. And so it became a really natural fit there. So on the one hand, it's nice to go in and, and create the sort of experience, smooth learners, like not locking people out of their accounts when the trial has expired and just getting to reap the automated rewards from that in perpetuity moving forward. But then also at the same time to have that working vocabulary within your company and also have what sounds like a few different successes that you've logged to be able to point back to and say, Hey, maybe we should go back and dip into that toolbox again.

Richard Hawkes:

Yeah, totally. And that's where, you know, the company sort of, Oh, like this was successful. So I mean, I think that's, again, I think with onboarding in general is there's a hesitancy to maybe work on it sometimes just because it isn't the shiny new feature, but when you can sort of prove to like, well, last time we worked on it, like this is what we saw, so we can point to that. And yeah, we continue to report on sort of those revenue metrics by those segments, like on a weekly basis. So if, for example, we ever start to see like the self service, you know, segment dipping a bit maybe it is something there that we can do. You know, why, you know, that's where we start to ask questions of why that is, and maybe it is related to the onboarding.

Samuel Hulick:

Yeah. And culturally you can, you can lean on the fact that you've already gotten some reps in there and it's not like you're going to be going in for the very first time.

Richard Hawkes:

Yeah, totally. So the sort of time the ramp up speed is much, much shorter because it's like, yeah, we've been there. And yeah, we all feel, we know we need to do. So let's just do it.

Samuel Hulick:

Yeah. Well, and on that note, is there anything that stands out to you that you learned through this process that if somebody else is trying to do some something similar to the approach that you took, you might be able pass along to them to, to help smooth the path for them as well?

Richard Hawkes:

I think one of my biggest learnings was actually just being surprised at all of the other seemingly smaller things we were able to improve. So that's, that's one where like, if I had to go make a case to prioritize onboarding in my company, like that's something that I would point to is like, Hey, like this person over here, they were able to, to work on this and like, look at all these other things that they improved along the way. And it's just sort of the question of like, what could that be for us? And so I think that's where like one of the biggest learnings I just wasn't expecting that were, you know, some of the more tangible stuff, like they should go over to user onboard and realize what they, what, you know, what are some actual, how tos on how they can improve their onboarding.

Richard Hawkes:

And they, yeah, they might just be surprised that some of those other things that they can help ship. And then I would say another learning was, you know, I gave that presentation at a company offsite three, four, five, six months before we actually had that sort of shift in sales and we were ready to work on it. But I have to believe that giving that presentation was like, Oh, okay, wait, we should maybe look at onboarding because this person is interested. So I know it's sometimes tough to find time to, you know, put together a presentation. But if you have an opportunity at your company to give an internal presentation about that type of stuff, like it can only help. You're sort of showcasing, you've done some research ahead of time. So the lead time to work on that project is going to be that much shorter because you're coming in with that sort of foundational knowledge of some of these concepts. And then you can sort of just focus on some of the user research stuff and digging into there.

Samuel Hulick:

That makes a ton of sense to me. I I certainly endorse that, that approach as well. And then as far as what's happening at customer IO and what's happening in your life, is there any, if people are interested in finding out more, do you have a good place for them to go to, to check those out?

Richard Hawkes:

Yeah, I would say that they could, they could check us out on a, on medium. So as a design team, we're trying to be a little bit more active there. So I think it's just medium.com/customer hyphen IO, hyphen design. But if they want to see what sort of the design team's doing there they can certainly search and find us there. I don't, I don't post too much on, on Twitter or anything. So good for you. I would encourage you to follow George you to follow us there. Cause also read some other great posts there too, that Madeline has written to about mental models and design principles and other things. So if designers are interested head there.

Samuel Hulick:

Cool. All right. Well, I am well familiar with the article that you wrote and I will certainly be linking to it somewhere in the description, I guess, of wherever this is described. And I, I will personally be going to check out the other articles momentarily, myself. Awesome. All right. Well thank you so much for being available and sharing your experience with everybody else. I think it's going to be really useful.

Richard Hawkes:

Yeah. Thank you. I appreciate the opportunity.

Samuel Hulick:

Cool. I guess we can just say we're, we're not on the air anymore. So --