Podcast: Experimenting for High-Growth Products

MMS Founder
MMS Rachel Obstler

Article originally posted on InfoQ. Visit InfoQ

Subscribe on:






Transcript

Shane Hastie: Good day folks. This is Shane Hastie with the InfoQ engineering culture podcast. Today I’m sitting down with Rachel Obstler from Heap Analytics. Rachel is the VP of Product there. Rachel, welcome. Thanks for taking the time to talk to us today.

Rachel Obstler: Thanks, Shane. It’s great to be here.

Shane Hastie: Probably a good starting point is, who’s Rachel?

Introductions [00:25]

Rachel Obstler: Wow. That is an existential question. So, as you said, I run the product team at Heap. I professionally have spent a lot of time running product teams at large companies, small companies. More recently, I’ve really focused in B2B SaaS. I love selling to businesses of all sizes, as well as having the software as a service product. My last role was as a head of product at PagerDuty. I’ve also held some other roles like product marketing and head of a business unit, but I find that I just love product management so I keep coming back to it because it’s that great mix of strategy and research and working with engineering and working with marketing. It’s just a very central role where you have to do a little bit of everything, plus you have to be really good at convincing other people to do things as well. And so it’s a really fun role.

Shane Hastie: And tell us a little bit about Heap.

Rachel Obstler: Heap is a digital insights platform. It’s used by largely product teams, but also marketers, can be customer success managers, growth teams, to basically understand how users are using their digital assets. So anyone who is responsible for a digital asset, whether it’s a website, it could be a product, could be an e-commerce site, could be a SaaS product, all of those things that you as a product owner or a digital product owner need to know what are your users doing, which features are being utilized, where are they getting stuck, where are the opportunities, so you can decide what are the best investments to make to drive impact for the business. And so that’s how Heap helps companies. Heap’s special sauce is that we collect really all of that data. So any activity a user does in your digital asset, we will automatically collect. So you have all of that data at your disposal. And then we also have a layer of data science that we apply. So you have all the data, that’s a lot of data. We will surface the most important insights automatically.

Shane Hastie: You’re known for talking about product-led growth. What does that really mean to us?

Product-led growth [02:37]

Rachel Obstler: Well, I think if you look at B2B SaaS as an industry or the software industry, it’s gone through a number of changes. One is, and I know you talk about this a lot, but the whole idea of how you do product development. So moving to being more agile. And then it also went through this move to SaaS, right? It’s a move that’s still happening, but many, many organizations are in the cloud, right? So you don’t have to deploy software on your site and worry about version updates and stuff like that.

Rachel Obstler: And then the next frontier is really that all of those SaaS products are going to move, and many of them have or started this way, to being self-service. So what that really means is that someone can come to your website, they can sign up for a trial, they can try it on their own, they can buy it on their own, they can add more, more. You can buy more, whatever your dimension of more is. It really means that everything that happens, the concept of product-led growth is first of all that the end user is the one that you’re often relating to. And that all of this interaction that you have with your users as a company comes through the product or the majority of it, at least.

Shane Hastie: My experience and what I see is a lot of organizations get this wrong. They don’t really understand that end consumer and the stakeholders in the value chain. How do we help them?

Users, buyers and other stakeholders [04:08]

Rachel Obstler: In my experience, a lot of people get it wrong too. I think they get it wrong in a lot of different ways. That is one way that they get it wrong. But let me go back to this notion of the user and I’ll talk about the user and the buyer because I think that may be what you’re alluding to a bit, which is who are the people in the value chain? So there are users, there are buyers, there are stakeholders, there’s all sorts of different people that could be in a chain of the company that you’re doing business with. The whole idea of product-led growth is that yes, sometimes your user is your buyer. Sometimes your user is not your buyer. And so it doesn’t mean that you never want to have a sales team, right? So if you want to have an interaction just with a buyer who’s not a user, you should have a facility for doing that.

But the way the world is going is that the majority of people out there prefer to try and buy software on their own,right? So they are users that are also buyers. And so what that means is that, let me say it a different way, if you’re not serving the buyers that are also users, like the buyers that prefer to use in order to buy, then you’re missing a huge part of the market. And if you have those people out there that prefer to self-serve try and buy on their own and your competitor allows them to do it and you do not, you’re not going to get an at-bat. So it’s not to say that the only way to sell to someone is by self-service. But what it does mean is that if you don’t offer that option, you’re going to lose out on a vast majority of the market that prefers that option to start interacting with the product.

Shane Hastie: One of the things you mentioned was the insights and analytics. Data science obviously really important today, the analysis of all that huge quantities of data. How do we make it useful?

Using data analytics to gain insights on use and encourage sales [06:06]

Rachel Obstler: So let me take, since we’re talking about product-led growth, the example of a trial, right? So most product-led growth starts with a self-serve trial. In that self-serve trial, you have someone going on your website, they’re logging in or giving you some information to sign up, and then they’re getting typically dumped into a product. And then they are doing whatever set up they need to do. So it’s typically called activation to get to value. And then they’re either becoming regular users, to maybe be followed by purchasing at some point or not. And so there’s a couple ways of making the data useful. But the first one I would say is that you as a owner of that, maybe that trial experience, should have a framework of how you want to evaluate the success of the users that are trying your product.

Pirate metrics (acquisition, activation, retention, referral, revenue) [06:59]

Rachel Obstler: There’s a lot of frameworks that have been written out there. In fact, I do like the pirate metrics. That’s a very famous one. That’s over 10 years old, but it still holds up very well. Which is you have your acquisition, so it’s the people who end up at your website and actually sign up for a trial. Then you have your activation, so those are the users who go through whatever setup is necessary to get to value. And then you have retention. They keep using the product, they’ve referral, they tell other people about the product, maybe invite other users in. And then ultimately, revenue. They decide to buy. It doesn’t have to go in that order. But I think you start, first of all, with the framework. You need to understand the path or the journey of your users. And then you need to understand where they’re getting stuck.

I think that’s the best way to start. Probably, I also like to start the top of the funnel because if you can’t get people to, for instance, sign up for your trial, then you’re never going to get an at-bat to get them to set it up. And if they can’t set it up, you’re never going to get an at-bat to get them to keep coming.

Shane Hastie: From an engineering perspective, you mentioned the interaction with engineering teams. Organizations have different approaches in different phases of growth as well. What does the engineering manager working with the product team need to take into account?

Rachel Obstler: So is this like any product team, a PLG? Like a growth product team? Because I think they’re actually very different.

Shane Hastie: So let’s start with a growth product team.

Engineering and product interaction [08:29]

Rachel Obstler: A growth product team often operates in a different way than other product teams. I actually think in some ways other product teams should look more like growth product teams, and I think that will happen over time. But a growth product team is really focused on, again, optimizing a trial experience for instance. Whereas a typical feature team is focused more on, “Hey, there’s some lacking functionality. Or there’s this next step in the roadmap. We need to add this functionality a competitor has.” Or it’s just, “We’re early in the product and we’re still developing it.” And so it’s a very different motion, right? One is, “We’re building a new feature.” The other one is, “We’re really optimizing existing capabilities to make them super seamless and easy and straightforward and understandable.” And as a result, a growth team is much more focused on experimentation, right?

So let’s say you have a trial already and you know where users are getting stuck. You’ve done that analytics. You’ve looked at what activation is, what are the steps, where are users falling down. And you have to come up with… So you know where the problem is. You can sometimes know exactly what to do, but a lot of times it’s not clear, right? If it was obvious to everyone how to build an amazing digital journey, then everyone would just do it from the beginning and there’d never be any friction. So oftentimes you need to sit with the team, as say a PM and say, “Hey, here are the opportunities” and come up with some potential solutions and then run the most promising experiment. And if that one doesn’t work, run the next one.

And so to get back to your question, which is what does an engineering manager need to do and need to look like? For a growth team, I would say that it’s someone who is really familiar with that mode of experimentation and how to encourage people to take risks, feel comfortable with big swings, small swings, celebrate experiments that are definitive versus successful, right? So a successful experiment is actually one that is definitive, whether it is definitive that it worked or definitive that it did not work, you’ve learned something. An unsuccessful experiment is you did something and really you can’t tell what the outcome was.

So it’s a very different mindset and it’s also a different type of engineering. And so that’s another piece that’s important is, this engineering manager needs to know how to hire the right engineers who are really into that type of work. Like, let’s try something and see if it works. And we’ll be excited to try as many things as we can. It’s around the velocity of experimentation versus say, again, like churning out features. It’s a very different mode. And I’ve found that it’s a very different personality and focus for the engineering team.

Shane Hastie: How do you instill this way of thinking into the engineering team and create tighter collaboration with product?

Engineers need an experimentation mindset [11:41]

Rachel Obstler: There’s a number of different ways to do that. A lot of those are similar to how you do it in general for product. But I’ll say first, I hate to give this as an answer, but I will admit that in my experience it has not really worked that well to take people who were on standard regular feature teams and then have them now do growth, right? If you find someone with the right mindset, then it’s fine. But if you just try to take someone and say, “Hey, now you’re doing growth,” it is a different mindset and a lot of engineers don’t like it. And part of the reason for that is that, again, it’s about the velocity of experimentation. Usually what you’re doing is you’re not doing really deep technical problems. If you had a really deep technical problem that you needed to solve, that would take three months or six months, and it’s a really big thing. You probably would not do that on a growth team. So it is just a different focus.

And so in any case, how do you get the team to work really well around this? Well, some examples of things you would do is, again, you would focus a lot on the data, right? So where are the opportunities? As a PM, you can say, “Where are the opportunities in the flow? Where are the places users are falling off? Let’s brainstorm as a team with some ideas, some things that we could try to see if we can improve that.” And then you can create things like an experiment log or list, right? And then as you work through them, you want the whole team involved in evaluating, right? It doesn’t have to be the PM that does it. The engineers can do it. Let’s say one engineer is responsible for working on an experiment. They can do the evaluation of “Did work or did it not work? What does the data show?” They can present it to the team.

So I think it’s really having the whole team aware of what you’re trying to accomplish, the process that you’re going through, like this experimentation process that you’re doing. And then have them own not just the work, but also the outcome.

Shane Hastie: How do these teams integrate in an environment where there are, dare I say, traditional feature teams?

Integrating and working closely with marketing [13:43]

Rachel Obstler: Yeah. That’s a really good question. So let me first talk about the challenges and then I’ll get to how do you do it. So one challenge is that a growth team has to typically partner a lot more closely with marketing, right? So remember I talked about that acquisition step. Usually an acquisition step is owned by a marketing team. How do you get people to your website? There’s also a lot of value messaging in a trial. There’s also a lot of things that happen outside of a trial. You have to send people emails. You have to make sure that in a trial, maybe you bring them back in, like there’s nurturing that happens. There’s a whole set of activity that can be inside the product as well as outside the product. So it’s a much closer partnership with the marketing team.

And so it is actually I would say more important to have the team integrated with the marketing team than it is to have it integrated with the rest of the product team. And so I think the bigger issue is that, right? So what you typically have to work on with the product team is how to get them to work well with the marketing team, not how to be integrated with the rest of the feature teams, because they will naturally be integrated with that because they’re part of the same org, they’re going to see what the roadmap is. The product manager’s part of the product management org. The engineering manager’s part of the engineering management org. And so there’s a natural integration that’s there. You almost have to break that integration a bit and have them be more integrated with the marketing team. I think that’s the bigger problem.

Shane Hastie: How do you overcome that problem? What are the practical things you have done over the years?

Rachel Obstler: I don’t think I have a perfect answer for this. I’ll tell you some of the things that I’ve done that I’ve seen work well. So as I mentioned, you have to really work on that integration because they’re different orgs, right? The natural alignment is with the org that you’re in. So some examples. My current company, last company too, we have a process, a quarterly business review, right? One of the things we started doing is, you’d have a quarterly business review for engineering product designed together. You’d have one for marketing. We actually took the growth team that had some people in marketing, some people in products, some people in engineering and design, had a QBR as a group, because it’s super important to make sure that cross-functional team has aligned objectives, knows what everyone else is doing, understands what all of the touch points are and start to view themselves as a unit and a team. So that’s one way to do that is actually the way you structure your quarterly process.

Other things that we’ve done, we actually have like a weekly standup for that group. So similar to feature teams of standups, we have a cross functional weekly standup. What is everyone working on, quickly going through any issues? So it really requires making sure that you have operational processes that don’t silo, that bring the team together and force working together. And so when I talked about the QBR, I also talked about aligned objectives. It’s worthwhile spending a little bit more time on that as well. But making sure that the whole team is aligned on what the objectives are. And so, whether those objectives are, and I’ll talk about some good objectives, it could be the number of new logos that you’re going to get through like self-service, it could be improving your activation like improving your signups, but whatever those metrics are, they have to be understood by, agreed to by the team, and it’s clear that this team works together to achieve them.

Shane Hastie: This aligns very well with my thinking. Regular listeners to the podcast will know me from the #noprojects book, the concept of value streams. For many engineering organizations, this is a difficult shift. How do we help people on that journey?

Rachel Obstler: When you say this is a difficult shift, you mean to product-led growth in general?

Shane Hastie: Yeah.

Shifting to new structures and new ways of working is hard [17:40]

Rachel Obstler: Yeah. It is a really difficult shift. I’ll tell you actually, it’s funny. I think it’s even a more difficult shift for the rest of the organization than it is for the engineering team. So from the engineering and product design point of view, as I mentioned, one of the big shifts is really having experimental point of view, really getting very aligned to metrics, the metrics we’re trying to drive, focusing on velocity of experimentation and learning as opposed to new features. So while that is a shift, I think you can find the right people and you can do that shift relatively easy. I think the harder shift is an organization that is used to taking every person who comes to the website and making that a sales lead. That there’s a typical process that sales led companies go through, which is get a lead, maybe nurture them a bit, get them to a point where they’re marketing qualified, get them to the point where they’re sales qualified, and try to carry it through.

There’s also the direct sales outreach, but let’s put that aside for the moment and say, “This is the way the world works.” Product-led growth means instead of taking that marketing qualified lead and handing it over to sales, you’re trying to get them to sign up for a trial. Once they sign up for a trial, you’re hoping that they move along, get value, find value in the product and want to go farther with you. And then they might just buy on their own, right? There’s no sales in that process.

And so the sales focus becomes instead of landing that new small customer, it becomes more when that customer gets to a certain size and they’re ready to make a bigger purchase and they feel uncomfortable maybe doing that bigger purchase with their credit card, like at some point you don’t want 40,000, $50,000 a year on your credit card, they want to talk to someone. They maybe want to know that they’re working with a company that cares about them, that wants to partner closely with them. They may want to buy services if they’re getting to a point where it’s a much bigger thing. It’s maybe they’re serving a much broader part of the organization. There’s all sorts of reasons why you might want to talk to someone at that point. You may just want to talk to someone because you want to negotiate a better price.

And so it changes the nature of the role. The same thing with marketing. It changes the nature of the role. You might spend a lot more time on thinking about the value messaging during a trial versus just giving sales a lot of slide decks that they can use. It changes the nature of the success role. It could be that your success team, because users can self-serve a lot better, will focus a lot more on best practices and adoption than just like, “Hey, have you used this feature?” Because presumably, people can self discover and use things on their own. So it just changes the nature of a lot of roles across the organization.

Shane Hastie: This doesn’t happen overnight.

Rachel Obstler: I wish it did.

Shane Hastie: How do we make it easy to change?

Adopt changes incrementally [20:51]

Rachel Obstler: It’s never easy to change. And so I think the best way to approach this is to start small, right? So first thing is get your trial working, get people through the trial. If the sales organization is uncomfortable with it, if other people in the organization are uncomfortable with it, divert a percentage of your leads. Divert only the small customers, right? So every small customer that comes to your site, let’s send them to self-service. And then let’s still send the rest of the leads to sales. And then what happens is over time you prove the efficacy of your workflow, right? It also gives you a chance to learn. You have to start somewhere. You have to start with a free trial. You have to start with allowing people to purchase. And when you do that, you’ll be measuring. How many people get through the process? Where are they getting stuck? What are the issues? You’ll be improving it over time, but you’ll get a very good view of what percent of users self-service are successful.

And then once you get more comfortable with that, with that percent of success, that you think it’s a good workflow, then you can start to extend it to bigger and bigger companies, to more and more skews that you may have, right? You can allow people not to just buy your core product, but maybe add on products. Maybe you have multiple land motions. At my company right now we use the nail it and scale it terminology. So it’s very similar to this. Start small, nail it. And then you can decide to scale it based on the success of what you’ve done.

Shane Hastie: Thank you so much. If people want to continue the conversation, where do they find you?

Rachel Obstler: So of course I can be found at Heap. So rachel… It’s such a long email, but rachel.obstler@heapanalytics.com. I also have Rachel_Obstler is my handle at Twitter. And LinkedIn also, I think I’m the only Rachel Obstler on there. It’s not that common of a name.

Shane Hastie: And we’ll include these links in the show notes. Rachel, thanks ever so much for taking the time to talk to us today.

Rachel Obstler: Very happy to do it.

Mentioned

About the Author

What are the major trends that matter right now in software development and technical leadership?
Uncover emerging software trends and practices to solve your complex engineering challenges, without the product pitches.
Save your spot now!,

.
From this page you also have access to our recorded show notes. They all have clickable links that will take you directly to that part of the audio.

Subscribe for MMS Newsletter

By signing up, you will receive updates about our latest information.

  • This field is for validation purposes and should be left unchanged.