The Politics of Pilot Teams

From what I can tell, more companies than ever before are working on transforming to the product model. I don’t know all the reasons for this, but I do know some of them.

I’m seeing company boards taking a more active role in pushing the CEO in this direction. Their interest is usually valuation.

And of course, generative AI has pushed many companies to the point that they now know they must change, or risk being disrupted. The technology is both an unprecedented opportunity, as well as an existential threat.

This next point is a bit technical, but it’s important to understand: while you can continue building conventional (deterministic) products using the old, feature-team roles and methods, building intelligent (probabilistic) products requires a specific set of competencies and heavy use of live-data prototypes, continuous product discovery and product delivery; competencies and techniques that are foundational to product model companies, but are usually foreign to those that have not yet transformed.

Whatever the various reasons for transforming may be, the more transformations I see underway, the more I’m reminded of the critical role of politics in any medium to large sized organization that is trying to tackle substantial and meaningful change.

I have written several articles on transformation politics, spoken about the topic at conferences and meetups, and it’s a major theme in the new book TRANSFORMED: Moving to the Product Operating Model.

One thing that’s become increasingly clear is the critical importance of a successful pilot team. In this article, I’d like to double click on this complex topic, in the hope that this may help you increase your chances of success.

The Purpose of a Pilot Team

First and foremost, the purpose of a pilot team is to prove to your product organization, your leaders, and, in some cases, your investors, that the product model can deliver the business results you need.

But we need to do this quickly. We can’t wait years for an entire organization to transform.

You can consider a pilot team as the MVP for a full transformation. It’s just enough to address the risks, and much faster than the whole eventual effort.

A pilot team lets the company learn quickly (it’s usually a quarter or two), inexpensively (it’s just one or two teams), and safely (without putting the bulk of the company’s revenue at risk), just what it means to move to empowered product teams operating in the product model.

But there are several important secondary purposes as well.

Often, the majority of the product and technology organization have never seen what good looks like, so because they don’t know, they assume all kinds of things about the competencies (“sure, we have product managers”), the concepts (“we already know what product discovery is”), and the techniques (“won’t this take longer?”).

The pilot team is how we demonstrate to the broader product and technology organization what good really looks like.

Similarly, the stakeholders are typically coming from a position where they’ve heard lots of promises over the years, and few materializing. They need to see for themselves what it looks like to collaborate with a product team to deliver real business results.

But more generally, until a company has completed a successful pilot, they really don’t even know what they don’t know. How can they talk about the training they need, or the new leadership responsibilities, or the cultural impact, if they don’t even know yet what they will need to do?

Which is why it’s so important to think very carefully about the pilot teams.

Who do you want on the team? What problem do you want the team to solve? What dependencies are involved? What is the outcome the team is striving for, and how will it be measured? What type of coaching will the team members need?

Let’s address each of these in turn:

The Staffing of the Pilot Team

This may be the single most important element, yet it’s counter-intuitive to many product leaders.

Strong product leaders hand pick the members of these pilot teams.

This is not meant to be an “average” product team. This is meant to serve as a form of bar-raiser. We want to show the other product teams as well as the stakeholders in the company what it means to be a skilled, empowered product team.

Sometimes we need to bring someone in from the outside because there may be skills or competencies that don’t currently exist in the company. Or, the product leaders might identify some very high-potential people, and then put them through an intensive coaching and training program. It’s obviously important that the members of the pilot team want to learn to work this way. There are always some people that hate change, and they would not be candidates for a pilot team.

You might select this pilot team from your existing product teams, but more likely you will hand-pick each member to create a new product team specifically for this pilot.

You also want to be sensitive to the size of the pilot team. Just about everything is easier and faster with a smaller team – role clarity, coaching, communication, collaboration, decision making, and dependencies are some of the big examples.

If you define a large set of new roles, and people think that they are all required in order to move to the product model, be aware of the optics of this. Many stakeholders could be inclined to suspect an unawareness or insensitivity to cost.

A typical pilot team would include a strong product manager, a product designer, an experienced engineering tech lead, and maybe one or two additional engineers. A small team of strong, motivated people is key.

The Problem to Solve and Outcome to Achieve

While staffing the product team might be the most important decision, the hardest decision is typically identifying an appropriate problem to solve and measure of success.

Now you could ask the pilot team to do some research to identify the problem to solve, but politically, this is rarely wise. The relevant stakeholders likely have years dealing with long-standing problems, so they are usually well aware of real problems, and part of the collaboration is to trust the stakeholders on the problem in the hopes that they will trust the product team to discover and deliver the solution (rather than handing them a roadmap of features they think might solve these problems). Keep in mind that the point of all this is to be able to solve these problems in ways that deliver the necessary business results.

While this speaks to who participates in identifying the problem to solve, the harder part is to scope the problem so that it’s in that sweet spot of impressive but not impossible.

You don’t want to pick something so simple that nobody will be impressed once the team solves it, and they think they could have solved this in the old model. You want the CEO to believe that this outcome is something that would have been very unlikely with the previous model.

On the other hand, you don’t want to set the team up to fail by picking something that even a very strong team would be unlikely to solve in a limited amount of time.

So the scope of the problem is critically important, but there are additional critical considerations.

You don’t want to pick something that is heavily dependent on one or more other product teams that may not have capacity during this period, or on major tech-debt or replatforming efforts that may not have happened yet. Now there will always be some dependencies, but you do want the team to have enough autonomy that they can build what’s needed, and work directly with other teams where necessary for typical dependencies.

As with any empowered product team, you’ll also need to ensure that the pilot team has ready access to customers, to the data those customers generate by using our products, and to the relevant business stakeholders.

A little less obvious is that this pilot team will need to collaborate with the relevant stakeholders, and so we want to choose problems where the relevant stakeholders are willing to participate in this pilot as a new way of working.

Once the problem to solve has been identified, you’ll want to work with the relevant stakeholders to gain agreement on the best way to measure the business impact in order to judge the outcome.

Again, there is some nuance here. It’s good to be able to tell the pilot team that the way their solution’s effectiveness will be measured is by this set of KPI’s (e.g. adoption, engagement, customer satisfaction, revenue), but while it’s fine to tell a team that the outcome that matters is something like incremental revenue, you don’t want to hand the team a specific absolute result they’re expected to achieve (e.g. $2M incremental revenue in the next quarter), as again, it’s easy for the pilot team to feel like they are being set up to fail if they think some particular amount is unrealistic or unachievable.

Instead, we work to be very clear on how a successful outcome is measured, and then the pilot team will work towards that outcome.

Ultimately, strong product leaders know that the pilot team will be judged by its ability to achieve a business outcome. Something that is truly meaningful to the business. It’s not enough to make our customers happy, we also have to solve for the business.

Coaching and Training of the Pilot Team

Finally, even with strong people on the pilot team, empowered with a clear problem to solve and measure of success, if the members of the pilot team have never worked this way before, we need to make sure they receive the coaching and training they will need.

The whole point of course is for the pilot team to show they can solve the problem and achieve the desired outcome, so the pilot team will still need to be able to do the work, but most people have never before been exposed to the product discovery techniques they will likely need to quickly identify and test a solution.

Ideally, you have at least one product or design leader with experience working this way, and that person can take the lead on providing the necessary discovery coaching. If not, you might want to enlist the help of a product leadership coach or product discovery coach.

Beyond The Pilot Team

If, for any reason, things don’t go well, then you would want to evaluate what the issues were, and consider whether you should iterate on the pilot team (as you would with a true MVP), and make another attempt. You might also consider getting the help of a product leadership coach.

However, if things do go well with the pilot, you can expect that many of your people will request to be trained to work this way. You can also expect that many of the stakeholders will want to try engaging this way as well.

This is what we hope for, and it leads to the more general discussion of a good plan to rollout across the organization. However, one important lesson learned: in large companies, you should generally consider each business unit as essentially its own transformation. As such, you would need to do pilot teams for each business unit. This is especially true when the business units are the result of earlier acquisitions.

The work of product is always hard, and there are no playbooks for guaranteeing success. However, this article covers the most common reasons we see transformations failing, so hopefully following these suggestions will increase your chances of success.

The Politics of Pilot Teams

Comments