This article investigates the challenges larger organisations face when implementing product analytics, how to overcome them, and how to successfully develop a data-driven product development culture that focuses on providing excellent customer experiences.
The impact product analytics drives depends on your organisational context
The cliché that tools are only a means to an end and that success depends on what you do with them is particularly true when it comes to product analytics. The role of product analytics is to surface insights on how users are behaving while engaging with your products (apps, websites, etc). Those insights can change the future of your product and thus organisation if, and only if, they are integrated into your product development cycle. Depending on the organisational type you’re in, this can either be a given or a monumental challenge.
A given when you’re operating in small organisational structures or organisations where there’s a clear focus and commitment to product development. These organisations are often centred around a product team that takes care of the end-to-end product development cycle. In enterprise-grade organisations, however, this could become a monumental challenge. These organisations have rigid structures, data silos and processes that don’t allow the insights sourced from your product analytics solutions to translate into new product updates. But why is that?
Why product analytics is hard in enterprise organisations
Deploying product analytics sounds self-explanatory. Generate an insight, and move it into production. In smaller structures, where product analytics and product development are two domains that are under the control of a product team, the integration happens almost naturally.
However, the luxury of this simplicity tends to be reserved for those smaller companies. Just like there is still confusion on the difference between marketing analytics and product analytics, there is still confusion in larger and more traditional organisations on who should lead product analytics. The reason for that? Your organisational structure, the distribution of competencies and skills, and the (lack of) processes.
In enterprise-level organisations, the usual structure has determined that any analytics-related topic belongs to the marketing team or the analytics team. Meanwhile, product development resides with IT. Product analytics is being looked at as just another analytics solution and therefore attributed to the teams already operating in these fields. Even though that could make sense in terms of skill set, these teams have little to no influence on the product roadmap as product roadmaps in larger organisations are usually managed by IT departments. This means that product analytics tools are used by teams to generate insights on how the product could perform better but these insights aren’t materialising into product development roadmaps, sprints or releases.
It’s not called product, it’s called DevOps
All organisations have teams focused on optimising the product further. In some organisations, like enterprise-level blue chip organisations, they’re not called product teams. But they’re still there. In organisations with bigger IT departments, the practice of developing the product, strategy, operations and the processes and multi-disciplinary teams involved is often referred to as DevOps.
Amazon defines DevOps as:
"DevOps is the combination of cultural philosophies, practices, and tools that increases an organisation’s ability to deliver applications and services at high velocity: evolving and improving products at a faster pace than organisations using traditional software development and infrastructure management processes. This speed enables organisations to better serve their customers and compete more effectively in the market."
In an ideal world the DevOps process looks like this:
Insight generation: Using a product analytics tool to understand how users are behaving in your application, website or platform. Essentially everything explained in the first chapter.
Hypothesis construction: Now that you have found an insight, aha moment or feature that should be focused on, it’s time to build a hypothesis. This is when you define what you believe should be done in order to drive more of the intended behavior.
Test: Deploy the variation you’d like to test.
Evaluate: Evaluate which variation leads to an increase in the desired behavior.
Develop: Push the winning version to the development team’s spring backlog.
However, in many organisations, the “official” DevOps process only picks up as of the “Test” phase. Insights are barely generated or are based on gut feeling rather than based on empirical evidence (read: product analytics data).
Knowing that product analytics is typically shipped off to a team already doing some sort of analytics, this begs the question: “is product analytics then managed by the wrong team in these organisations?” Not perse. Operating a product analytics tool successfully requires a specific skill. One that is rarely present in DevOps teams. But what is the solution then?
I need product analytics in an enterprise organisation. What are my options?
You basically have two options:
Create a team. Organisations that don’t have dedicated product teams could create one. Today’s IT teams in large scale organisations often apply DevOps processes but the scope they have to deal with goes beyond true customer-centric product development and optimisation (security, hardware, etc). Building a product team allows an organisation to truly adopt customer-centric DevOps processes with a sustained focus on driving best-in-class customer experiences.
Adapt your processes. Alternatively, organisations can adapt their processes to build bridges between departments that hold the skill sets required to operate them (marketing, analytics) and teams that have organisational power and control over the product roadmap (IT). The result is an adaptation of the existing DevOps processes and workflows, with product analytics acting as the bridge connecting teams.
As the first option would fundamentally shuffle things around in organisations and likely be more difficult to implement, the second option is often more appealing as it can be put into production by simply focusing on process + cultural change. Let’s have a look at how the existing siloes can be bridged in order to drive the results we’re looking for.
Integrating product analytics in the DevOps cycle
What does such a reviewed process look like? When done well, it’s very pragmatic.
Let’s have a look at the DevOps process as shared earlier on and explore how it might come to life with product analytics integrated.
The observing reader noticed that steps 2-4 are actually steps from a traditional A/B testing process. This makes sense; commonly, we might like to test different variations of a possible UI configuration before letting the development team spend time perfecting a permanent version of the solution. What we add to this process, when integrating product analytics, is the insight generation that fuels decision-making within your experimentation program.
Conversion rate optimisation (CRO), therefore, sits between product analytics and development. Or better yet, is simply an integrated part of a customer-centric DevOps process. In practicality, this means that the new process spans multiple entities within your organisation and enabling different teams to work within it becomes the key to building a data-driven product culture.
How you deploy this processes and who executes what will be specific to your organisation, and that’s ok! What matters is everyone agrees on the setup and is bought in to the result it is meant to drive.
Here are a few common ways we see teams aligned to our integrated approach to DevOps:
All of these variations are viable options for a customer-centric DevOps process executed by distributed teams.
Technologies are tools, your people have to take care of the rest.
While we hear it time and again, many of us choose to ignore that technology is a means to an end, not a solution in itself. But we urge enterprises looking to integrate product analytics into DevOps to place an emphasis on people and culture when rolling out these processes.
Combining technology with the human capital in your company, as well as designing the processes that allow you to leverage both of them in an optimal way, is the key to successfully building a data-driven culture–and ultimately increasing your rate of innovation.
Comments