How Do You Measure The Success Of Your Agile Transformation?

In this article, Agile Coach Michal Vallo shares with us his thoughts on the topic of measuring the success of agile transformations. Based on his experience and research, he developed a framework to assess the benefits of the transformation process. Dive deeper into the four dimensions described in his framework: Organizational, Motivation, Value Creation and Response to Change. Enjoy reading!‍

There is one thing about agile transformation, that attracts my attention. The number of transformations is exponentially growing. However, do we receive true results? And what are the motivation or expectations from them?I have met recently several agile transformations leaders in a few companies. I asked them, why they undergo the transformation, and what their organizations are currently trying to achieve. Their answers surprised me a lot. There are of course many reasons, I will mention only those most common here. If you haven’t found your one, please, write to me and I can mention it in one of my future articles.

• Because company A is going through a transformation, we have to join the train, too.
• We were ordered from our HQ to do Agile,
• (so) we have to implement (add your favorite) framework.

In general, I observed that asking the questions about results or achievements were considered impolite. When I dig deeper, I found some of the underlying motives – we need to deliver faster, we need to make more money, we need to get better control, we need better quality, we need better process design, we need guarantee of the framework, we want to utilize our human resources better, and many more. Are these reasons indeed those that agile transformation is supposed to deliver?

‍If you want to kick off the transformation, you need to understand your problem first!

‍I wanted to understand better, so I organized a meetup in Prague. I intended to learn how organizations measure the results of their agile transformations. Several people came from different industries like bank, body shop and nearshore development, telco, or data processing. All people were very close to agile activities in their organizations. I began with the question. “What are your expectations from this evening?" One visitor told me – “I am for 10 years involved in Agile, but I have never seen it work so I came here for curiosity whether I could indeed meet somebody who truly has the results”. “Our bank claims we are in agile transformation, but it seems to be so chaotic with often changing priorities so I want to learn how others do”, said another one.

We have discussed what are possible agile deliveries, but the concluding part, how do we measure whether we have achieved anything remained void. All visitor agreed, in fact, that they do not measure specifically anything. To justify the transformation, the same cliché is used again and again – deliveries of IT will be faster, quality higher, a satisfaction of unknown stakeholders bigger. Simply the stuff tough at any agile training. They are trying to do in their organizations whatever technique is most popular these days, and it is rarely if ever coupled with process change or deeper organizational change.When I hear this, I no longer wonder it gives Agile a bad name. In my opinion, Agile is an umbrella for a set of (managerial) techniques for continuous improvement of the organization.

It is the way how we can modify the work arrangement and adapt it according to the changing external environment. Especially to the shifting from industrial to the knowledge economy and furthermore toward the creative economy. I write about this change in my article Introduction to Agile and Agile Transformation – Environment and for a better explanation using a color-coding scheme from the book Reinventing Organizations by Frederic Laloux. Only a summary: there are different types of organizations today. Among them, those build around Orange paradigm that represents the industrial economy.  And those build around Green paradigm that represents the knowledge economy and perhaps a bit of rising creative one. I am certain, that expectations of companies within both different types of paradigms should be different.

And there is one more difference. In the Orange paradigm, it is expected that results will be delivered no matter what. While along the way we destroy the tropical rainforest, contaminate ecosystem, poison the food, or corrupt the authorities, the organization tolerate this. In the Green paradigm and the following ones, such behavior is not tolerated by the general public and customers. For simplification, I have ignored many other parameters.When I look at the agile transformations in the organizations I came across, I have an impression that most of the expectations originate in Orange paradigm. Just very few if any comes from the Green paradigm.

So what to measure instead?

I think, there are no particular standardized metrics related to Agile Transformation. And there can't be. Even if we do only the superficial transformation of updating a few processes I would still focus measurement toward the business problem, which initiated the transformation, or the problems which we encountered along the way. What to measure is usually not static. Along the journey, we can add more attributes to look into the detail of something. Or we can abandon some measurements if they are no longer necessary. There will be only very few measures which will stay the entire journey.Johanna Rothman puts it nicely in her article “Where I Think “Agile” is Headed”. She suggests a very simple and straightforward approach there. We are interested in business parameters:

• Keeping and acquiring new customers.
• Keeping and hiring new people who are excited about the work.
• Deciding when it's time to retire old products and services and when it's time to launch new products and services for the customers and the people.

And she adds that these are not about a direct approach to “increasing shareholder value”, you increase value as an outcome of those three bullets. In my opinion, this fits the Green paradigm nicely.

A holistic approach to the evaluation of the Agile Transformation

When I dig deeper into plans and strategies of the organizations, I can often see the disconnect between planning process, expected results, and the operations. Expected results (aka goals) are often defined by management artificially. It is like “the same revenue/profit as we had last year + 20, 30, or 50%”, which means it is just a number. Expectations are rarely translated into activities and objectives for various parts of the organization. This is the situation when there are imposed unrealistic financial goals, while no sales intelligence or data are supporting such expectations. There is further no capacity, skill or resources available to achieve for such goal, and there is no action planned for dealing with it. The result is frustration and confusion. People in the organization will configure themselves into a business-as-usual mode then. It makes them disconnected from the organization and urges them to “run at 120%” while having no time to look right or left. Such a situation we know from the traditional organizations, however, it is not uncommon in those who are pretending they are doing Agile.

I suggest looking at the agile transformation holistically. In my experience, I want to observe and measure the progress of organizational change. And change can't happen just in the one part of the organization e.g. software development, because according to LEAN local optimization may cause harm. I want to see changes in the entire organization as the transformation will be spreading across the company and sometimes beyond.Agile Transformation is most of all learning journeyFor measurement of the success in our agile transformation, we can try to find measures, which shows us how the organization progress along the transformation journey. There are the leading and lagging indicators, which we can define and observe. The leading indicators will enable us to measure what we do, or how much we apply some activity. Lagging indicators will tell us about how the organization responds to the changes. Note, an organization is the complex adaptive system, therefore there is mostly not known the direct correlation between our actions and the response of the system upfront.

Organizational level

At the organizational level, it is perfectly ok to monitor business results, revenue, profit or number of the new customers. These, however, tells us just little about how the organization changes. Therefore I propose to add measurements to see the number of innovations or improvements we were able to implement, and their consequent impact. Further, it makes sense to measure transformational changes in each quadrant of the agile transformation model.

Motivation

This is the most crucial parameter of the entire transformation. We can monitor, how many people from all employees are already involved with the change and how their engagement changes. Do people have the possibility to decide or they still need approvals? And do they use their empowerment toward the good of the organization? One of the indicators here could be the number of unsolicited ideas for improvements that people bring to the table. Does the company have an organizational structure designed the way it supports engagement? What is the alignment of people behavior and company culture?

Value Creation

Value creation is formed through activities in the value chain – the key processes and how the organization makes money. We can monitor, whether the chain is transparent and whether operational decisions are done with the understanding of the value chain and the context. Are the activities that the organization does deliver an impact on the customer? Can we measure how much? Do we have technical quality? E.g. number of bugs. Are our products usable? E.g. feedback from customers about their experience. What is the cycle time for delivery? What is the length of the delivery path? And how many of the activities we do are creating value?

Response to Change

With the agile transformation, we will redesign organizational processes to allow for feedback. The feedback should be delivered quickly. And the organization needs to obtain the capability of responding appropriately once information from the feedback is available. We try to eliminate unnecessary decision making authorities. Authority to decide is delegated to those people in the organization, who are closest to the customer and have the best information for such a decision. We can monitor the progress of ongoing changes. Do the processes enable for rapid feedback? Do the processes allow changing the course? How much time does it take to process customer order? And how much time it takes to implement the managerial decision? Is the information about the state of the operations radiated consistently and transparently toward all who needs it? Do we know where are we heading?

Knowledge Management

To be able to act in the previous three quadrants, organizations need to invest in creating knowledge. Not once, but rather to implement a system of the continuous building of knowledge. The activity is not only for the people in the front line, or some developers. It is for everyone in the organization including senior management. Everybody needs to learn to acquire knowledge quickly and regularly. We can monitor here organizational ability to systematically gain new expertise. Are there plans for developing human capital? E.g. personal development plans integrated into intelligence. What are the missing skills and how fast can be acquired? Does the organization attract the right talent from the outside? What are the recruitment practices? Is there a clear recruitment strategy about what type of people organization needs and with what skills? How many patents organization registered?Agile transformation is always a holistic activity, whose purpose is having an organization fit. The fit organization can deliver better value and better economic results, and this is what we want.

‍This framework will help you measure the success of the agile transformation on your own company, and indicate the importance of leading and lagging indicators. Do you have insights into measuring the impact of the agile transformation? Then, become part of the Workpath Community to discuss your experiences with other practitioners and experts.