Despite the development of agile ways of working, continuous delivery and product orientation, many companies are still working using the format of projects. In all reality, projects will never completely go away. In the larger enterprise, there will always be a need to coordinate a more significant effort across teams. Projects live and breathe to the beat of the RAG (Red Amber Green) status, like the traffic at a busy junction moves to the rhythm of the lights. Green, all is good. Amber, we need to watch out, maybe ask the team to provide an extra weekly status report (as if extra bureaucracy was going to help them). Red, hell breaks loose, internal audit commands actions and management steps-in to coordinate the crisis.
Most corporate organisations care about IT delivery when the RAG flags Red. Digital leadership stems from taking an interest in continuous improvements
As a delivery manager by trade, supplying services to corporate organisations, I have seen this pattern repeat over and over again. The management is too busy to care for a Green status. The team identifies possible improvements, and they get rebuked for not focusing on delivering more functionality instead. The team identifies the weak signals of the emergence of systemic challenges, and they remain ignored until a milestone is missed. Once the status is of the appropriate shade of Red to attract the needed attention, it is all hands on deck. Firedrill, all the parties are rallied to the table, at last, the systemic challenges get some attention, and the crisis eventually gets resolved. I have to hold my hand up, I used the Red status a few times when I knew that the systemic issues would not get any attention otherwise. I was discussing this matter with a senior programme manager recently, in an organisation starting with their agile effort. The programme manager was telling me how effective their collaboration was in crisis mode, and he had been through a few incidents lately. I was pointing out to him that good agile leadership was about making such collaboration happen in everyday work, without breaking a sweat.
Chaotic and the Cynefin framework
The Cynefin framework from Dave Snowden is a good start to explain the cognitive drivers linked with the RAG status. Cynefin is a SenseMaking framework, which defines five domains. The starting domain is Disorder. In Disorder, we have not yet established how to categorise the nature of the challenge, and consequently, have not established the right course of action. There are then two ordered domains, which are characterised by an established causality between action and effect. This causality may be of an Obvious nature or a Complicated nature. Cynefin defines how to manage under those conditions:
In the Obvious domain, we sense the condition, we categorise it and apply the adequate response, generally policy based.
In the Complicated domain, we sense the condition, analyse it, calling in the experts if necessary, and respond with the right pattern for the problem.
Mirroring the ordered domains, Cynefin establishes two unordered domains of Complex and Chaotic. In the unordered domains, the causes and effects do not appear connected, at least apriori. It means that:
In the Complex domain, we probe through experimentations to determine the right path. We sense the reaction to the probes and amplify what appears to work best.
In Chaotic, we take action to come out of the undesirable situation as soon as possible by setting a direction (good or bad). There is urgency. We then rethink the course of action once the crisis is contained.
Chaotic, the land of the heroes
The Chaotic domain is very much the context where managers lead from the front, and it reinforces a command style. Sure enough, this is a good feeling when you are a manager. Like captains on the battlefield, you have to keep your wits, think quick and direct action. It is very satisfying driving a team and a project back from the chaos and into normal again. Leadership feels in charge through chaos. Heroes get made from solving chaos. Managers bring all the people around the table, pull the strings that only rank would allow to, trump other demands by pulling more strings with the hierarchy. They ask the team to put in a Herculean effort, hail in praise those that dedicated all-nighters and weekends to the cause. They make sure that they get themselves noticed by the senior management through the whole ordeal too. One would think that going from one crisis to the next would not ring well, but it does. It is a sign of not shying away from a challenge and battling your way through it, with an unquestionable commitment and sense of duty to the business. Organisations do like heroes. The ones that grit their teeth long enough through such situations earn their medals and get the promotion. Yes, let's spell this again, the management that leaves situations to rot to the point of creating a crisis and then proceed to burn-out their people to come out the other side, are the ones that eventually get promoted. The ones that kept their project Green, on the other hand, tend to be regarded as not sufficiently battled-hardened for the next role up. As leaders tend to identify their seconds using patterns they rode themselves, the situation perpetuates and gets engrained into the culture.
Though the world loves heroes, the agile organisation does not need them!
With this in mind, is it so surprising that organisations have cultivated a dysfunctional relationship with their IT department? As technology is ever more present in the product/service mix of most organisations, businesses on their digital journey have to take this situation very seriously, or they'll only scale the chaos (and many do indeed!). Progressing through cycles of chaos is not sustainable. Businesses operating in this fashion are left exposed to considerable risks and tend to go from one crisis to the next. In the process, they burn-out the talents that they need to retain.
How do we turn such a culture around?
We are quick to critic the middle management for their behaviour and contribution to the problem. Blame is rarely constructive toward an answer. Antagonising managers results in camped positions and status-quo. The middle management also holds the keys to transforming the ways of working and, for most, are capable of changing. Unfortunately, most organisations give them no reason to. What is the reward for running a more efficient IT with a Green RAG? Less budget for next year, fewer people and a "try harder" from the boss.
Turning this state of play around is actually where successful Digital change starts:
Systemic management
Culture of continuous improvements
Organising for excellence
Systemic management of the teams
Agile Thinking insists on the need for good collaboration, rightly so. The digital corporate is also aiming to replicate the organisation model of the internet unicorns with flatter structures and autonomous teams, in favour of faster decision making. However, most still manage teams as a collection of individuals and focus on tasks. Can't we see the paradox in this?
If we manage from the individual dimension, we need micro-managers to organise and coordinate the tasks. Leaders justify their worth, insisting on being at the cross-road of every decision and becoming a bottleneck to progress. The team members surrender their autonomy and collaborative incline to the leader on the basis of once bitten, twice shy. Without an approach to managing systemically, teams cannot move to the desired levels of autonomy and collaboration.
Systemic management starts with the managers connecting with the team where the work happens (i.e. in the team's space, not in the manager's glass office). This concept has existed for many decades with the Gemba. Gemba is a term from the Toyota Production System. Gemba consists in visiting the place where the work happens. It is evident in manufacturing, thanks to the physicality of the production chain, but less so in a digital team, where all is very virtual. Going to the Gemba in IT reinforces the need for radiating information in support of visual management. Flow and quality get progressively plastered on the walls (or online boards for distributed teams), and teams can discuss improvements. Managers also need to be equipped with skills and techniques to work with the team as a whole. Hearing the Voices of the System (VoS) generates transparency and highlights systemic challenges early. In turn, problems are anticipated in favour of a better flow of work. By using such practices, the teams become more autonomous, and leaders and teams are at one, with fewer needs for intermediary micro-managers.
Continuous Improvements (Kaizen)
In the Cynefin framework, Dave Snowden draws a little sign on the limit between the Obvious and the Chaotic domains. This sign indicates a cliff of complacency. When an organisation relies too much on established procedures, it becomes exposed to disruption. When disruption happens, it falls into chaos. As mentioned above, the same happens to our projects. I even got told this week that this had a name: "Watermelons Projects" (All credit to Jay. Thanks!). Green on the outside, Red on the inside. It is not an IT-specific problem either. We heard this week that, a few months away from launch, Crossrail got delayed by a year! Not a few weeks, but an entire year. We are not monitoring the signals on the inside, so the bad news bubble-up and end-up bursting in spectacular style.
To avoid complacency (and drive continuous performance), the Toyota Production System used Kaizen, a process for continuous improvements. Part of everybody's job is not just to do your job, but to figure out ways of improving how you do it and improving the product-outcome of it. This constant effort is an excellent way of stimulating brains, especially as work on a production chain can be dull. Chihiro Nakao, the father of Shingijutsu Kaizen, was insistent on driving simple improvements that did not need investments. One of his objectives was to stimulate people. Stimulation prevents complacency. Good Kaizen also creates a different dynamic between the leadership and the teams. The teams develop autonomy, self-reliance and collaboration, while management becomes more about facilitation/coaching and inspiring creative thinking. Would it be that Toyota invented 40 years ago what we are trying to achieve today in IT?
Many are quick to point out that manufacturing production is not the same as IT. It is true. Continuous improvements are much easier to experiment within the virtual environment of IT. IT workers are also much more natural knowledge workers than manufacturers on a production chain. There is no excuse really. Only that corporate IT has been too busy tracking Red RAG statuses, that we forgot to implement the discipline of continuous improvements on Green RAGs. Time for a reboot.
Organising for excellence
No organisation objects to driving excellence, yet very few genuinely do so in IT. The DevOps movement has contributed to bringing the topic onto the agenda, but the effort remains contained into IT. Quality is everybody's job, starting from the demand initiation in the business. Lack of quality is easy to spot on physical goods, but harder on virtual software. You can get some idea from the usability of the front-end or testing some edge cases, but those are only the tip of the iceberg. What quality of code lurks below the surface? When will this come back to bite you? The RAG status does nothing to fix this, in fact, it generally contributes to worsening it. I had a project some time back, which was fluctuating between Amber and Red. The teams were under constant pressure from business and management to hit the deadline. The only part that could give was quality. Did anybody care? The team did, but nobody would listen. So, we got the team to start capturing wasteful practices and logging technical debt, for anything that would not meet the standard that they would expect of their work. The team felt better for it, information got radiated, and it became too inconvenient to ignore. It should not have to come to this point however. IT leadership and product managers should have a vested interest in producing quality. Working from projects does not promote such behaviours. Projects are by nature transient, and the ones that are delivering are rarely the ones that are maintaining. Quality, sustainability, maintainability, rarely registers very high in day-to-day decisions. The short-term attitude brought by a project focus does not support the development of a solid IT foundation for Digital. Effective Kaizen is also unlikely to happen unless there is a long-range commitment from the team and leadership. Beyond the better alignment with customers, the product/service based alignment enables to maintain long-lived teams and leadership. It enables investing in people, in their understanding of engineering and in raising quality in the prioritisation decisions. Such alignment is however mostly orthogonal from how traditional organisations have been set-up (inherited by economies of scale over economies of flow) and constitutes a major systemic challenge in making progress towards Digital performance. We will cover this topic another time. The digital organisation will be the topic of our November Digital Leadership meetup as well.
In summary, the dysfunction of IT in many corporations has been mostly self-inflicted as a result of project-based work and driving the wrong incentives in the leadership. It has promoted behaviours that are incompatible with building the foundations for successful Digital businesses. Solving the situation requires small steps for leaders and a giant leap for leadership (I borrowed this sentence… :-) ). Digital change starts as simple as working with the team to understand how we can make tomorrow better than yesterday: Smarter processes, more autonomy, better products/services, better technology. It is, however, a fundamental departure from an IT leadership based on accountancy over project budgets to an IT leadership enabling product and engineering performance: Systemic management, continuous improvements of products/services and processes, excellence of engineering, and resolving the systemic impediments that stand in the way.
