Agile is a business thing - not just an IT thing

Business people think that agile is an IT thing. But really, it is a business thing.

Because the decision to "do agile" cannot be made at the point where IT takes over. The decision must be made when the project is conceived, in terms of its business goals.

Project inception generally has two phases in a large organization: when the business case is made, and when the project work starts. To make the business case, you have to estimate the timeline and the costs. To do the cost estimate, you have to do some up front requirements analysis. To do the timeline, you have to make assumptions about the delivery process. Agile changes both of these. Thus, agile must be considered during preparation of the business case.

When project work begins, the business sponsor often contracts for a detailed requirements analysis to be performed. The theory is that this can then be used to solicit bids on building to the requirements; but that theory is deeply flawed, and the horrible track record of IT projects bears witness to this. This up-front detailed requirements is a root cause of failure, because these up-front requirements are usually wrong on many levels. That is where agile comes in: agile allows requirements to evolve.

This means that if you perform an up-front detailed requirements process, you have killed the project from the outset: if done in a waterfall manner, you can be sure that the requirements will be wrong, and if done in an agile manner, you have locked in the requirements so they cannot change, and so agile cannot work.

This is why agile is not just an IT thing: it is a business thing. Project sponsors need to learn and understand how agile processes work so that they can think in terms of the agile cycle from the outset - even when they are making the case.

Don't use frameworks (like SAFe) out of the box (continued)

In my prior post I promised to provide an example of how SAFe is best thought of as a model rather than as a design: a model is a tool for thinking whereas a design is something to implement precisely.

Consider SAFe’s model for portfolio management. The SAFe model defines a portfolio “kanban” process in which work is defined as a set of “business epics”. Just think of a business epic as a project: it is not really, but for our purposes you can think of it that way.

The SAFe process defines the lifecycle of a business epic, from identification of the need for the project, through alternatives analysis, through implementation. This is all pretty standard. The SAFe process is kanban-like in that it defines a single pipeline through which all business epics pass.

That works fine in many organizations, but there are many organizations that have multiple portfolios. In that case, one would need several pipelines. Some organizations tier their portfolio based on the source of funds, or by investment amount: the latter is typical in government agencies. In fact, government agencies usually have mandatory investment management processes and so one could not even use the SAFe model, but one could still use other parts of SAFe. Indeed, the SAFe guidance says that one should expect to have multiple “kanban systems”.

Such complications means that there often cannot be a single backlog. Another complicating factor is that it is often the case that one investment serves multiple strategic goals: yet SAFe presumes that there is a hierarchy of investment themes which are associated with epics, comprising a “release train”.

SAFe also differentiates between “business epics” and “architectural epics”. This is sometimes a useful thing to try to do, but it is not always so clear cut. For example, when a telecommunications company invests in a new network, is that a business epic or an architectural epic? Hmmm. When one adds servers to reduce customer wait time, is that a business epic or an architectural epic? Hmmm. But as the SAFe guidance points out, there might be different sources of funding and oversight for the business and architecture investments, and this might cause these investment categories to be separated.

These are not flaws in SAFe. As I explained, SAFe is a model - at least, that is how I look at it. So to apply SAFe, one should first understand the intent of the model, and then consider how that intent can be realized in one’s situation. In the example at hand, it might mean adjusting the portfolio process defined by SAFe.

My point is: do not view frameworks as designs or templates. View them as models. Create your own design.

Don't use frameworks (like SAFe) out of the box

Is there a "template" for life?

Can you directly implement the advice your mother and father gave you? Or was the advice intended as abstract, requiring you to incorporate it into your thinking, so that you can integrate it with other advice and other knowledge and apply it to each of life's unique situations?

Applying a framework like SAFe exactly as defined is like applying your parents' advice exactly as articulated: it won't work.

SAFe - and the countless other frameworks that have come from IT thought leaders and organizations - is an excellent model, but a model is food for thought. Models always leave out details. Models are a basis for discussion, for analysis, and for design: a basis for design - not a design.

To apply SAFe, you have to think about it, and customize it to your organization. In the next post I will discuss one particular aspect of SAFe in order to illustrate the point.

Does agile encourage bad behavior?

All agile coaches are familiar with projects that “do agile” but don’t really do it. Agile can be used as a license to not plan (throw out the master schedule), not coordinate (cancel all the formal meetings and expect ad hoc collaboration to just occur), have the team commit to a sprint backlog and yet allow the product owner to change the stories during the sprint while still expecting the team to deliver on their commitments at the end of the sprint. These are all known issues to all coaches, but these issues all have to do with behavior that is imposed on a team. What about team behavior? Do teams themselves mis-apply agile ideas in ways that are enablers for bad habits and dysfunctional group behavior?

It took decades for disenfranchised groups to get managers to understand that conversations on the golf course or in the men’s room leaves people out (e.g., women and those who are not personal friends with the boss). Now agile comes along and advocates a return to ad hoc conversations: are we risking a return to patterns of exclusion?

Let’s remember that agile was designed for software development. It should not be applied to general business processes without careful thought. Agile assumes that teams work in close proximity, and so if an ad hoc conversation starts, others can hear it and join in. If teams do not work in close proximity, ad hoc does not work.

Another dysfunction that I see a-lot is when teams do not keep meeting notes. Meeting notes? Isn’t that old-school? Doesn’t that sounds like those old pre-agile dysfunctional meetings where it took three weeks or more to schedule it, and lots of people sat around a table and no one said what they really thought, or there was very low quality discussion, or worse, someone in the meeting got mad because he felt that others had not included him in discussions that occurred prior to the meeting and he felt that the meeting was an ambush? (I have seen and lived through all these things.)

No. Effective meetings require that all participants have take-aways. One of the core practices of Extreme Programming is to document decisions on the team’s wiki: that is a form of meeting note. Meeting notes - in any form that is appropriate - are essential for remembering what decisions were made, and if the notes are done well, they also record why the decision was made. Meeting notes do not have to record what everyone said, but they must mention key discussion points, issues that were identified, and decisions. That’s it. Very short and sweet, very to-the-point, but sufficient for others to read and know and understand the outcomes of the meeting.

Even ad hoc meetings should result in meeting notes, in an appropriate form, and too often they do not: agile’s ad hoc philosophy is being misused to excuse bad business behavior and laziness.

Planning: isn’t that old-school too? No. Agile requires planning. The only difference is that an agile plan focuses only on what matters and not all the details; and the plan is reviewed and updated continually, as things change. But planning is essential. It’s just not about the plan: it is about the planning that is crucial. But the plan is important too, in that it needs to be an information radiator so that everyone can see the current plan and be aware when it changes. The plan forms a reference-able shared understanding across the team and other stakeholders.

Agile should not be an excuse to not plan.

I think you get the idea. Before we throw out every old practice, or apply an agile value to a new situation, we should ask ourselves what purpose existing non-agile practices serve, and make sure that we are improving things and not making them worse. Most traditional practices have an agile equivalent: simply abandoning old practices is not sufficient to become agile.

Agile coaching is not transformation coaching

The Agile Coaching Institute has a wonderful breakdown of the skills that are needed to successfully coach agile teams. The model is useless for the work that I do, however, and I am an agile coach - a transformation coach.

Consider basketball. When pro basketball players think about their sport, they think about the things that they experience, and that are important to them: the plays during the games, the practices, the endorsements. Aspects of basketball - aspects that are central to the sport, such as sponsorships and team management - are on the periphery of a player's thinking.

But if you were to ask a team owner what things matter, they would have a very different perspective. The two perspectives are compared in the figure below.

The same holds true for agile coaching. Agile team coaches experience activities pertaining to their teams, and aspects of software development that are external to teams are experienced - by team coaches - in terms of the way that those things interface to the team; those externalities are experienced as somewhat peripheral. It is kind of like the famous "View From NYC" picture from New Yorker magazine that we have all seen: to a New Yorker, the features of New York loom large, but the features of the rest of the world - while just as important - diminish toward the horizon. In other words, one's perspective depends on one's experiences.

That is why "transformation" is only a single item in the Agile Coaching Institute's list of skills needed for agile coaching: because to a team coach, transformation is just one thing going on, and it is not usually central to what teams think about. Teams are affected by a transformation program, and they participate in it if it exists, but it is not what they focus on each day.

In contrast, an agile transformation coach thinks about transformation every day, and their view of coaching - transformation coaching - is very different from the view of a team coach. The two views are illustrated in the figure below.

The two diagrams are linked: in the team coach's view (adapted from the Agile Coaching Institute's list of core coaching skills), "Transformation Mastery" is a single slice of the pie. In the transformation coach's view, "Transformation" is the whole pie; and in that pie, "Team Coaching" is a single slice. Thus, one set of skills does not subsume the other; rather, they inter-connect.

Disconnect: why business does not "get" the idea of "experiments"

Experimentation is an important agile concept. Experimentation is a risk management tool: the idea is that one tries out a new approach out as an "experiment" before committing wholesale to the new approach. If the approach does not work out, one can quickly change course, and the experiment was a "contained failure".

What business hears is, "Let's play: we will experiment, and reward failure. We will spend your money trying out things that we are not sure will work."

This is a big disconnect, and the agile community bears much of the blame for this disconnect. It is the hubris of the agile community that emboldens it to speak about experiments and other agile practices as if the case for experiments need not be made. When speaking to senior IT people who have decades of pre-agile experience, one really should have some deference, and not expect them to swallow practices with names that are purposefully controversial - even frivolous ("team happiness").

As Carl Sagan said, "extraordinary claims require extraordinary evidence".

Perhaps instead of talking about "experiments", we should talk about "proof-of-concept" or "pilot". Experienced IT people understand those concepts, and they are really the same thing. It is counter-productive to use new terms - terms that have a shock effect - when trying to convince management in an established organization to adopt a new practice.

One thing that is new about the concept of experiments is that failure should not be viewed as negative: failure causes learning, and therefore better decisions are made after that point. It is a sad fact that in most large organizations, any kind of failure is damaging to one's career - even if the experiment was daring and innovative and caused learning. Agilists want to encourage a culture where prudent and careful risk taking is accepted and rewarded - even if it sometimes results in a contained failure.

But failure is still failure: it results in sunk costs - lost time, wasted effort, wasted money. Management needs assurance that failure - even contained failure - actually results in learning, and that the failure was unavoidable. They want to know that teams are being thoughtful and are using their best judgment and the best information available before they try something that results in failure. It is up to teams to instill that confidence, and it is up to management to be open to encouraging risk if the team demonstrates that it is cautious and thoughtful before it undertakes an experiment.

Are we upholding our end of the bargain?

Apply critical thinking at agile conferences

The book Critical Thinking Strategies For Success compares what it calls "sophistic thinking" with "strong sense critical thinking". The former is when there are doctrines and everyone nods their head yes to anything that supports those doctrines.

Recently I attended AgileDC 2013, and I noted that there was a talk by someone who I know to be incompetent and who does not know what he is talking about: in fact, he was fired from his last company for that reason; yet he was presenting at AgileDC and he has a large following in the community. That community does not know, however, that in real world situations, this person cannot perform because his knowledge does not extend any deeper than platitudes. He does not have enough real world experience to turn the platitudes into action.

Another person speaking at the conference laid out an approach that I know for a fact is not the approach used in the organization in which that person works, yet this approach was presented as a cornerstone approach. Again, after sufficient platitudes, all the heads nodded yes. More sophistry.

We are not doing enough critical thinking in the agile community. We need to be skeptical. Just because someone says something at a conference does not make it so, and where is the proof that they actually did what they say they did? Unlike scientific conferences, agile conferences are practitioner conferences, and the work presented is not research that has been replicated under controlled conditions, and there is no standard of ethics that is being enforced to ensure that people are held accountable by their respective organizations for presenting accurately. In fact, there is plenty of incentive to spin things because it enhances the careers of the presenters and the reputations of their organizations sponsoring those presenters. AgileDC - and most practitioner conferences - are more marketing than they are reality and we have to keep that in mind.

These conferences are still valuable though. There are lots of good ideas that are shared: we just need to be skeptical because some bad ideas can be made to sound viable when they are not. There is networking that happens at agile conferences, and that is always worthwhile. But don't believe something just because it was presented at an agile conference. Practice critical thinking.