Are agilists turning their backs on the very technologies that their clients are building?

I remember three decades ago standing in the bookstore of my university reading sections in a new - and first ever - textbook about the design of large scale integrated circuits. The book was noteworthy because it was created through a collaboration of people from different countries and universities around the world, all communicating via email and other protocols. This was before the Internet, back when networks were owned by universities and DOD and companies, and those private networks using proprietary protocols (e.g., DecNET) were tied together by ad-hoc leased line connections. But it was usually possible to email someone if you knew how to "reach" them, and at the start of any project the first thing we did was establish was way to email everyone.

Technology has opened up the possibility of collaborating in near real time with people around the world. This made that book possible: it would not have been possible before. And I remember reading articles about the book, explaining how continent-spanning networks had made this book possible for the first time: that the knowledge needed for the book had to be pulled together from people working in far-flung companies and universities. And the book was turned around in one year - before the information became obsolete. That was important. Not only was the book a breakthrough, but the process by which it was written was a breakthrough.

We are turning our backs on the very technologies that our clients are building.

The Internet revolution is about communication. Agile did not invent this idea. In fact, in many ways, agile is undoing some of the benefits. Agilists have observed that face to face meetings and physical proximity enable quick discussion and learning by osmosis, but they have drawn the wrong conclusion. The conclusion should not be that "more proximity is better" or that "all collaboration is best if it is face to face". To enforce that is to unwind the benefits of the Internet.

Agilists often point to the value of brainstorming to come up with innovative solutions. But as Susan Cain explains in her breakthrough book Quiet,

"There’s only one problem with Osborn’s breakthrough idea: group brainstorming doesn’t actually work...Studies have shown that performance gets worse as group size increases...The one exception to this is online brainstorming...This shouldn’t surprise us; as we’ve said, it was the curious power of electronic collaboration that contributed to the New Groupthink in the first place. What created Linux, or Wikipedia, if not a gigantic electronic brainstorming session?" *

This is important: it turns out that brainstorming is most effective when it occurs online and not in real time, so that the participants are able to think at their own pace before they respond, rather than having to think everything through in real time.

Historically, the agile insistence on person-to-person collaboration is really a rejection of communication by documents, and for good reason. During the 1970s and 1980s it was common practice to build systems and poorly document them. I remember this. The complaint was with respect to "key person dependencies" - the fact that the people who built a system would leave and no one could maintain it. And there was a response by the IT community to fix this, by insisting that everything be documented. I remember this also. Like the agilist thought leaders today, the thought leaders who were then in control took this to an extreme and the result was that projects started to be planned and measured around documents. Documents became king, and waterfall flourished.

And that did not work.

Agile pushes back on that, but it is also going to an extreme, insisting that all communication be face to face. That is the wrong conclusion, and it is very linear thinking. Rather, the conclusions should be much more nuanced, something like,

Give people the opportunity to talk when they find it best for them, and to write when they find it best for them, and enable this to happen just in time when it is needed. And balance the value of proximity with the value of access to remote talent and the need of people for undistracted thought when doing complex tasks. And don't rely on documents to convey information: the people who wrote the documents must be around to answer questions. But do make sure that you document important decisions as they are made. Little else needs to be documented.

There is a history to this, and we would all do well to not forget it. Otherwise, we are making the same mistakes.

Extremes do not work. Absolutes do not work.

* Ref: Quiet: The Power of Introverts in a World That Can't Stop Talking, by Susan Cain (2012-01-24). (p. 88). Crown Publishing Group. Kindle Edition.

The Emperor has no clothes: Verbal communication is NOT more effective than written communication

It is not true that people communicate better in person that in writing.

The preference for face to face meeting over written communication is deeply entrenched in agile values. The Agile Manifesto (an effort initiated by Alistair Cockburn) enshrines it as "The most efficient and effective method of conveying information to and within a development team is face-to-face conversation".

Cockburn's well known diagram of the effectiveness of forms of communication illustrates the various forms in terms of a tradeoff between "richness" (real time nature, bandwidth) and effectiveness. But this is linear thinking and it does not reflect how effective collaboration actually occurs. To be fair, Cockburn's article on this topic delves into the issues at length, but ignores an important fact: collaboration occurs over time - not in an instant. His diagram reflects an instant, not an process.

Scientific conferences have it right: Their tried and true approach is to first distribute papers on topics, which interested attendees read ahead of time. The attendees then attend the presentations on those topics. Then afterwards, they gather to discuss the topics in person.

The advantage of this approach is that a paper allows the author to lay out a complex argument, without interruption, from beginning to end. Complex issues often require a lengthy statement of one's point of view before the point of view starts to make sense. In face to face conversation, it is too easy to be interrupted, and too easy for the conversation to be diverted into side issues; and a one hour meeting is most certainly too short to lay out a very complex issue and discuss it to resolution.

Cockburn talks about what to do when lengthy discussion is needed. For example, he says, "They worked on it over the weeks, experimenting with representations of their concerns that would allow them to view their mutual interdependence." But he is talking about having on ongoing discussion in which issues evolve over time, because software development is occurring. That is a different situation than what I am talking about. I am talking about when a decision on a very complex issue must be made, and you don't have weeks to mull it over. In that situation, the issue is essentially static - at that point in time - and you need to decide on a course of action, soon.

In that situation, a much more effective process is to first lay out one's position in writing - to peel the onion - and then, after others have had a chance to read and absorb it, during which they build their own mental models of the issues and your position on it; and then meet to discuss it. The discussion can then focus on the points of contention, making the discussion much more effective.

More effective: isn't that what we are after? Certainly, if the issue at hand is not complex, it is often better to just meet on it and talk it through. But if the issue is multi-faceted and requires deep thought, it is far better to first have each person write up their thoughts, and for each to read the other's thoughts, possibly have a few written discussions on certain points, and then meet in person to talk through the points of disagreement and drive to consensus.

That is how effective collaboration occurs.

Contrary to what the Agile Manifesto implies, written and verbal communication are not an either-or proposition: they do not compete with each other - rather, they complement each other.

Does devops change security practices?

Security is still the elephant in the agile room - or should I say the fly in the ointment.

Agile people and business people like to think about functionality that will be delivered quickly; but the sad reality is that if something is delivered that is not secure, it can kill you, in terms of your business reputation. The problem is, how does security fit into agile?

I address this at length in Value-Driven IT and in High-Assurance Design. But now with devops, the concepts need some refreshing. Does devops change things, with respect to security?

On the level of fundamentals, no. The only effective approach to application security is to have developers that understand and appreciate application security, because security affects the design - at least, it should. That often means embedding a security expert in a team to mentor the team, with the strong expectation that some of the people on the team will strive to become experts as well, at which point the embedded security expert can leave. That approach is far more effective than a control-based approach that tries to externalize security by defining rules from a distance.

A very naive approach is to rely on scanning. Scanning is now thought to only detect 5-10% of actual vulnerabilities. (See http://www.cs.umd.edu/~pugh/MistakesThatMatter.pdf and http://samate.nist.gov/docs/SA_tool_effect_QoP.pdf.) Scanning is important - critical even - but it is not sufficient. There is no substitute for developers who are knowledgeable and current on application security.
Since it is hard to teach security and to get people to learn application security, this implies smaller teams of very experienced developers - developers who either know application level security or who want to learn about it. And it implies having enough security experts who can be embedded at least part time to work directly with teams on an ongoing basis. Of course, this needs to be done based on a risk model - one need not make everything as secure as possible.

These things have not changed. But what has changed is that virtualization makes it possible to do security testing earlier.  If one can provision entire test environments (VMs, software-defined network, virtual storage, other resources - including production-like test data) from images on demand via a cloud using OpenStack, AWS, etc., then one can perform security testing at will and do it at regular intervals throughout a development cycle rather than waiting until the end. I am talking about exploratory types of testing of course - things that are manual, like penetration testing. One should also embed automated scanning within the continuous integration test suite.

One of the core agile values, implicit in many of the principles of the Agile Manifesto, is that we really need to elevate people rather than trying to make things better only through better processes: ultimately, you can't improve things unless people increase what they know. This applies to security as well.

Agile is a business thing - not just an IT thing

Business people think that agile is an IT thing. But really, it is a business thing.

Because the decision to "do agile" cannot be made at the point where IT takes over. The decision must be made when the project is conceived, in terms of its business goals.

Project inception generally has two phases in a large organization: when the business case is made, and when the project work starts. To make the business case, you have to estimate the timeline and the costs. To do the cost estimate, you have to do some up front requirements analysis. To do the timeline, you have to make assumptions about the delivery process. Agile changes both of these. Thus, agile must be considered during preparation of the business case.

When project work begins, the business sponsor often contracts for a detailed requirements analysis to be performed. The theory is that this can then be used to solicit bids on building to the requirements; but that theory is deeply flawed, and the horrible track record of IT projects bears witness to this. This up-front detailed requirements is a root cause of failure, because these up-front requirements are usually wrong on many levels. That is where agile comes in: agile allows requirements to evolve.

This means that if you perform an up-front detailed requirements process, you have killed the project from the outset: if done in a waterfall manner, you can be sure that the requirements will be wrong, and if done in an agile manner, you have locked in the requirements so they cannot change, and so agile cannot work.

This is why agile is not just an IT thing: it is a business thing. Project sponsors need to learn and understand how agile processes work so that they can think in terms of the agile cycle from the outset - even when they are making the case.

Don't use frameworks (like SAFe) out of the box (continued)

In my prior post I promised to provide an example of how SAFe is best thought of as a model rather than as a design: a model is a tool for thinking whereas a design is something to implement precisely.

Consider SAFe’s model for portfolio management. The SAFe model defines a portfolio “kanban” process in which work is defined as a set of “business epics”. Just think of a business epic as a project: it is not really, but for our purposes you can think of it that way.

The SAFe process defines the lifecycle of a business epic, from identification of the need for the project, through alternatives analysis, through implementation. This is all pretty standard. The SAFe process is kanban-like in that it defines a single pipeline through which all business epics pass.

That works fine in many organizations, but there are many organizations that have multiple portfolios. In that case, one would need several pipelines. Some organizations tier their portfolio based on the source of funds, or by investment amount: the latter is typical in government agencies. In fact, government agencies usually have mandatory investment management processes and so one could not even use the SAFe model, but one could still use other parts of SAFe. Indeed, the SAFe guidance says that one should expect to have multiple “kanban systems”.

Such complications means that there often cannot be a single backlog. Another complicating factor is that it is often the case that one investment serves multiple strategic goals: yet SAFe presumes that there is a hierarchy of investment themes which are associated with epics, comprising a “release train”.

SAFe also differentiates between “business epics” and “architectural epics”. This is sometimes a useful thing to try to do, but it is not always so clear cut. For example, when a telecommunications company invests in a new network, is that a business epic or an architectural epic? Hmmm. When one adds servers to reduce customer wait time, is that a business epic or an architectural epic? Hmmm. But as the SAFe guidance points out, there might be different sources of funding and oversight for the business and architecture investments, and this might cause these investment categories to be separated.

These are not flaws in SAFe. As I explained, SAFe is a model - at least, that is how I look at it. So to apply SAFe, one should first understand the intent of the model, and then consider how that intent can be realized in one’s situation. In the example at hand, it might mean adjusting the portfolio process defined by SAFe.

My point is: do not view frameworks as designs or templates. View them as models. Create your own design.

Don't use frameworks (like SAFe) out of the box

Is there a "template" for life?

Can you directly implement the advice your mother and father gave you? Or was the advice intended as abstract, requiring you to incorporate it into your thinking, so that you can integrate it with other advice and other knowledge and apply it to each of life's unique situations?

Applying a framework like SAFe exactly as defined is like applying your parents' advice exactly as articulated: it won't work.

SAFe - and the countless other frameworks that have come from IT thought leaders and organizations - is an excellent model, but a model is food for thought. Models always leave out details. Models are a basis for discussion, for analysis, and for design: a basis for design - not a design.

To apply SAFe, you have to think about it, and customize it to your organization. In the next post I will discuss one particular aspect of SAFe in order to illustrate the point.

Does agile encourage bad behavior?

All agile coaches are familiar with projects that “do agile” but don’t really do it. Agile can be used as a license to not plan (throw out the master schedule), not coordinate (cancel all the formal meetings and expect ad hoc collaboration to just occur), have the team commit to a sprint backlog and yet allow the product owner to change the stories during the sprint while still expecting the team to deliver on their commitments at the end of the sprint. These are all known issues to all coaches, but these issues all have to do with behavior that is imposed on a team. What about team behavior? Do teams themselves mis-apply agile ideas in ways that are enablers for bad habits and dysfunctional group behavior?

It took decades for disenfranchised groups to get managers to understand that conversations on the golf course or in the men’s room leaves people out (e.g., women and those who are not personal friends with the boss). Now agile comes along and advocates a return to ad hoc conversations: are we risking a return to patterns of exclusion?

Let’s remember that agile was designed for software development. It should not be applied to general business processes without careful thought. Agile assumes that teams work in close proximity, and so if an ad hoc conversation starts, others can hear it and join in. If teams do not work in close proximity, ad hoc does not work.

Another dysfunction that I see a-lot is when teams do not keep meeting notes. Meeting notes? Isn’t that old-school? Doesn’t that sounds like those old pre-agile dysfunctional meetings where it took three weeks or more to schedule it, and lots of people sat around a table and no one said what they really thought, or there was very low quality discussion, or worse, someone in the meeting got mad because he felt that others had not included him in discussions that occurred prior to the meeting and he felt that the meeting was an ambush? (I have seen and lived through all these things.)

No. Effective meetings require that all participants have take-aways. One of the core practices of Extreme Programming is to document decisions on the team’s wiki: that is a form of meeting note. Meeting notes - in any form that is appropriate - are essential for remembering what decisions were made, and if the notes are done well, they also record why the decision was made. Meeting notes do not have to record what everyone said, but they must mention key discussion points, issues that were identified, and decisions. That’s it. Very short and sweet, very to-the-point, but sufficient for others to read and know and understand the outcomes of the meeting.

Even ad hoc meetings should result in meeting notes, in an appropriate form, and too often they do not: agile’s ad hoc philosophy is being misused to excuse bad business behavior and laziness.

Planning: isn’t that old-school too? No. Agile requires planning. The only difference is that an agile plan focuses only on what matters and not all the details; and the plan is reviewed and updated continually, as things change. But planning is essential. It’s just not about the plan: it is about the planning that is crucial. But the plan is important too, in that it needs to be an information radiator so that everyone can see the current plan and be aware when it changes. The plan forms a reference-able shared understanding across the team and other stakeholders.

Agile should not be an excuse to not plan.

I think you get the idea. Before we throw out every old practice, or apply an agile value to a new situation, we should ask ourselves what purpose existing non-agile practices serve, and make sure that we are improving things and not making them worse. Most traditional practices have an agile equivalent: simply abandoning old practices is not sufficient to become agile.

Agile coaching is not transformation coaching

The Agile Coaching Institute has a wonderful breakdown of the skills that are needed to successfully coach agile teams. The model is useless for the work that I do, however, and I am an agile coach - a transformation coach.

Consider basketball. When pro basketball players think about their sport, they think about the things that they experience, and that are important to them: the plays during the games, the practices, the endorsements. Aspects of basketball - aspects that are central to the sport, such as sponsorships and team management - are on the periphery of a player's thinking.

But if you were to ask a team owner what things matter, they would have a very different perspective. The two perspectives are compared in the figure below.

The same holds true for agile coaching. Agile team coaches experience activities pertaining to their teams, and aspects of software development that are external to teams are experienced - by team coaches - in terms of the way that those things interface to the team; those externalities are experienced as somewhat peripheral. It is kind of like the famous "View From NYC" picture from New Yorker magazine that we have all seen: to a New Yorker, the features of New York loom large, but the features of the rest of the world - while just as important - diminish toward the horizon. In other words, one's perspective depends on one's experiences.

That is why "transformation" is only a single item in the Agile Coaching Institute's list of skills needed for agile coaching: because to a team coach, transformation is just one thing going on, and it is not usually central to what teams think about. Teams are affected by a transformation program, and they participate in it if it exists, but it is not what they focus on each day.

In contrast, an agile transformation coach thinks about transformation every day, and their view of coaching - transformation coaching - is very different from the view of a team coach. The two views are illustrated in the figure below.

The two diagrams are linked: in the team coach's view (adapted from the Agile Coaching Institute's list of core coaching skills), "Transformation Mastery" is a single slice of the pie. In the transformation coach's view, "Transformation" is the whole pie; and in that pie, "Team Coaching" is a single slice. Thus, one set of skills does not subsume the other; rather, they inter-connect.

Disconnect: why business does not "get" the idea of "experiments"

Experimentation is an important agile concept. Experimentation is a risk management tool: the idea is that one tries out a new approach out as an "experiment" before committing wholesale to the new approach. If the approach does not work out, one can quickly change course, and the experiment was a "contained failure".

What business hears is, "Let's play: we will experiment, and reward failure. We will spend your money trying out things that we are not sure will work."

This is a big disconnect, and the agile community bears much of the blame for this disconnect. It is the hubris of the agile community that emboldens it to speak about experiments and other agile practices as if the case for experiments need not be made. When speaking to senior IT people who have decades of pre-agile experience, one really should have some deference, and not expect them to swallow practices with names that are purposefully controversial - even frivolous ("team happiness").

As Carl Sagan said, "extraordinary claims require extraordinary evidence".

Perhaps instead of talking about "experiments", we should talk about "proof-of-concept" or "pilot". Experienced IT people understand those concepts, and they are really the same thing. It is counter-productive to use new terms - terms that have a shock effect - when trying to convince management in an established organization to adopt a new practice.

One thing that is new about the concept of experiments is that failure should not be viewed as negative: failure causes learning, and therefore better decisions are made after that point. It is a sad fact that in most large organizations, any kind of failure is damaging to one's career - even if the experiment was daring and innovative and caused learning. Agilists want to encourage a culture where prudent and careful risk taking is accepted and rewarded - even if it sometimes results in a contained failure.

But failure is still failure: it results in sunk costs - lost time, wasted effort, wasted money. Management needs assurance that failure - even contained failure - actually results in learning, and that the failure was unavoidable. They want to know that teams are being thoughtful and are using their best judgment and the best information available before they try something that results in failure. It is up to teams to instill that confidence, and it is up to management to be open to encouraging risk if the team demonstrates that it is cautious and thoughtful before it undertakes an experiment.

Are we upholding our end of the bargain?

Apply critical thinking at agile conferences

The book Critical Thinking Strategies For Success compares what it calls "sophistic thinking" with "strong sense critical thinking". The former is when there are doctrines and everyone nods their head yes to anything that supports those doctrines.

Recently I attended AgileDC 2013, and I noted that there was a talk by someone who I know to be incompetent and who does not know what he is talking about: in fact, he was fired from his last company for that reason; yet he was presenting at AgileDC and he has a large following in the community. That community does not know, however, that in real world situations, this person cannot perform because his knowledge does not extend any deeper than platitudes. He does not have enough real world experience to turn the platitudes into action.

Another person speaking at the conference laid out an approach that I know for a fact is not the approach used in the organization in which that person works, yet this approach was presented as a cornerstone approach. Again, after sufficient platitudes, all the heads nodded yes. More sophistry.

We are not doing enough critical thinking in the agile community. We need to be skeptical. Just because someone says something at a conference does not make it so, and where is the proof that they actually did what they say they did? Unlike scientific conferences, agile conferences are practitioner conferences, and the work presented is not research that has been replicated under controlled conditions, and there is no standard of ethics that is being enforced to ensure that people are held accountable by their respective organizations for presenting accurately. In fact, there is plenty of incentive to spin things because it enhances the careers of the presenters and the reputations of their organizations sponsoring those presenters. AgileDC - and most practitioner conferences - are more marketing than they are reality and we have to keep that in mind.

These conferences are still valuable though. There are lots of good ideas that are shared: we just need to be skeptical because some bad ideas can be made to sound viable when they are not. There is networking that happens at agile conferences, and that is always worthwhile. But don't believe something just because it was presented at an agile conference. Practice critical thinking.

Pre-agile helps one to understand agile

One of the greatest mistakes of the agile community is to compare agile with waterfall. There is an assumption that before there was agile, there was waterfall, and that most projects were waterfall. That is not my experience.

It is not fair to compare a well run agile project with a highly dysfunctional project using waterfall, yet that is the comparison that is routinely made.

During the 1980s I was on a string of very successful IT projects, and none of them were waterfall. On the other hand, I was on one project that was not successful, and it was a waterfall project.

The projects that were successful (all non-waterfall) were at two companies: Intermetrics, and CAD Language Systems. These companies built compilers and other advanced tools that were used to design hardware systems. This was major league programming. Our projects were characterized by small teams (3-15 people), lots of collaboration, evolutionary design, and lots of other practices that agile claims credit for.

These experiences have helped me enormously to understand agile, because I can look at agile practices and compare them to earlier practices that I saw work well - even though they were often done slightly differently - so I can discern what really makes those agile practices important. I can also discern that certain agile practices are not critical, because I saw projects be successful without those practices. Standups for example: of all of the successful projects that I was on during the 1980s, none used the practice of standups, and so I am confident in saying that standups are not important. Another practice that is a red herring is the team room: during the 1980s, programmers had their own offices (at least they did everywhere I worked), yet we collaborated continually - separate offices were not an impediment, as long as we were co-located. Co-location was important. And I distinctly recall closing my door from time to time so that I could maximize the quiet to think deeply about a problem, and then open it again when I had finished thinking. Thus, the ability to shut out the world to think deeply was also important. The open door was a universal signal that you were open to someone walking in to discuss something: the closed door was the reverse.

Each of those projects that were successful had a person who was responsible for making sure that everything fit together: someone who was charged with thinking about the entire system in an end-to-end manner. That was essential, and when there was no such person, or when the person was incompetent (that was the case on the waterfall project) things went wrong very quickly. I have also seen agile projects flounder for lack of such a person. The theory that the entire team is responsible for design is kind of like communism: it is a nice egalitarian theory, but in practice it seldom works - I won't say "never" because there are always exceptions. Generally speaking, there needs to be a qualified person whose main job is to think end-to-end, even if that person also does coding. The real issue is what type of person that should be, because at other times in my career I have had nightmare project managers or technical leads who almost single-handedly made everything go wrong (the waterfall project was like that).

In the course of these 1980s projects, the two things that I found to really make a difference in a project were:
    ▪    Small team: if there are so many people that they cannot keep track of what they are each working on, then communication breaks down and code diverges.
    ▪    Servant leadership: Someone who provides gentle leadership to the team: not someone dictatorial, but someone who keeps track of what everyone is doing and what challenges they have on a daily basis; ensures that people discuss issues that need to be discussed; asks hard questions, listens, and proposes solutions but rarely dictates them; and who also understands all of the issues - someone who is knowledgeable about what the team is working on and how it will work. In my experience, self-organization cannot substitute for a good servant leader.

From there, things kind of take care of themselves! With good servant leadership, you will end up with continuous daily regression testing (we did), you will have information radiators on testing results and on the evolving design (we did), you will have a continuous feature-driven or story-driven process with testable features or stories (we did), you will have continual design discussions as needed throughout the project (we did), the team members will feel empowered to work in their own way and contribute ideas and innovation (we did and did), and there will be a sense of harmony, order, and calm rather than an atmosphere of crisis and frustration. Servant leadership is really the key: everything else will follow, as long as the project is not hamstrung from the beginning by having a team that is too large or by having other poisonous situations imposed from the outside.

Even the practice of developing requirements incrementally is not new: circa 1980 I worked at American Electric Power as a nuclear physics simulation engineer, and there was a programming team that supported us, and the method in which we interacted with them was such that the programmer would sit with us and talk about what we wanted, they would go away and develop some of that, then come back and show us to get feedback, then go away and build some more - sound familiar?

So when I reflect on the Agile Manifesto today - or when I did after it was published - I see it as a rejection of the wrong paths that some projects - waterfall projects - took before that and a return to what worked. It was not new, but rather it was a validation of key things that had worked in the past, and that historical perspective helped me to understand the motivation behind each value and each principle and what its intent really was. And yes, agile does add some tweaks to some of those historical practices: that is valuable contribution, but the historical perspective is just as - I would say more - valuable.

I want to run an agile project

There is a very humorous cartoon on YouTube called “I want to run an agile project”. It depicts a young and enthusiastic project manager who sets out to run an agile project in an organization that is not accustomed to agile. The video follows the poor project manager as he goes from department to department trying to overcome one institutional barrier after another.

Of all the barriers that he encounters, only one pertains to the software development team: it is a scene in which he tries to convince two team member to pair and collaborate. All of the other barriers have to do with policies and rules that the organization has – rules that impede the agile process.

This is why agile IT transformation actually has only partly to do with agile teams: it has much more to do with the way that various organization functions are run, including IT and its internal functions, as well as external functions such as contracting. Agile transformation consists of convincing and educating these various stakeholders; it also consists of training teams and coaching teams, but if one does not give equal – or greater – attention to the impediments that are external to the teams, then the transformation will proceed very slowly and possibly lose momentum.

The problem of agile transformation is therefore not so much a problem of scaling agile: it is a problem of enabling agile. Scaling pertains to having many teams on a project, or coordinating multiple agile projects. That is certainly part of the problem of becoming agile, but becoming agile must also address how teams are supported by the various IT support functions that large IT organizations have, including data center operations, enterprise architecture, IT risk management and governance, IT security, data architecture, release management, IT portfolio management, and so on. Many of these functions need to change to accommodate agile, but these changes are huge and impact the missions of these groups, and so this change must be worked in a gradual and inclusive manner. This is an enterprise change management process – often the province of management consulting – informed by agile values and practices. It is much more than “scaling agile”.

In undertaking an agile transformation, one must focus on the goal. The goal is not to implement agile: that is not a business goal. Rather, the goal is usually to make the organization more nimble (“agile”, in the dictionary sense) – i.e. to increase business agility. Business agility is not the same thing as agile in the sense of agile software development. Agile software development is a tool for enabling business agility, but business agility is more than that and differs in many ways. Some business agility strategies rely on significant command and control – approaches that are antithetical to agile software development. Melding agile software development with the way the rest of the organization works, so as to enable business agility – and doing so with approaches that are compatible with the strategies that are being adopted by the other parts of the organization – is the challenge of an agile IT transformation.

Problems with facilitation methods

Facilitation is a core skill for agile coaches, and most of us are pretty good at it. There are some practices that I have seen that can be problematic however.

Dot Voting

The whole point of dot voting is to rank things by importance, priority, urgency, or some other scale - and to use the participants as the deciders so that they feel ownership of the ranking.

But what if the participants do not have the judgment needed to properly rank something?

For example, consider a group of diverse participants - including many agile novices - that is ranking the agile practices that they want to focus on. The ranking will most likely end up reflecting the sources of pain that they currently feel. What it will likely not reflect is the root causes, because it takes a Ri level agilist to understand root causes. And we all know that if we do not address root causes, we will not solve a problem.

So the implication here is that if the facilitator has not drilled into the practices and discussed root causes with the participants, the root causes will not be reflected in their ranking, because the participants are diverse and many are therefore new to agile and will not appreciate the root causes.

The lesson: be careful what you rank, and what you do with the ranking. In the example above, if the goal is to identify practices to talk about, and talk through root causes, voting will achieve that. But if the goal is to identify what practices to focus on, it will not be effective, because the participants do not have the judgment required to make good choices about that.

Not Allowing the Facilitator To Voice an Opinion

A central aspect of facilitation is that the facilitator should not bias the group. But what if the facilitator is an expert in the topic being discussed? What do you do then?

We probably all know the answer to this: you guide the group by asking hard questions, rather than telling them the answer. In fact, they probably have some local domain knowledge that you do not. But what if the group needs to be informed by your expertise?

One technique is to explicitly take off your facilitator "hat" by saying something like, "Ok, allow me to explain what I know on this", and then give a brief explanation based on your expertise. When doing this, I usually punctuate it by saying something like, "So that is the accepted approach to that, but it is not necessarily what we have to do here, because our situation might be unique". That last part lets the group know that they are still in control: they can decide to go against standard practice. Every time you share your expertise, you again re-iterate that it is an accepted view, but that the group can depart from that if it wants to. You then resume facilitating and have complete willingness to record and support choices that go against what your expertise advises. You have done your job to inform, but then the group decides the content of the discussion.

Putting Cards On the Wall

Putting cards on the wall is a long-standing practice for facilitation. I personally first encountered this technique when I participated in a six week (all day, six days a week) modeling session with Peter Coad, David Anderson, Jeff DeLuca and others in Singapore in the late '90s. The purpose of this technique is to encourage people to voice their opinion on something: if you just ask for opinions, some people remain silent. If you give them cards and tell them that they have to write something, they will. It gets all the ideas out in the open.

The problem is, people often write small, or illegibly, and so you cannot read what they wrote unless you go up close to the cards. And if you have a group of more than five people, it starts to become difficult for people to see past others as they crowd around - especially the smaller people. Further, if there are many cards (say, more than 20), some people will not read them all.

Having people stand up close to the cards has another problem: standing uses working memory and consumes a tiny bit of your focus, and standing in close proximity to other people who are shifting around uses even more working memory and focus. Try this experiment: while standing, perform some long division in your head. Now sit, and repeat the experiment (using different numbers of course). You will find that while sitting, you can think more deeply and therefore do the arithmetic more easily. You might think that standing is something that you can do on autopilot, but it actually does consume some mental energy. Sitting, with everyone else in the room stationary, allows you to focus better on purely mental tasks. Sitting is therefore better for the participants of a facilitated session if you want to get their best - their deepest - thoughts. This does not apply to the facilitator because the facilitator's attention is mostly on the group - not the topic. The facilitator has to focus somewhat on the topic, but his or her primary focus is on the people, and the direction things are taking, and standing is also important for the facilitator in order to establish a sense of authority over the process. The people who need to think deeply are the participants.

One myth about the use of cards is that writing cards enables things to go more quickly. The purpose of facilitation is to establish a shared way of thinking about a problem. That means that all ideas that are expressed - as cards or otherwise - need to be mentally processed, one by one, by everyone in the room. It is an inherently serial process, so don't be fooled into thinking that you gain time by having people simultaneously writing their ideas on cards. Saving time through concurrency is not the purpose of the cards. Each card still needs to be read by each person, or the facilitator can read each card aloud. But if people cannot see the cards, they cannot then sit back and reflect on them: they will not remember what each card said and they will not be able to "connect the dots" in their heads. Even if you do affinity analysis, it is often the case that very critical things are mentioned by some cards in an affinity group, and so just looking at the grouping is likely to miss major ideas.

In order to enable participants to sit, and to ensure that all ideas are heard and read and can be contemplated by everyone, I re-write each card on the whiteboard. I do this as I read each card, so it consumes little additional time. I write each idea large and cleanly (legible writing is an important facilitation skill) so that everyone in the room can read it, and then we discuss it. Once all ideas have been written and discussed, we can discuss all the ideas as a whole, coming up with holistic strategies that address all of the ideas. I find that this works much, much better than having cards on the wall.

Apple Donut Headquarters - Agile, or Anachronistic?

Everyone reading this post has no doubt heard about Apple's new headquarters, under construction:
http://bangphotos.smugmug.com/001-News-1/Bay-Area/apples-proposed-new-office/i-QgsWpVh/0/L/ssjm1013apple004-L.jpg

From an agile perspective, the Apple Donut seems very "agile":
It promotes lots of collaboration, because it is only four stories (no getting on a elevator to go see someone) and it encourages one to walk past other teams on the way to a meeting or one's primary work area.

But on the other hand, it seems to me like the logical conclusion of 20th century industrial age thinking, in which masses of people travel to a central location every day, work intensely for a hierarchical organization, and then travel back at the end of the day - kind of like Metropolis (http://www.imdb.com/title/tt0017136/). A glance at the photo (see link above) of the planned Apple headquarters shows the massive highway leading underground to the parking area - not too unlike the river of people flowing in and out of Metropolis every day! I can almost hear the factory siren signaling the start of work. And as the San Francisco Mercury News put it, the new headquarters "promises to bring a world-class real-estate project - along with a lot of traffic congestion - to the heart of Silicon Valley." I don't know about you, but it takes a pretty high incentive for me to suffer an unpleasant grid-locked commute every day.

To be fair, the elite of Metropolis did not espouse agile principles: the movie depicts no signs of collaboration, but rather only hierarchical control with pre-defined jobs.

But is it really that different?

No matter, because a much more pertinent question is, Is Apple a model for other organizations? Should we be trying to learn from it, to inform our guidance of our clients, for how to structure their organizations?

I contend that the answer is usually no, and in the cases where the answer is no, it is emphatically no.

The reason is that most companies are not like Apple: most companies are not as "cool" as Apple, and don't have an inspiring mission the way that Apple, Google, and some of the other most glamorous tech companies have. Most companies - and most IT work in most companies - is relatively hum-drum, and such companies cannot attract the best and brightest as a mere result of their mission or their "cool factor". Most companies have to attract IT workers based on other traits - including working conditions and compensation. In other words, if getting in and out of the workplace every day is a miserable experience, consuming two hours of one's day in a horrible commute, then the organization had better (1) offer very high compensation, (2) be very "cool" to work for, or (3) expect to obtain only the least qualified talent, because the best talent will choose opportunities that offer either #1 or #2.

But what is the alternative? Apple can get away with a four story Tower Of Babel absurdity, because it is so "cool". But what about the countless other companies? What is the right kind of agile workplace for them?

We have to be careful here, because agile principles emphasize certain things like face-to-face conversation and working together in real time, and it is easy to take those things to their logically absurd conclusion and arrive at the Apple headquarters. But what is the right way to scale those agile principles? Does it mean forcing every conversation to be face-to-face? Every meeting to be in-person? Every collaboration to be real time?

Of course, the answer is no. In fact, global trends are the reverse. In our increasingly global economy, we see more and more workers who are needed in many geographically separated places on the same day, because their skills are move valuable than the value of proximity. We also see an inexorable trend toward flexible working patterns. Two income households and removal of the barriers preventing more flexibility - combined with the increasingly global nature of work - are making this come about. In the IT world, early agile has inserted a small hickup in this trend, by sending IT people back to the office for core hours to be on teams, but the overall trend is there. Now that agile has matured, and the focus is on continuous delivery, teams are discovering that they need to be in continual contact with diverse stakeholders in other parts of the organization - people who cannot be physically present - and it is often the case that the business stakeholders are in other parts of the country.

Early visionaries such as Alvin Toffler were not wrong on this: the trend is toward less commuting, more flexibility, a return to the organization of populations around communities rather than commuting corridors, and the substitution of electronic collaboration for physical presence. Commuting was a 20th century anomaly.

So the question is not whether agile values are right - they are - but rather, how does one achieve agile when the trend is toward more work flexibility, more time zones, and workers who are needed in many places at once - and in and environment in which the best workers can get jobs that give them the flexibility that they need?

That is the real question for agile. And Apple's new headquarters does not answer that question.