Are agilists turning their backs on the very technologies that their clients are building?

I remember three decades ago standing in the bookstore of my university reading sections in a new - and first ever - textbook about the design of large scale integrated circuits. The book was noteworthy because it was created through a collaboration of people from different countries and universities around the world, all communicating via email and other protocols. This was before the Internet, back when networks were owned by universities and DOD and companies, and those private networks using proprietary protocols (e.g., DecNET) were tied together by ad-hoc leased line connections. But it was usually possible to email someone if you knew how to "reach" them, and at the start of any project the first thing we did was establish was way to email everyone.

Technology has opened up the possibility of collaborating in near real time with people around the world. This made that book possible: it would not have been possible before. And I remember reading articles about the book, explaining how continent-spanning networks had made this book possible for the first time: that the knowledge needed for the book had to be pulled together from people working in far-flung companies and universities. And the book was turned around in one year - before the information became obsolete. That was important. Not only was the book a breakthrough, but the process by which it was written was a breakthrough.

We are turning our backs on the very technologies that our clients are building.

The Internet revolution is about communication. Agile did not invent this idea. In fact, in many ways, agile is undoing some of the benefits. Agilists have observed that face to face meetings and physical proximity enable quick discussion and learning by osmosis, but they have drawn the wrong conclusion. The conclusion should not be that "more proximity is better" or that "all collaboration is best if it is face to face". To enforce that is to unwind the benefits of the Internet.

Agilists often point to the value of brainstorming to come up with innovative solutions. But as Susan Cain explains in her breakthrough book Quiet,

"There’s only one problem with Osborn’s breakthrough idea: group brainstorming doesn’t actually work...Studies have shown that performance gets worse as group size increases...The one exception to this is online brainstorming...This shouldn’t surprise us; as we’ve said, it was the curious power of electronic collaboration that contributed to the New Groupthink in the first place. What created Linux, or Wikipedia, if not a gigantic electronic brainstorming session?" *

This is important: it turns out that brainstorming is most effective when it occurs online and not in real time, so that the participants are able to think at their own pace before they respond, rather than having to think everything through in real time.

Historically, the agile insistence on person-to-person collaboration is really a rejection of communication by documents, and for good reason. During the 1970s and 1980s it was common practice to build systems and poorly document them. I remember this. The complaint was with respect to "key person dependencies" - the fact that the people who built a system would leave and no one could maintain it. And there was a response by the IT community to fix this, by insisting that everything be documented. I remember this also. Like the agilist thought leaders today, the thought leaders who were then in control took this to an extreme and the result was that projects started to be planned and measured around documents. Documents became king, and waterfall flourished.

And that did not work.

Agile pushes back on that, but it is also going to an extreme, insisting that all communication be face to face. That is the wrong conclusion, and it is very linear thinking. Rather, the conclusions should be much more nuanced, something like,

Give people the opportunity to talk when they find it best for them, and to write when they find it best for them, and enable this to happen just in time when it is needed. And balance the value of proximity with the value of access to remote talent and the need of people for undistracted thought when doing complex tasks. And don't rely on documents to convey information: the people who wrote the documents must be around to answer questions. But do make sure that you document important decisions as they are made. Little else needs to be documented.

There is a history to this, and we would all do well to not forget it. Otherwise, we are making the same mistakes.

Extremes do not work. Absolutes do not work.

* Ref: Quiet: The Power of Introverts in a World That Can't Stop Talking, by Susan Cain (2012-01-24). (p. 88). Crown Publishing Group. Kindle Edition.

The Emperor has no clothes: Verbal communication is NOT more effective than written communication

It is not true that people communicate better in person that in writing.

The preference for face to face meeting over written communication is deeply entrenched in agile values. The Agile Manifesto (an effort initiated by Alistair Cockburn) enshrines it as "The most efficient and effective method of conveying information to and within a development team is face-to-face conversation".

Cockburn's well known diagram of the effectiveness of forms of communication illustrates the various forms in terms of a tradeoff between "richness" (real time nature, bandwidth) and effectiveness. But this is linear thinking and it does not reflect how effective collaboration actually occurs. To be fair, Cockburn's article on this topic delves into the issues at length, but ignores an important fact: collaboration occurs over time - not in an instant. His diagram reflects an instant, not an process.

Scientific conferences have it right: Their tried and true approach is to first distribute papers on topics, which interested attendees read ahead of time. The attendees then attend the presentations on those topics. Then afterwards, they gather to discuss the topics in person.

The advantage of this approach is that a paper allows the author to lay out a complex argument, without interruption, from beginning to end. Complex issues often require a lengthy statement of one's point of view before the point of view starts to make sense. In face to face conversation, it is too easy to be interrupted, and too easy for the conversation to be diverted into side issues; and a one hour meeting is most certainly too short to lay out a very complex issue and discuss it to resolution.

Cockburn talks about what to do when lengthy discussion is needed. For example, he says, "They worked on it over the weeks, experimenting with representations of their concerns that would allow them to view their mutual interdependence." But he is talking about having on ongoing discussion in which issues evolve over time, because software development is occurring. That is a different situation than what I am talking about. I am talking about when a decision on a very complex issue must be made, and you don't have weeks to mull it over. In that situation, the issue is essentially static - at that point in time - and you need to decide on a course of action, soon.

In that situation, a much more effective process is to first lay out one's position in writing - to peel the onion - and then, after others have had a chance to read and absorb it, during which they build their own mental models of the issues and your position on it; and then meet to discuss it. The discussion can then focus on the points of contention, making the discussion much more effective.

More effective: isn't that what we are after? Certainly, if the issue at hand is not complex, it is often better to just meet on it and talk it through. But if the issue is multi-faceted and requires deep thought, it is far better to first have each person write up their thoughts, and for each to read the other's thoughts, possibly have a few written discussions on certain points, and then meet in person to talk through the points of disagreement and drive to consensus.

That is how effective collaboration occurs.

Contrary to what the Agile Manifesto implies, written and verbal communication are not an either-or proposition: they do not compete with each other - rather, they complement each other.

Does devops change security practices?

Security is still the elephant in the agile room - or should I say the fly in the ointment.

Agile people and business people like to think about functionality that will be delivered quickly; but the sad reality is that if something is delivered that is not secure, it can kill you, in terms of your business reputation. The problem is, how does security fit into agile?

I address this at length in Value-Driven IT and in High-Assurance Design. But now with devops, the concepts need some refreshing. Does devops change things, with respect to security?

On the level of fundamentals, no. The only effective approach to application security is to have developers that understand and appreciate application security, because security affects the design - at least, it should. That often means embedding a security expert in a team to mentor the team, with the strong expectation that some of the people on the team will strive to become experts as well, at which point the embedded security expert can leave. That approach is far more effective than a control-based approach that tries to externalize security by defining rules from a distance.

A very naive approach is to rely on scanning. Scanning is now thought to only detect 5-10% of actual vulnerabilities. (See http://www.cs.umd.edu/~pugh/MistakesThatMatter.pdf and http://samate.nist.gov/docs/SA_tool_effect_QoP.pdf.) Scanning is important - critical even - but it is not sufficient. There is no substitute for developers who are knowledgeable and current on application security.
Since it is hard to teach security and to get people to learn application security, this implies smaller teams of very experienced developers - developers who either know application level security or who want to learn about it. And it implies having enough security experts who can be embedded at least part time to work directly with teams on an ongoing basis. Of course, this needs to be done based on a risk model - one need not make everything as secure as possible.

These things have not changed. But what has changed is that virtualization makes it possible to do security testing earlier.  If one can provision entire test environments (VMs, software-defined network, virtual storage, other resources - including production-like test data) from images on demand via a cloud using OpenStack, AWS, etc., then one can perform security testing at will and do it at regular intervals throughout a development cycle rather than waiting until the end. I am talking about exploratory types of testing of course - things that are manual, like penetration testing. One should also embed automated scanning within the continuous integration test suite.

One of the core agile values, implicit in many of the principles of the Agile Manifesto, is that we really need to elevate people rather than trying to make things better only through better processes: ultimately, you can't improve things unless people increase what they know. This applies to security as well.

Agile is a business thing - not just an IT thing

Business people think that agile is an IT thing. But really, it is a business thing.

Because the decision to "do agile" cannot be made at the point where IT takes over. The decision must be made when the project is conceived, in terms of its business goals.

Project inception generally has two phases in a large organization: when the business case is made, and when the project work starts. To make the business case, you have to estimate the timeline and the costs. To do the cost estimate, you have to do some up front requirements analysis. To do the timeline, you have to make assumptions about the delivery process. Agile changes both of these. Thus, agile must be considered during preparation of the business case.

When project work begins, the business sponsor often contracts for a detailed requirements analysis to be performed. The theory is that this can then be used to solicit bids on building to the requirements; but that theory is deeply flawed, and the horrible track record of IT projects bears witness to this. This up-front detailed requirements is a root cause of failure, because these up-front requirements are usually wrong on many levels. That is where agile comes in: agile allows requirements to evolve.

This means that if you perform an up-front detailed requirements process, you have killed the project from the outset: if done in a waterfall manner, you can be sure that the requirements will be wrong, and if done in an agile manner, you have locked in the requirements so they cannot change, and so agile cannot work.

This is why agile is not just an IT thing: it is a business thing. Project sponsors need to learn and understand how agile processes work so that they can think in terms of the agile cycle from the outset - even when they are making the case.

Don't use frameworks (like SAFe) out of the box (continued)

In my prior post I promised to provide an example of how SAFe is best thought of as a model rather than as a design: a model is a tool for thinking whereas a design is something to implement precisely.

Consider SAFe’s model for portfolio management. The SAFe model defines a portfolio “kanban” process in which work is defined as a set of “business epics”. Just think of a business epic as a project: it is not really, but for our purposes you can think of it that way.

The SAFe process defines the lifecycle of a business epic, from identification of the need for the project, through alternatives analysis, through implementation. This is all pretty standard. The SAFe process is kanban-like in that it defines a single pipeline through which all business epics pass.

That works fine in many organizations, but there are many organizations that have multiple portfolios. In that case, one would need several pipelines. Some organizations tier their portfolio based on the source of funds, or by investment amount: the latter is typical in government agencies. In fact, government agencies usually have mandatory investment management processes and so one could not even use the SAFe model, but one could still use other parts of SAFe. Indeed, the SAFe guidance says that one should expect to have multiple “kanban systems”.

Such complications means that there often cannot be a single backlog. Another complicating factor is that it is often the case that one investment serves multiple strategic goals: yet SAFe presumes that there is a hierarchy of investment themes which are associated with epics, comprising a “release train”.

SAFe also differentiates between “business epics” and “architectural epics”. This is sometimes a useful thing to try to do, but it is not always so clear cut. For example, when a telecommunications company invests in a new network, is that a business epic or an architectural epic? Hmmm. When one adds servers to reduce customer wait time, is that a business epic or an architectural epic? Hmmm. But as the SAFe guidance points out, there might be different sources of funding and oversight for the business and architecture investments, and this might cause these investment categories to be separated.

These are not flaws in SAFe. As I explained, SAFe is a model - at least, that is how I look at it. So to apply SAFe, one should first understand the intent of the model, and then consider how that intent can be realized in one’s situation. In the example at hand, it might mean adjusting the portfolio process defined by SAFe.

My point is: do not view frameworks as designs or templates. View them as models. Create your own design.

Don't use frameworks (like SAFe) out of the box

Is there a "template" for life?

Can you directly implement the advice your mother and father gave you? Or was the advice intended as abstract, requiring you to incorporate it into your thinking, so that you can integrate it with other advice and other knowledge and apply it to each of life's unique situations?

Applying a framework like SAFe exactly as defined is like applying your parents' advice exactly as articulated: it won't work.

SAFe - and the countless other frameworks that have come from IT thought leaders and organizations - is an excellent model, but a model is food for thought. Models always leave out details. Models are a basis for discussion, for analysis, and for design: a basis for design - not a design.

To apply SAFe, you have to think about it, and customize it to your organization. In the next post I will discuss one particular aspect of SAFe in order to illustrate the point.

Does agile encourage bad behavior?

All agile coaches are familiar with projects that “do agile” but don’t really do it. Agile can be used as a license to not plan (throw out the master schedule), not coordinate (cancel all the formal meetings and expect ad hoc collaboration to just occur), have the team commit to a sprint backlog and yet allow the product owner to change the stories during the sprint while still expecting the team to deliver on their commitments at the end of the sprint. These are all known issues to all coaches, but these issues all have to do with behavior that is imposed on a team. What about team behavior? Do teams themselves mis-apply agile ideas in ways that are enablers for bad habits and dysfunctional group behavior?

It took decades for disenfranchised groups to get managers to understand that conversations on the golf course or in the men’s room leaves people out (e.g., women and those who are not personal friends with the boss). Now agile comes along and advocates a return to ad hoc conversations: are we risking a return to patterns of exclusion?

Let’s remember that agile was designed for software development. It should not be applied to general business processes without careful thought. Agile assumes that teams work in close proximity, and so if an ad hoc conversation starts, others can hear it and join in. If teams do not work in close proximity, ad hoc does not work.

Another dysfunction that I see a-lot is when teams do not keep meeting notes. Meeting notes? Isn’t that old-school? Doesn’t that sounds like those old pre-agile dysfunctional meetings where it took three weeks or more to schedule it, and lots of people sat around a table and no one said what they really thought, or there was very low quality discussion, or worse, someone in the meeting got mad because he felt that others had not included him in discussions that occurred prior to the meeting and he felt that the meeting was an ambush? (I have seen and lived through all these things.)

No. Effective meetings require that all participants have take-aways. One of the core practices of Extreme Programming is to document decisions on the team’s wiki: that is a form of meeting note. Meeting notes - in any form that is appropriate - are essential for remembering what decisions were made, and if the notes are done well, they also record why the decision was made. Meeting notes do not have to record what everyone said, but they must mention key discussion points, issues that were identified, and decisions. That’s it. Very short and sweet, very to-the-point, but sufficient for others to read and know and understand the outcomes of the meeting.

Even ad hoc meetings should result in meeting notes, in an appropriate form, and too often they do not: agile’s ad hoc philosophy is being misused to excuse bad business behavior and laziness.

Planning: isn’t that old-school too? No. Agile requires planning. The only difference is that an agile plan focuses only on what matters and not all the details; and the plan is reviewed and updated continually, as things change. But planning is essential. It’s just not about the plan: it is about the planning that is crucial. But the plan is important too, in that it needs to be an information radiator so that everyone can see the current plan and be aware when it changes. The plan forms a reference-able shared understanding across the team and other stakeholders.

Agile should not be an excuse to not plan.

I think you get the idea. Before we throw out every old practice, or apply an agile value to a new situation, we should ask ourselves what purpose existing non-agile practices serve, and make sure that we are improving things and not making them worse. Most traditional practices have an agile equivalent: simply abandoning old practices is not sufficient to become agile.

Agile coaching is not transformation coaching

The Agile Coaching Institute has a wonderful breakdown of the skills that are needed to successfully coach agile teams. The model is useless for the work that I do, however, and I am an agile coach - a transformation coach.

Consider basketball. When pro basketball players think about their sport, they think about the things that they experience, and that are important to them: the plays during the games, the practices, the endorsements. Aspects of basketball - aspects that are central to the sport, such as sponsorships and team management - are on the periphery of a player's thinking.

But if you were to ask a team owner what things matter, they would have a very different perspective. The two perspectives are compared in the figure below.

The same holds true for agile coaching. Agile team coaches experience activities pertaining to their teams, and aspects of software development that are external to teams are experienced - by team coaches - in terms of the way that those things interface to the team; those externalities are experienced as somewhat peripheral. It is kind of like the famous "View From NYC" picture from New Yorker magazine that we have all seen: to a New Yorker, the features of New York loom large, but the features of the rest of the world - while just as important - diminish toward the horizon. In other words, one's perspective depends on one's experiences.

That is why "transformation" is only a single item in the Agile Coaching Institute's list of skills needed for agile coaching: because to a team coach, transformation is just one thing going on, and it is not usually central to what teams think about. Teams are affected by a transformation program, and they participate in it if it exists, but it is not what they focus on each day.

In contrast, an agile transformation coach thinks about transformation every day, and their view of coaching - transformation coaching - is very different from the view of a team coach. The two views are illustrated in the figure below.

The two diagrams are linked: in the team coach's view (adapted from the Agile Coaching Institute's list of core coaching skills), "Transformation Mastery" is a single slice of the pie. In the transformation coach's view, "Transformation" is the whole pie; and in that pie, "Team Coaching" is a single slice. Thus, one set of skills does not subsume the other; rather, they inter-connect.