Why Agile task planning does not work

In Extreme Programming Explained, Kent Beck wrote,

In XP, the [elements of planning] are the stories. The [scope units] are the estimates attached to the stories. The [scope constraint] is the amount of time available.

Yet, it seems like every time I have coached an Agile team, the team is compelled by management to do task level planning - that is, decomposing each story into work tasks. On top of this, most of the popular Agile planning tools, including VersionOne, TFS, Rally, and Jira, all have a heavy emphasis on task level planning: e.g., in Rally, you cannot define a story without defining its tasks. As someone who has used Rally a great deal, I found this to be a horrible nuisance.

Task level planning is very counter to Agile in many ways, and I have seen task planning greatly undermine Agile teams. Some of the problems with task level planning in an Agile project are,

1. Task level planning is excessively time consuming; and since planning involves the entire team, this ties up the team for too much time - the team would rather get to work. 

2. Task level estimates are usually wildly wrong, in contrast to story level estimates - which are often very accurate, in terms of their consistency. 

3. The actual tasks needed to complete a story do not reveal themselves until the developer starts working on the story. 

4. Party because of #3, adding up a story's tasks does not yield the time required to complete a story.

5. Task completion does not prove progress - only story completion does: that is the entire point of stories - that a story represents demonstrable progress, and that completion is defined by the story's acceptance criteria and the team's definition of done for its stories. Tasks do not have these attributes. This is central to Agile: waterfall projects are notorious for being "on schedule" until release day, when they suddenly need more time - yet the project hit all of its prior milestones, with all tasks such as "design", "code", etc. completing - but with nothing actually demonstrable. It is the crucible of running software, passing acceptance tests, that proves progress - nothing else does. 

6. Completion of a task often (usually?) does not mean that the task is really complete: since tasks are often inter-dependent, completing one task might reveal that another task - which was thought to be done - is actually not done. For example, a test programmer might write some acceptance tests, but when the app programmer runs them against the story's implementation, the programmer finds that some tests fail that should pass - indicating that the tests are wrong, and meaning that the testing task was not actually done - yet it had been marked as done. Only running software, passing tests, proves that the story is done. Task progress is suspect.

That said, some level of task planning is useful. For example, it makes sense to especially when more than one person is involved in implementing a story, such as a test programmer and an app programmer. One can then have tasks for the story, such as "write automated acceptance tests" and "write unit tests and app code". But, progress should not be measured based on task completion; and it is a total waste of time to come up with estimates for these tasks ahead of time. Instead, it is better to merely have people estimate on the spot the day they plan to work on a task - that is likely to be more accurate than an estimate done a week or two before.


Some of the consequences of paying too much attention to tasks in an Agile project are,

• Parties external to the team, such as the project manager, start to think of the work at a task level, and report progress based on that, with all of its pitfalls (see #5 above).
• Parties who pay attention to task estimates, such as the team lead, will be constantly disappointed, because of #2,3,4 above.
• Teams will lose an entire day or more to planning each sprint, because of #1.
• Team members will collaborate less, feeling that "I did my task - now its in your court", instead of working together to get app code to pass its tests.

Even though many Agile authors talk about tasks, and many "Agile" tools support task level planning, task level planning is antithetical to Agile. As the Agile Manifesto says,

Working software is the primary measure of progress.

Not task completion. Measuring task completion is waterfall. It's Earned Value Management. It's Gantt charts. It is not Agile.