The Agile Balanced Scorecard – Tracking the Reliability of an Agile Software Development Team

reliabilityReliability is one of the four performance measures that you should consider when building an Agile Balanced Scorecard. The other three are Value, Velocity and Quality.

The big question: Is the Agile Software Development Team delivering what they say they will?

The main Key Performance Indicator for Reliability in an Agile Sotware Development environment is: the difference between Committed Story Points vs. Delivered Story Points over time – you should aim to deliver as close to 100% as possible (no higher, no lower).

If  the team is consistently under delivering then consider the following:

  1. Have the team calculated their velocity?
  2. Are the team facing regular sprint disruptions?
  3. Are the team bottlenecking at a particular point in the development cycle e.g. QA?
  4. Are the team managing expectations?
  5. Are the team delivering to the original release plan? – Are they delivering the Minimum Marketable Feature(s)?
  6. Are the team sufficiently empowered to only accept what they feel they can deliver?

Delivery reliability exists at two levels.

  1. Ability to deliver the highest value sprint as per their sprint-on-sprint commitments
  2. Ability to deliver the project or programme-level commitments i.e. their release plan

Although Scrum ensures that the highest value products are delivered on a ‘sprintly’ basis, however we must not lose track of delivery at a project-level. This becomes particularly important when there is a Minimum Marketable Feature(s) (“MMF”) associated with a project i.e. we need to deliver a minimum amount in order to realise a return. Release-level or Programme-level delivery reliability becomes less of an issue when we consider Business As Usual or non deadline-driven/MMF-driven commitments.

If the team consistently over commits and under delivers then you could consider this to be a lead indicator (if it hasn’t happened already) to stakeholder dissatisfaction.

If you are experiencing a lot of disruption during your sprint, you may choose to under commit or introduce stretch tasks into the mix – this will protect the perceived reliability of your team.

If your team bottlenecks at particular points in the development cycle then you should consider introducing Work in Progress limits on different steps e.g. only 3 things are allowed to be ‘In Progress’ or the ‘QA’ column at any given point in time. In order for a developer to start a new task, they might need to help QA a task and see it through to the ‘Done’ column.

I’ve included a template that you can use to track reliability over time using percentage delivery plotted against velocity. Feel free to download it below!

Measuring the reliability of an agile software development team.xls

Related Articles:

Reblog this post [with Zemanta]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s