The big question: Is the Agile Software Development Team delivering what they say they will?
The main Key Performance Indicator for Reliability in an Agile Sotware Development environment is: the difference between Committed Story Points vs. Delivered Story Points over time – you should aim to deliver as close to 100% as possible (no higher, no lower).
If the team is consistently under delivering then consider the following:
- Have the team calculated their velocity?
- Are the team facing regular sprint disruptions?
- Are the team bottlenecking at a particular point in the development cycle e.g. QA?
- Are the team managing expectations?
- Are the team delivering to the original release plan? – Are they delivering the Minimum Marketable Feature(s)?
- Are the team sufficiently empowered to only accept what they feel they can deliver?
Delivery reliability exists at two levels.
- Ability to deliver the highest value sprint as per their sprint-on-sprint commitments
- Ability to deliver the project or programme-level commitments i.e. their release plan
Although Scrum ensures that the highest value products are delivered on a ‘sprintly’ basis, however we must not lose track of delivery at a project-level. This becomes particularly important when there is a Minimum Marketable Feature(s) (“MMF”) associated with a project i.e. we need to deliver a minimum amount in order to realise a return. Release-level or Programme-level delivery reliability becomes less of an issue when we consider Business As Usual or non deadline-driven/MMF-driven commitments.
If the team consistently over commits and under delivers then you could consider this to be a lead indicator (if it hasn’t happened already) to stakeholder dissatisfaction.
If you are experiencing a lot of disruption during your sprint, you may choose to under commit or introduce stretch tasks into the mix – this will protect the perceived reliability of your team.
If your team bottlenecks at particular points in the development cycle then you should consider introducing Work in Progress limits on different steps e.g. only 3 things are allowed to be ‘In Progress’ or the ‘QA’ column at any given point in time. In order for a developer to start a new task, they might need to help QA a task and see it through to the ‘Done’ column.
I’ve included a template that you can use to track reliability over time using percentage delivery plotted against velocity. Feel free to download it below!
- Agile Balanced Scorecard – Measuring the Effectiveness of an Agile Team (agile101)
- The Agile Balanced Scorecard – Tracking the Quality of an Agile Team (agile101)
- Using Velocity to measure the Productivity of an Agile Development Team (agile101)
- Minimising the impact of Sprint disruptions (agile101)
- What’s my team velocity? (agile101)
- Work-In-Play Limits in Agile Software Development (agile-software-development.com)