Introduction to Agile BI: Converting a traditional development team to using agile methods

Part 3 of 4

Bob Blackburn

In part 3, we will look at converting a traditional development team to using agile methods.  Read part 1 reviewing of the key differences between Agile and waterfall here and Part 2 modifying agile methods for a BI project here. Part 4 will conclude with a brief introduction to some Agile Data Modeling concepts.

Here are will look at the steps you go through to transition a team to agile methods. If your organization is new to Agile, think of this as a Proof of Concept project. Start small and allow for some challenges and use it as a learning experience. You should have at least one experienced Agile/Scrum person to educate and guide the project team.

Stage 1 - Time Box and Story Points

1.      See how much working code you can get done in a sprint. Everyone will get better at estimating.
2.      Introduce estimation, product backlog, burn down, and velocity
3.      Most likely fall into Waterscrum or Scrummerfall. Waterscrum is dividing each sprint into a mini waterfall methods where each phase is only a few days each. ScrummerFall is making each phase of a waterfall project equal one sprint.
4.      Can take 2 sprints

This is the initial adjustment phase. You introduce Scrum concepts and work through resistance to change.

Stage 2 - Pipelined delivery

1.      May be hard to keep everyone productive at first
2.      Colocation can now be seen as a benefit
3.      Introduce Pipeline development when productivity drops. By Sprint 2 (See chart in Part 1), everyone is a full capacity and are not waiting for handoffs. Each member is doing design, development, and testing.
4.      When quality of hand offs (Architecture to Developer, Developer to Test) reaches a high level, move to next stage.
5.      Time: 2 –4 Sprints

Stage 3 - Developer Stories and current estimates

1.      Transition to developer stories (E.g. Load Customer Dimension)
2.      Manage Sponsor’s expectations. Change in estimation process. If you are moving to story points, or you can stay with hours.
3.      Develop new current estimate with buildup of experience. After a few months, you will be much better at estimating what can get done (Velocity).
4.      Time: 1 –2 Sprints

Stage 4 - Manage Development Data and TDD

1.      Eliminate data churn. Do this by using a small test data set describe in part 2.
2.      Test Driven Development (TDD). Use automated tools.
3.      Time: 2 –4 Sprints

Stage 5 - Automatic and Continuous Integration

1.      Fail Fast. Use Continuous Integration tools to automate builds and testing.
2.      Can be used as a short cut to Agile if implemented early (stage 1)
3.      Time: 2 –4 Sprints

The average time to become comfortable with agile methods is 8-16 sprints. Try and make your first candidate project something that is in the 9 to 12 months range with a senior development team and a good agile practitioner.

How is Agile worth the effort of converting to? Here is a chart to review the benefits.

Direct Acceleration
Avoids time-Consuming Mistakes
Self-Organized Teams

80/20 Specifications

Better Estimates
Test-driven Development

Coding starts early, feedback sooner

Frequent review of deliverables

Automated Testing

Paying off tech debt early

In the conclusion of this series, we will look at an introduction to agile data modeling.

Additional Reading

Agile Data Warehousing Project Management: Business Intelligence Systems Using Scrum by Ralph Hughes

Labels: ,