Anexinet BI Blog: Introduction to Agile BI: Modifying agile methods for a BI project

In part 2, we will look at modifying agile methods for a BI project. Read part 1 reviewing of the key differences between Agile and waterfall here. In later installments we will discuss converting a team to use agile methods and a brief introduction to some Agile Data Modeling concepts.

How is a BI project different than a typical agile OLTP project?

First we will take a look at the architecture of each.

Web Architecture BI Architecture

As you can see, there are more pieces. Multiple sources, repositories, downstream databases, and applications that increase the complexity. This can be broken down into the Breadth and Depth of Complexity. Breadth includes extracting, cleansing, integrating, and transforming. While depth includes conflicting data definitions, business rules, and high data volumes.

Agile Adaptations

Creating another layer of requests under user stories

These are developer stories. If the user story is being able to view sales by product type and region over time. You may need to break this down behind the scenes and have tasks for product, customer, geography and sales data loads.

Adding additional team roles (A person may fill more than one role.)

Project Architect –business model. Balanced tech solution

Data Architect –logical/physical model

Business/System Analyst –Processing patterns, transform rules, Source to Target Mapping (STTM)

System Tester –Validation

Organizing the team into a pipeline of delivery activities

The architect has to be just ahead of the ETL developer. And, the ETL developer has to be just ahead of the report developer. Etc.

Conducting two lead-off iterations

This will give the Project Architect and the Data Architect time to establish the environment.

Splitting user demos into two stages

The first will be static test data that can be used to testing. The second is production data for full scale testing.

Maintaining many small managed test data sets

Relating to the previous adjustment, the agile team needs quick turnaround of test cycles. Often consisting of multiple times per day. A small set of data is to facilitate this.

Visually the Roles for the Sprints look like this:

As you can see, there are a few steps to come up to full speed; but, you quickly reach full velocity and producing functional BI applications.

In Part 3, we will look at converting a traditional development team to using agile methods.

Sprint	Project Arch	Data Arch	Developer	Testing
-1	A+
0	B	A+
1	C	B	A
2	D	C	B	A
3		D	C	B