Abstract

Enterprise Business Intelligence solutions are complex from an implementation standpoint because of the Develop – Support (Growth-Sustain) cycle followed concurrently. Every enterprise wide BI system continuously evolves over a period of time with new business functionality getting added at regular intervals and they need to be in conformance with existing ones. Also, with continuous evolution of functionality comes the question – “How does one measure the progress”?

This paper addresses the two major problems in managing Enterprise Business Intelligence initiatives, namely:

1. Sustenance of concurrent Develop-Support cycles
2. Calibrating the evolution of business functionality

 
The solution to the vexing problem in development & maintenance of large data warehouses lies in the adaptation of Agile Methodology. Agility in the Data Warehousing context is an approach that "cycles" through the different phases, with the ultimate aim of adding new functionality and stabilizing what is already present. Agile Methodology also provides the platform for measuring / calibrating the progress of Business Intelligence initiatives.

1. Introduction

Business intelligence (BI) is a business management term which refers to applications and technologies which are used to gather, provide access to, and analyze data and information about company operations. Business intelligence systems can help companies have a more comprehensive knowledge of the factors affecting their business, such as metrics on sales, production, and internal operations, and they can help companies to make better business decisions.

Data Warehousing can be considered as the technology domain that facilitates Business Intelligence in any organization. This technology realm of BI has 3 major components:

1. Back-room architecture – Technology components that are used to extract data from source transactional systems, integrate them, transform the data using business rules and load into target data repositories that aid decision making.
2. Front-room architecture – Technology components that help the business users analyze the information present using pre-built & ad-hoc reports and utilize the whole range of analytical solutions.
3. Data Repository – Typically called a Data Warehouse / Data Mart / Operational Data store, this layer models the data in a subject oriented, integrated, non-volatile, time-variant fashion that enables the back-room & front-room architectures to work seamlessly.

Figure 1: Business Intelligence Architecture Landscape

2. Problem Domain

Enterprise Business Intelligence solutions are complex from an implementation standpoint because of the Develop – Support (Growth-Sustain) cycle followed concurrently. Every enterprise wide BI system continuously evolves over a period of time with new business functionality getting added at regular intervals and they need to be in conformance with existing ones. Also, with continuous evolution of functionality comes the question – “How does one measure the progress”?

This paper addresses the two major problems in managing Enterprise Business Intelligence initiatives, namely:

1. Sustenance of concurrent Develop-Support cycles
2. Calibrating the evolution of business functionality

The key challenges involved in managing enterprise BI systems are:

• Evolution in functionality over a period of time
• BI systems drive business decisions - Needs “Total alignment” with the corporate vision
• Development and Support to be managed concurrently
• Very highly user-centric in nature than compared to other technology areas
• Robust measurement techniques are not in place to calibrate such systems

3. Solution Domain

The solution domain is divided into 2 parts:

1. Managing the evolution
2. Measuring the evolution

3.1) Focus Area 1 - Managing the Evolution of BI systems

In this section, we will focus on the different implementation methodologies and establish the fact that Agile Framework provides the best solution for managing the evolution of BI systems. The different phases of “Agility” in the BI context are also elaborated.

3.1.1) Implementation Methodologies
Data warehouse implementation methodologies can be one of the following:

• Waterfall Model
• Spiral Development Model
• Iterative Development Model
• Agile Methodology

Agile Frameworks provide the best value for managing the Data warehouse implementations as they satisfy the key criteria given above.

Agile development is a software development approach that "cycles" through the development phases, from gathering requirements to delivering functionality into a working release.

The ultimate goal of any bottom-up development project should be to roll out new data mart functionality on a regular and rapid basis with a high degree of conformance to what was already there. By adopting specific practices from MSF and XP, we can facilitate the bottom-up, frequent release approach and, even more importantly, change our project team culture and associated behaviors to create better, more customer-focused applications than with the traditional waterfall approach.

Some of the salient points are:

• Shared vision and small teams working on a specific functionality
• Make frequent releases - Agile development strives to deliver small units of functionality that make good business sense.
• Relentlessly manage scope - Meeting a fixed release schedule will not happen unless the resource triangle is actively managed. The resource triangle is the three-way combination of requirements, time and resources. Any change to one leg of the triangle (misunderstood requirement, less time or fewer people) requires a corresponding change to at least one other leg.
• Create a multi-release framework - Agile development stresses that there must be a master plan and a supporting architecture. Use releases to add more customer functionality, not constantly rework what was done in the past

3.1.2) Agility” in Business Intelligence

Two phases to the Agile framework implementation are:
1) Planning Phase
2) Execution Phase

Agile Framework – Planning Phase
Planning is typically done at the end of a particular year for the subsequent year, once the business plans & budgets are finalized.

Assumptions / Pre-requisites:

1. Enterprise BI infrastructure is already present in the organization
2. Technology Architecture (BI Tools/Technologies) and Process Architecture (Standards, Policies, Procedures) are already defined

Agile Framework – Execution Phase
The Execution Phase is for implementing the monthly release. This has the following components:

Agile Framework – Putting Everything Together

The diagram below illustrates the implementation of the Agile Framework in the real-world.

Figure 2: Illustration of Agile Implementation

Some of the salient points in the illustration are:

• Two stories / business functionality envisaged in the planning phase are:

a) Integrating Sales & Marketing Data,

b) Project Accounting Analytics

• The story related to Integrating Sales and Marketing data has 3 phases associated with it namely: Loading Dimension, Facts and Forecast data. Similarly the Project Accounting analytics story also has 3 phases
• Each of the phases is divided into multiple cycles. There are 2 two types of cycles, viz. Development (D1,D2 etc.) and Stabilization (S1,S2 etc.) cycles
• Development cycles involve addition of business functionality while Stabilization cycles ensure that the functionality that gets added is in conformance to the standards enforced in the environment
• Multiple cycles from different phases and different stories are combined together in a release done on a periodic basis

Key Challenges and Mitigation Strategy

3.2) Focus Area 2 – Measuring the Evolution of BI systems

This section describes the way enterprise BI systems can be measured by calibrating it against pre-set goals. At a high level, measurement can be looked at as the alignment of process to certain calibration factors so that the health of the process can be measured with respect to those factors.

Business Intelligence systems are measured with the following objectives:

• Strategic tool to prioritize and align the EDW with the corporate vision
• Measure the evolution of EDW against pre-set goals
• Mechanism to identify technology pain areas and take appropriate actions
• Is a way to objectively communicate the progress of EDW to business stakeholders
• Helps the DW project manager in tactically planning for the immediate future

The Measurement Framework

The Measurement Framework for Enterprise Business Intelligence combines the practical implementation power of the Agile Methodology and the statistical robustness of the Analytic Hierarchy Process (AHP).

There are 3 levels of scorecards that are part of the measurement framework:

• Level 1 (Highest Level) – Actual and Planned Rating of the environment shown on a periodic basis (Figure 3 illustration is for periodicity of 1 month)
• Level 2 – For each period, the rating for different components (“Stories” in the Agile terminology) are arrived at.
• Level 3 – For each component, the score till the end of that particular period is calculated using appropriate calibration factors.

Level 1 Scorecard – Salient Points

• Captures the Planned rating, Revised rating and Actual rating for each period
• Captures the key comments for each point and provides the management a quick snapshot of the progress
• Planned rating is arrived at during the planning phase of the Agile methodology
• Ratings are revised due to changes in priorities of different projects

Figure 3: Measuring BI evolution: Level-1 Scorecard

Level 2 Scorecard – Salient Points

• Level 2 scorecard is a drill-down from the Level 1 scorecard and is focused on the ratings for a particular period. (In this case it is October 07)
• At Level 2, the planned and actual ratings are provided for each component. These components are analogous to projects / stories
• Weightage for each project/story is arrived at using the Analytic Hierarchy process (AHP) during the Planning stage
• The score for each component is multiplied by its corresponding weightage to arrive at a final rating for the component
• For example, October 07 had an actual rating of 78% in the Level-1 scorecard (Figure 3). The scorecard shown below (Figure 4) illustrates the way by which the score for October (78%) is arrived at.
• October Rating = ∑ i=1-8 (Component Rating for i * Weightage for i) = (98*0.05 +85*0.15 +86*0.2 + …….+67*0.1) = 78%

Figure 4: Measuring BI evolution: Level-2 Scorecard

Level 3 Scorecard – Salient Points

• Level 3 scorecard is a drill-down from the Level 2 scorecard and is focused on component ratings (Figure 5 depicts the HR Analytics component)
• Sub-components in the Level 3 scorecard relate to the different phases for a particular project/story in the Agile framework.
• For each sub-component, a list of calibration factors is arrived at. Examples of calibration factors can be Business Functionality, Performance, Data Quality, Stability etc.
• Weightage for each sub-component and calibration factors are arrived at using the Analytic Hierarchy Process (AHP)
• Calibration Factors play a very important role in arriving at the rating. These are arrived at based on organizational needs. Some sample ways of quantifying them can be:
  • Functionality (Dimensions) = Number of Dimension tables created / Total number of targeted dim tables
  • Functionality (Facts) = Number of use cases completed / Total number of identified use cases
  • Performance (Facts) = (1 / Actual Time taken by Fact load) / (1 / Targeted load time as per standards)
  • Data Quality (Facts) = Actual ‘System of Record’ identified for HR measures / Total number of measures
• Sub-component scores (Row scores) are arrived at by multiplying the calibration factor ratings with the calibration factor weightages. Similarly the calibration factor scores (columns) are arrived at by multiplying the calibration factor ratings with sub-component weightages.

For example:
Developing Facts: (Sub-component Score) =
(75% * 0.4 + 47% * 0.3 + 10% * 0.3) * 0.4 = 18.8%
Data Quality: (Calibration Factor Score) =
(12% * 0.2 + 10% * 0.4 + 10% * 0.2 + 10% * 0.2) * 0.3 = 3.1%

• The total score for a particular component (Ex: HR Analytics) is the sum of sub-component scores (row total addition) or sum of calibration factor scores (column total addition). In this example, the total score of 56% is equal to (13.6%+18.8%+11.9%+11.6%) or (32.8%+20%+3.2%)

Figure 5: Measuring BI evolution: Level-3 Scorecard

To summarize, the scores that are measured for each component in the level-3 scorecard are taken for further aggregation at level 2 to arrive the monthly (any periodicity can be defined) scorecard. Level 1 scorecard provides a snapshot of the trend across different periods. The expectation is that all the stories would achieve a score of 100% at the end of the pre-defined timeframe.

Analytic Hierarchy Process (AHP) – A Primer
AHP is one of the powerful methods used in Multi-Criteria Decision Making (MCDM). MCDM is a discipline aimed at supporting decision makers faced with multiple conflicting alternatives. AHP is a systematic procedure that helps to:

• Represent the elements of any problem, breaking it down into smaller constituents
• Assign weightages to each constituent by following a pairwise comparison technique
• Leverages expert judgment and intuitive feel into a coherent framework for problem solving

In this paper, AHP is used at 2 levels:

a) Assign Weightage to each component (Stories) that forms part of the EDW
Illustration in this paper: MDM has a weightage of 0.15, Informatica Upgrade has 0.2 and CEO dashboard has a weightage of 0.3

b) Within each component (Stories) assign weightages to the Sub-components (Phases) and Calibration Factors

Illustration in this paper: For the HR Analytics component, sub-component of “Develop dimensions” had a weightage of 0.2, Developing Facts had 0.4 etc. Also each of the calibration factors had their weightages assigned: Functionality - 0.4, Performance - 0.3, Data Quality – 0.3

4. The Conclusions

The key conclusions from this paper are:

• Enterprise Business Intelligence systems are complex to manage as they constantly evolve over time
• Agile Framework does provide an elegant way for managing the concurrent “Develop-Support” cycles required for Business Intelligence projects.
• AHP based measurement techniques provide a powerful way for calibrating and enhancing BI application performance
• AHP is a simple yet comprehensive way of determining relative importance / weightages among sub-projects that makes up complex systems.

About The Author:
Karthikeyan Sankaran (Karthik) is currently working as a Principal Consultant in the Business Intelligence practice at Hexaware Technologies, a global provider of Information Technology Solutions based in India. Karthik has over 10 years of experience in Business Intelligence domain, having worked as an architect, consultant and project manager for data warehousing projects. Karthik can be reached at karthikeyans@hexaware.com or +91-98400 96512