Every data warehouse and business intelligence initiative has risks. Learn to identify them at the start and select the optimal remedy or mitigation technique to address each risk.
It is essential to identify the potential risks within every project and to address those risks before they become major problems. Risk is inherent in any project but the risks involved in a data warehouse project seem to be greater than in others, and there are different types of risks in these larger endeavors. However, each risk has at least one proven remedy or mitigation technique to enable a successful resolution.
No Mission or Objectives
Neither the mission nor the objectives for the data warehouse have been defined. This is a boat without a rudder. It is not clear where the data warehouse is going or what business problem it is supposed to solve. Clichés may be used but they may have no real meaning to either management or to the team.
To avoid this situation, examine the following remedies:
Identify the Data Warehouse Sponsor –The sponsor – usually on the business side – is someone who has a strong stake in the success of the data warehouse. It may be a department head who desperately needs the information the data warehouse will provide. Occasionally it is an IT executive who, aligned with an important business requirement, has a career stake in the success of the data warehouse.
Define the mission and objectives first –Insist – or strongly recommend that the mission and objectives be defined prior to any serious activity on the project. This is an issue of what comes first. Management may see defining the mission and objectives as an activity that is not contributing to moving the project along. Now is the time to bring out the examples of those projects that failed or were delayed because their direction was not clear. Do not start the project without clear mission and objectives, and define the DW scope.
Develop a sample set of missions and objectives –The fastest way to build and get approval for a set of missions and objectives is to develop a sample set of missions and present this set to the IT Advisory Board and the Business Advisory Board and ask for their approval. Expect discussion, disagreement and changes to this set.
Convene a committee –Bring a small group of people together who have a strong interest in the data warehouse and its purpose. This will take a little longer than the previous approach but should result in more support from the organization. The list of missions and objectives would still need the approval of the IT and Business Advisory boards. Carefully select the members of the committee. There will be some important influencers and decision-makers who will not have time for the committee, but be sure to copy them on drafts and to solicit their input.
Data Warehouse Mission and Objectives Do Not Map to Those of the Enterprise
Document explicit enterprise objectives –These objectives can be found in the Chairman of the Board’s letter to the shareholders. It can also be found in the organization’s mission statement, vision statement and in documents outlining the strategic direction. Some examples are:
- Outstanding customer service
- Superior quality products
- Low cost provider
- Fastest delivery to the market
Document implicit enterprise objectives – If there are no explicit enterprise objectives, there may be implicit or assumed objectives to which most people in the enterprise subscribe. These should be documented and mapped to the data warehouse objectives.
If the Data Warehouse does not support the enterprise objectives – If enterprise objectives exist but the data warehouse does not support them, rethink the team’s goals with the data warehouse, and consider data warehouse applications that do support the strategic objectives of the enterprise.
Quality of the Source Data Is Not Known
In most organizations, the quality of the operational data is either unknown or grossly overestimated. Organizations often do not even know how to assess the quality of their source systems. The effort involved in transforming and cleansing the source data can be significant. Without knowing about the source data quality, it is impossible to estimate the effort and time to cleanse the data.
Data quality profiling tool – Use a quality evaluation tool to determine the current state of the source data (good, poor, bad, ugly). Identify operational data with quality problems to someone high in the organization, perhaps to the CIO, but definitely to the business executive who owns the data. Identify the impact of the dirty data on the reliability of decision making, determine the cost for cleansing, determine the benefits for cleansing, prioritize the cleansing effort, and negotiate time, effort, cost, and scope with the user and the owner of the data.
Implement the cleanest data first – There are often multiple files and databases that can be used for the source data. Choose the cleanest source. The data warehouse is always implemented in phases, which are discreet deliverables given to the users. Different phases will be using different source data. Try to structure the phases so that the first phase is able to use the cleanest source data.
Metadata reflecting quality – Quality indicators can be stored in the metadata repository. Choose to assign one of four categories to each data element: “Pristine,” “Questionable,” “Dirty” and “Not Evaluated.”
Skills Are Not Available
It is rare that the team initially has the right number of people in the right roles with the right skills, and that they are available at the right time. To mitigate the risks associated with shortage of skill, choose some or all of the following approaches:
Define responsibilities – Define the functional responsibilities of Data Administrators, Database Administrators, Application Developers, both Back End ETL developers and Business Intelligence developers, Data Stewards and any other required roles.
Develop a project plan – By defining the resources required and when they are needed in the project plan, the DW Project Manager (PM) will be able to identify the need for the right people with the right skills. Without such a plan, the project manager has nothing to support his or her requests.
Identify candidate resources – List all candidate staff members who are being considered for the data warehouse team and evaluate their skill levels. If there is a shortage of skills in the organization, consider supplementing the team with contractors and consultants until appropriate training can be arranged.
Convince management – Management will often ask the PM to operate with a substandard team – inadequate resources and skills. Convince management on the need to have skilled people on the data warehouse team. Also, convince management of the need to have these people sufficiently dedicated to the project. Management may expect to use a person who is 100% committed to another project. This will not work. Use the documentation from discussions with external references where they explained how many experienced and trained people were required to implement their data warehouse. Based on size and project complexity, extrapolate and provide management with a realistic resource requirement.
If management will not support full time resources, hire an expensive consultant (management does not listen to low-cost consultants) and ask them to either present or write a resource recommendation.
Inadequate Budget
It is often difficult to know in advance how much a data warehouse will cost. Data warehouse budgets are often underestimated.
Research – Compile industry publications, presentations and consultant reports that indicate what a typical average data warehouse will normally cost. Watch out for those who give figures for selected subsets of the effort, for those that do not count software or hardware they already have in place or those who do not include costs assigned to some other departments.
Estimate the costs – Itemize each of the costs for your project. Do not pad the numbers, but do not underestimate if the true cost will cause management paralysis.
Think smaller – If the costs are too high, consider a smaller project or one that does not require some expensive items (a new RDBMS, fancy tools, or major new hardware)
Lack of Supporting Software
In many cases, supporting software (ETL, cleansing, BI tools, RDBMS, etc.) have not been chosen or have not been installed in time.
Evaluate software benefits – Understand the benefit of the software to the project. If it does not benefit this specific project, justification can only be accomplished if major follow-on projects will significantly benefit from its use. However, if it does not benefit the project, do not waste time and energy justifying the software.
Cost of not using the software – Quantify the costs of not using the software. These costs should include the additional effort to write code or create operational procedures, the ongoing costs to maintain the code, the costs of a delayed implementation, the increased risk and the potential for reduced quality of the deliverable.
Choose the big hitters – Identify only the software that can make a major contribution. Avoid recommending a piece of software that is fun, leading edge, and a resume enhancer, but does not significantly contribute to the project’s success.
Source Data Not Understood
Most organizations do not have a documented understanding of the source data. The knowledge is probably in someone’s head or is documented on his laptop but this information is generally unavailable to the rest of the organization.
Inventory and model source data – If source data has been neither inventoried nor modeled, it is probably because users and IT management do not recognize the importance of these activities. Any such recommendations may be seen as delaying the project. In truth, the metadata inventory and modeling effort is long and laborious. If management has not already recognized their benefits, it is unlikely that the data warehouse project will support it either. The danger here is that, whereas stand-alone systems could get by without this type of analysis and documentation, integrated systems cannot.
Do not try to model the entire organization but analyze and model those data elements that are targeted for the data warehouse as part of the DW/BI project. Integration cannot occur unless the data relationships are known and can actually be built from the source data, and if the business and technical metadata has been documented and can be managed according to best practices.
Reverse engineering – If a modeling tool is in place that has reverse engineering capability (the ability to take database definitions (DDL), capture them in the tool repository, and generate rough models), this reverse engineering could be the least costly and most acceptable starting point. Unfortunately, many existing databases contain unintelligible content and the reverse engineering process will require significant effort to make the documentation on those databases useful.
Weak Sponsor
For the project to succeed it needs a strong, well-placed user sponsor who makes reasonable decisions.
Solicit the best – Research. Make a list of sponsors and put the strongest ones at the top of the list. Research their decision support requirements and determine which problems could be well-served by the data warehouse. Invite the top candidate to lunch, present the research results to that user, and try to sell them on the benefits of a data warehouse. Explain to them the benefits of the DW/BI system and initiative, outline what would be needed from them and from their department, and ask for their sponsorship.
Solicit the second best – If the top candidate is not agreeable, invite the second on the list. If the second-best person is not willing, stop and do something else. In fact, it is not productive to go too far down the list, since the top candidates are the ones with the most visibility and authority to accomplish a big goal.
Users Not Comfortable with Technology
There will always be users who, based on their experience and their willingness, will be open to try something new. However, there will be other users who may be reluctant to use new technology.
Expand User Support – If users are not comfortable with technology, budget more money for User Support, since the new applications will require additional interactions with this community. Supporting power users is a different approach.
Adjust expectations – Less comfortable users will generate fewer queries. Projections of the volume of queries may have assumed users who were more comfortable with the system. Allow more time for the expected volume of queries to be achieved. Allow more time for the users to be satisfied with the system and to be able to use the system productively. Readjust expectations and those of the sponsor.
Training – Slow down the training so as not to frighten the students. Be sure the students are successful in their workshops. Provide mentors in the training process.
Pre-defined queries and reports – Develop a more comprehensive set of pre-defined queries and reports and communicate their availability to the user community. Take the initiative to add pre-defined queries and reports to the library.
User friendly front-end – Choose an extra-user-friendly BI tool. Choose “warm and fuzzy” over “power and function.” It does not matter about the wonderful features of the tool if the users are afraid to use it. The ideal tool is both easy to learn and easy to use.
Conclusion
Every data warehouse and business intelligence initiative has risks and every risk has at least one proven remedy. Do not let the risks overcome the opportunity to deliver a successful data warehouse, search for the optimal remedy or mitigation.
