Implementing a Business Intelligence strategy can be painful for any organization. Multiple systems generating and consuming data, data existing on individual machines, propriety formats, multiple sources containing copies of similar data, these can all conspire against a company trying to glean actionable information from their organizational data.
There are ways to mitigate these types of implementation problems by creating a comprehensive plan and organizational structure at the beginning of developing your BI Strategy. It is best to develop this plan prior to the identification of any particular technology. Invariably, if the technology has already been chosen, decisions will be made as to what the technology can and can’t do, as opposed to what the organization needs. That is not to minimize the importance of choosing robust data storage or reporting and visualization technology, but, it is best to have a clear understanding of the business need first.
Business Intelligence professionals are often brought into organizations to help them implement their Business Intelligence strategy after tool and technology selection has been completed. In order to gain efficiency, savings and truly partnering with your consultants, having them as part of the plan before implementation is the best way to increase your ROI. Brining Business Intelligence focused people are in before tool and technology selection will translate to large gains in efficiency and savings in the cost to implement and significantly increase the ROI.
The first step in creating a successful BI Strategy should be creating this Data Governance Plan. Charts and graphs are wonderful to look at, and can offer real insight into your business, but, the technology and software used to provide those are based on the solid foundation of data that you establish at the beginning, through governance. This governance will also ensure better accuracy of data so those beautiful charts and graphs will have real tangle meaning for business decision making.
Creating clear policies at the outset will save time and money. It will provide clear direction to the developers, architects and organizational stakeholders. It will reduce the short term risk of inefficient development and the long term risk of a partially implemented strategy that is full of gaps in capabilities. As such, your data governance plan should be technology agnostic as much as possible. Try to avoid trap of trying to conform a plan to existing or planned technologies. That may not be possible in all cases, but, we’re trying to deal with a data governance utopia.
This data governance plan will lay the appropriate foundation that will support whatever technology is determined to be correct for the company. Without it, there is increased risk of complete failure, spending too much money and time for implementation, or, giving up because getting to your idealistic end state is just not going to happen within your budgetary constraints.
Companies don’t undertake a major initiative without having a plan in place, and a full BI Implementation is a major initiative. Just as a successful author wouldn’t be able to write a full-fledged novel or play without the outline of the structure, plot points and characters already well formed, neither should you implement Business Intelligence without the same level of forethought and detail.
This paper discusses the critical items that need to be addressed before creating your organization’s Business Intelligence strategy. Following these items will establish a concrete, sustainable, forward looking, adaptable and flexible plan around data governance. This will make creating the BI Strategy that much easier as it move towards development and implementation.
The creation of the Data Governance (DG) Plan begins with the people within the organization. While identifying some of the major people such as the Director of IT, the DBAs, CTO, CIO is relativity easy, we should also include a cross-section of the users of the systems that are in the organization. These are the people who know what data the systems capture, what business rules and workflows have to be followed to do their jobs. Their insight will be valuable in establishing the governance plan. A cross functional, cross section of the company will provide the organization with a team that has every area of business covered.
The following questions need to be answered about the people in the organization:
1. Who are the people that have the capability to create and implement the data governance plan? Capability is not just regarding technical skills, but, political clout and the ability to reach across different business units. While they may have some self-interest in some of the plan, they also have an understanding of how the plan can affect the organization as a whole. The have perspective on interplay between business units and how policies implemented in one BU will impact the functioning of another BU.
2. Who are the organization stakeholders for each business unit? These people will be responsible for being your advocates in implanting the plan. If they are not identified and included in the development of the plan, they will feel that the plan is being thrust upon them. Additionally, they may actually have some good ideas.
3. Who are the people that have the ability to implement the technology that is identified? They will be able to help identify any challenges inherent in technology choices.
4. Subject Matter Expert (SME) identification. There are two types of SMEs. There are the ones that know all about how a system is used from a practical perspective. This information is contained in this module, you can look up information here, etc. They also know many of the rules that must be adhered to when entering or updating data. Then, there is the person who knows where and how the data is stored systematically. Identify these people and use them as a resource in your overall strategy. Not only will they help when establishing the plan at the outset (they know where the data is and how to get it), but, they can also be your quality control during testing and final implementation. Nobody knows their data better than them. Use them!
Your DG Plan should attempt to normalize your organization wide nomenclature. Time after time, even in small organizations, certain words and phrases mean different things to different people. This usually isn’t a problem when dealing with people within the same realm of the business, but, it can become a serious problem when interacting with people in different business areas. Terms such as, “Units Sold”, may mean “a count of orders for a particular product” to the sales force, but, to finance, the same term may only be inclusive of “order for products for which we have received payment”, while the shipping department may regard orders sold as “orders for products which have been shipped and delivered”.
It is necessary to come to an agreement on what specific terms mean to the organization. Then, when completed, make sure to compile the dictionary of terms organization wide so that everyone, everywhere is fluid in this new language.
Even in smaller companies, there are very often different systems which contain similar data. This is especially true for Employee and Customer data. Establish which system will be considered the “System of Record”. Establishing which system holds the “Master Record” will help ensure that data pushed to other systems (or a data warehouse) will contain the same data as a the system of record. Some things to consider when choosing the SOR are:
· the current volumes contained,
· how clean is the current data, how long you expect the SOR to be in service,
· how much new data is coming into the system
· Is data pushed to a system from someplace else?
Establish the arbitration rules for disagreements on data and or procedures. Decide who will have final say. This is usually the chair of your data governance committee with the backing of the Executive Sponsor or the Executive Sponsor themselves. Establishing these rules at the outset will help you come to an agreement faster, and reduce the chance that someone’s idea or point of view is being ignored. Give each party in the disagreement a chance to make their case, then, make a decision that is best for the organization, NOT the person making the arguments. Egos are best left at the door.
There are many different sources of data that organizations use. Identifying the source of data at the outset will enable your governance team to see the interplay and overlap that may be existent in your company. There are different types of source data:
If the organization are like most other organizations, then the business doesn’t just consume data, it create lots and lots of data every day. Identify every system in the organization that creates some data. Whether it is a transactional system (order entry, etc.), or a finance system (Billing, accounts payable, etc.), the following information will need to be identified about each and every one of the systems
1. Type of data storage (SQL, Oracle).
2. Is the data storage proprietary?
3. Can the data be accessed programmatically, and how it is it done?
4. What is the security around the data?
5. Does this data source contain any SOX controls that you need to be aware of?
6. Is there an active archiving strategy in place (note: if a Data Warehouse build is part of the BI Strategy then an established archiving strategy should be created.
7. Does this system currently feed data to another system?
8. How are updates to the software applied (automatically, scheduled patches).
9. Which group or person in the organization ‘owns’ this system.
Almost every organization has some piece of data that is stored in a non-production system. It could be stored in spreadsheets, text files, or access databases. Some of this data might be, sales territory assignments, city/state/region hierarchies, etc. It is important to identify this extraneous data that has made its way into your organization lexicon, and establish procedures for production use of this information. Whether a database will be created in the Oracle or SQL Server databases with custom code created to edit it, or establish a Master Data Services solution to maintain this, a clear plan needs to be identified at the outset. This will save pain when one of those other methods accidentally gets deleted or changed and there is no way to revert back to the original.
This is where most companies spend most of their time and energy evaluating tool after tool, having meeting after meeting and grading this feature of this package against this feature of that package. This is a necessary step, but, can be time consuming. It is best to evaluate the tools in the context of the following:
Oracle, SQL, NoSQL are just a few of the very crowded data system and storage marketplace. Not everything has to be stored the same way. Don’t be afraid to build a hybrid environment that contains data in multiple places. The obvious answer for your data warehouse for transactional systems could very well be an RDBMS, but, that doesn’t necessarily work when you are dealing with voice captures from the IVR system, or, image collection and storage for the sales catalogues. Identify what needs to be stored where, and then do the due diligence on what is available. Some of the things to keep in mind when evaluating Data systems and storage are:
1. Licensing costs (Up front and on-going, yearly costs). Is it per seat/core?
2. Implementation costs (FTE Time to implement)
3. Server and disk storage. Is this implementation going to be on premise or in the cloud? If so, rack space and a server will be needed? What will the long-term impact be on the SAN? Is fast storage necessary, or will slow spindle disk storage suffice? If the solution will be hosted in the cloud, make sure to have a complete understanding of what that means. How and who will implement and maintain? What are the costs associated with a cloud implementation, both short term and long term?
This may be where most of the time and money is consumed. Charts and graphs and reams of reports are nice, but, if there is no trust the data that they represent, then the implementation is incomplete. Most visualization tools can give the organization what they want, as long as the data that it is presenting is sound. If, as part of the governance plan, there is a data warehouse implementation then a determination of the order in which different pieces of the information will be implemented. Establishment of the order of priority is necessary. Everyone thinks that their data is the most important, and it is, to them, at least. Try not to let those who speak the loudest set the order. Determine what is important. Quick path to implementation (Quick wins) is one approach, but be careful, the quick win may not have been the wisest play. If your hardest data to put in the warehouse will have components that can be shared in the final solution and benefit other business units, consider following that path, first.
When creating a Data warehouse or an ODS then there will be a need for ETL (Extract, Transform and load) developers. Make sure to choose a tool that has wide availability of external talent to implement, or if choosing a proprietary ETL method that has few available resources, identify where they will come from, and what is the short term and long term costs for them. Remember these things when choosing the tool that will be the backbone of getting the data into the DW or ODS
1. How easy is the implementation?
2. Do you have internal resources that can easily learn to use the tool or already have the skills, or, will you need to see outside assistance?
3. Does it enable you to conform to widely accepted standards?
4. Has the provider shown a track record of not supporting previous development efforts in their latest offerings?
5. Can the technology identified support multiple development paths at the same time?
Now that the governance plan has been completed, the technologies for the systems being implemented and managed have been identified, it is time to make the data available to the organization. Different people will have different needs and different dependencies. Make sure answers are found to the following questions when choosing the data visualization packages. Remember, if the data is good, there is no reason that a hybrid approach can’t work. Just have a plan, and stick to it. If something changes, go back to the governance committee and talk it through.
1. What are the needs of the different data consumers? Some may want just charts and graphs that show trends over time, while some others need to have a list of products shipped the prior day. Charts and graphs don’t do them any good when they are trying to track inventory turns. Determine each group’s needs and wants.
3. How well does the tool chosen implement security?
4. Is it necessary to have scheduled delivery of reports? In what format?
5. Should the data in reports be exportable to other productivity applications? Is this native to the product or is it a customization?
6. Can the tool chosen be used for ad-hoc analysis and data discovery? How?
7. What tool will present the visualizations (SharePoint, Excel, etc.)?
8. Will the visualization tool be on-premises, intranet based, or, client computer based? Will it be available in the cloud (O365)?
9. Will the users be able to create their own reports and visualizations and share them within the organization?
There is much that belongs in your data governance plan. Most of it needs to be established long before the implementation begins, but, taking the time, inventorying the people and the organization, establishing rules and then identifying and implanting the appropriate technology will take the Business Intelligence strategy to the next level.
Most often, it is most effective to have someone outside your organization help design your governance plan. The person or organization chosen will not be burdened by the “We can’t do that because we’ve never done it that way before” mentality. They also have no secret agenda in place and can advise based on what they discover as fact. They can be the true advocate for a data governance plan in your organization.