In a two-part discussion paper, we address some key concerns about data:
- Who owns the Data Governance?
- How to mitigate against self-service polluting data.
Who owns Data Governance?
$600M is estimated to be lost annually through poor quality data in US Businesses, according to the Data Warehousing Institute and then there is the fact that poor data is also the cause of many IT project failures – so who should be responsible for the data?
Data accuracy and consistency of data through organisation should be the remit of data governance. But in reality, data governance straddles both the business needs and the IT function within any organisation and there are no strict rules as to who has the upper hand. Indeed, in some companies it makes more sense to run the project and governance out of IT and in others, it is more logically managed from a business perspective. However, it is clear that both business strategy and IT logic have a necessary part to play, no matter where the ultimate responsibility lies.
What is adding to the confusion in more recent years, is the dark art of marketing who have introduced the term Self-Service Business Intelligence, giving the impression to the uninitiated that they will be able to slice and dice key corporate data in a free-form manner to be able to extract the necessary insight without the ward of IT.
So, what is Self-Service BI?
It’s all in the name when it comes to marketing, but confusion is increasing in the Business Intelligence sector as there is a propensity to claim to be a Self-service BI Tool and yet in many instances what is on offer is purely a means to extract data. Self-Service BI tools are “in vogue” and imply an ease of use but many are simply data extraction tools which move a flat file to and from an excel spreadsheet. However, most users are likely to need information from multiple sources for which you begin to need an understanding of the structure of the databases as well as the skills to do multiple look-ups and data consolidation.
In reality the differentiation between more traditional BI and self-service BI tools is that the new tools are prettier, with dashboards and templates, but invariably they don’t conform to the rules of data governance which should prevent free access to extract and manipulate data.
Having direct access to the data can be a disaster without the right data governance and ownership rules in place.
Once a level of data manipulation is allowed there is the potential to adjust raw data, pollute it with other data sources and also, when you introduce a personal interaction, you are by default providing room for error. Allowing manipulation allows for errors and miscalculations and this opens up a whole world of pain around data governance
Who runs the Data Governance rules?
IT or Finance – who runs data governance?
Different departments will run different rules but in essence the “Business” needs to take ownership of its own data and the IT function, procedures and processes should support that. IT need to provide the tools to the business to enable business insight which comes from the intelligent manipulation of data and that is held in an IT system.
These tools and systems need to know the business needs and then IT can put the right processes, governance and solutions in place to adhere to the necessary data governance rules. These rules can more easily be applied through a data warehouse than a free-form self-service BI tool which jeopardises data integrity. A data warehouse has inherent restrictions and multiple security levels (from standard Microsoft SQL to Active Directory) and so dimensions and hierarchies of the data are restricted to users accordingly. IT can frame what input is mandatory (even at the field and user level) and what output is allowed and even though users will still want access, if they are only provided the information from a data warehouse they know they can respect the value and accuracy of the data.
IT cannot create the business rules nor should it be held responsible to make business decisions concerning the data. IT can only ensure that electronic rules, based on business rules, operate correctly. The dynamic over data governance has shifted from the domain of IT to that of the business who may now set policies and processes that manage, maintain and optimize information. Simply the word “governance” implies a more business-focused approach to managing data, rather than that of the IT department. And this is where problems can emerge as IT try to lead a project or implement a system where they do not have control over the key elements – the data and the rules that apply to it. Undergoing such projects frequently uncovers data governance issues, and the familiarity with the data sources and its journey is where the IT team can lend considerable weight to the resolution of these problems. This is not to say that IT should be overall master of data governance, just that they have an equally important part to play.
A simple 3-point plan is needed;
- Define the project goals (Business)
- Define the policies and processes (IT)
- Define what success looks like and build metrics (both)
Simply put, a partnership between the business team and the technology teams is essential for any data quality management effort to succeed.
- Why Data Governance before BI?
- BI and data warehouse – Why wouldn’t you?
- BI user adoption – Why not?
- “data governance”
- BI implementation
- Forester BI project best practices
- Self-Service BI vs. Data Governance https://tdwi.org/Articles/2015/03/17/Self-Service-BI-vs-Data-Governance.aspx?Page=1
PrecisionPoint – Where data becomes trusted insight