Series of Discussion 1: Challenges of BI



In the world of BI, there are key challenges that affect all projects:

(1) Data Governance

(2) Partner Support – The value of

(3) BI User Adoption? Why Not?


Part 1 

Data Governance


Active Data Governance

First and foremost, let’s be practical, data governance is not passive; it is not a review at the end of an IT deployment project – only to see what’s gone wrong! Data Governance is an active system of checks and balances and processes that ensure quality information is delivered.

Governance needs to be considered from solution architecting through implementation to the results, reports and dashboards. It should be an integral part of the data management from input, handling, manipulation and utilization and there should be a clear chain of command that takes responsibility for the technology element of governance.

But is that what usually happens? You and I both know that’s not the case!

And the end result of system deployment; users need to be able to “trust the numbers” and be empowered to access and use them securely. And that won’t be the case if governance has not been applied at every step of the way.

Who, what, where and when

Data governance provides the business practices and IT processes that define and manage the way valuable data is formally protected and used through the organisation, so decisions need to be made prior to any BI project commencing. Detailed thought is required at every point through which the data passes as to;

• Who has the right to access

• Who has the right to take action

• Who has responsibility and accountability and finally

• Who has authority to execute on the data.

The impact on the business of inadequate data governance can result in data breaches, loss of revenue and untold damage to the business. The single most prevalent point of attack is personal credentials which underpin this need for responsibility tracking through governance with policies that dictate how to enter data and who has user rights.

It is worth considering hiring a 3rd party to help build a set of policies that govern and dictate who does what, where and when.

Said Ed Paice of Harness IT (3) who acts as Data Governance and Project Assurance for large ERP conversions; “…I would argue that a BI project will start to ask fundamental questions about a company’s data sources, longevity and accuracy and it’s better to ensure you have those building blocks established before launching into designing of cubes or reporting metrics. We all know that a lot of BI projects come on the back of replacement of an organisation’s operational or ERP system and therein lies the challenge as reports will likely require year against year comparison, however you might not be taking that historical data into the new ERP system. So how to populate your BI cubes?”

This is where a good data warehouse can be the lifeline to legacy data.

After the fact is too late

As a business intelligence project unfolds, it often becomes clear there was not sufficient forethought and planning around the strategy, design and governance of the solution. One significant KPI is the level of customer satisfaction. Recent results emerged from the MSDynamicsWorld (1) survey that showed only 37 % of Dynamics customers who responded expressed that their needs had been met by their BI project.

So that’s 63% who felt it missed the mark!

Mostly the issues at fault were quoted to be complexity and empowerment – demonstrating a lack of data governance from the outset.

To endorse the importance, the Business Intelligence Guide (4) states that “If the data management capability is not in place before rollout of any BI applications, then there will be no-one on the ground to execute data policies, with the end result of governance in a vacuum – no one enabling these decisions.”

Further discussions with Ed Paice of Harness IT (3) endorsed this view; “I had to interject when one project lead was having sleepless nights worrying about how the new operational systems would support the BI metrics which had been mandated from the board. Something had to give, so I put the team together and getting executive buy-in to the problem added a touch of realism and they were able to agree what could realistically be delivered that was still valuable.”

Moral of this story is don’t get hung up on what you can’t accurately report on and spend time struggling to make the metrics fit but work collaboratively to deliver what is possible.

It’s not all about the Tech

So now we have established the importance of data governance and the need to empower users and remove complexity, here are a few pointers that are less about comparing technology, more about the softer issues that should influence your choices

1.Ensure data access rights are made for business reasons and not governed by technology limitations. A steer here is to look for good self-service analytics dashboards from your vendors.

2. Consider agility and flexibility in your system of data governance, as well as a sound methodology. You need to strike a balance between accurate recording and the needs of the business to change over time.

3. Think collaboration when in design mode across all departments. To achieve a great result, you will need buy-in and cooperation.

4. Think of all of your data sources right at the start of the project, looking at both structured and unstructured data and consider a data warehouse as part of the solution.

5. Some things go without saying, but let’s say it anyway; security, platform, mobile, monitoring and management capabilities all need to be tip top.

6. Visual is important; some people think in Microsoft Excel or Microsoft PowerBI or even Microsoft, so ensure you can deliver everything from a pie, bar and line charts, to heat and tree maps, geographic maps and scatter plots and make sure its intuitive to drill down to the detail.

The scary big name vendors

When considering vendors, there is a scarily long list of providers, including the big names like SAP, IBM, Microsoft, Oracle, MicroStrategy and SAS. They all talk about agility, ease of use and the ability to deploy data governance at every step. However that doesn’t mean that governance is always well executed. This is where your choice of VAR and partner is essential. Further endorsing this point is the MSDynamicsWorld survey (1) which revealed that only a staggering 26% had a data governance strategy in place prior to commencing their BI projects.

Room for improvement

So, although the technology has the tools in place, it appears that there is ample room for improvement in the execution. Technology alone will not achieve nirvana; there needs to be a positive strategy to take data governance seriously and where there’s a will there’s a way!


Part 2

 Partner Support:  The Role of the Dynamics Value Added  Reseller


Does the VAR even have a role?

There are probably 3 distinct phases in which the VAR could play a role;

  1. Assessment, shortlist, choice and sourcing – the planning
  2. Scoping, environment, provisioning and deployment – the doing
  3. Resourcing, data governance, budgeting and project management – the managing

In-house rules?

As a customer, you must decide what outside resources you need to deliver the project, phase 1, 2 and 3 or just one element of the above, depending on your in-house skills.

VAR just a little bit more?

Personally, I believe that, as the trusted advisor, the VAR should aim to have the skills and capability to perform all three roles to be able to deliver an excellent BI project.  As the only way to ensure delivery is to be in control of those three key elements in their entirety.  Also for the customer it gives “one throat to choke” and “one head to roll” or hopefully “one” to stand in confidence and liberate the internal resources from the responsibility of delivery, once the goals, activities and metrics have been agreed.

Vendor agnostic?

And what role should the vendor play?  I am a firm believer in the power of the channel, although not always unequivocally “independent” their opinion is certainly going to be more independent than that of the vendor.

Sadly, the MSDynamicsWorld (1) report quoted that 38% of customers stated that their VAR gave them no guidance.  Interesting!  So where is the advice coming from?  Are customers allowing the vendors to employ their own consultants, who may well be experts in delivering their own component, but unlikely to be true system integrators or have the overall project goals top of mind?

Some quotes from the MSDynamicsWorld report give us a better clue as to where the process and project tends to go astray as these VAR comments show;

  • “Their requirements are usually unclear so the solution cannot be implemented. The cost-benefit of third party solutions is not as expected for the customer. Third party solutions are still ‘boxes’ that [do not] generate the info our customers require in the way they need it.”
  • “This is like any other additional initiative involving the original effort in ERP – championed by a few who either get pressed for time or lose interest when as the project rolls on and out.”
  •  “These projects are technically complicated and require bigger investment than expected, not just for the software but customers need to invest in the labour to bring the project together”

Some additional insight can be gained here by talking to those who lead these types of projects every day of the week.

Edward Paice of Harness IT (3) noted; “I would argue that the VAR should be selected because of their industry knowledge as well as their technical skills.  They ought to bring value in both technology and an understanding of the business, so consultants can guide the client through workshops and strategy sessions and assist in framing the project goals and metrics.”

Ed elaborated with an example; “An example might be if you are looking at coverage in a retail or omnichannel world, industry experts would understand when looking at data what it actually means to that organisation, so for instance what would you actually count as stock?  How can that calculation be meaningful without understanding that stock includes that at the warehouse, on the shelves, in the supply chain, at inspection and in quarantine?  And the impact of promotions and discounting that need to be considered for stock coverage.  So data sources may include those outside the immediate company and so this is where the technical knowledge kicks in; knowing that a data warehouse will be required to pool together disparate data structures to make meaningful decisions possible.  A good VAR who understands the convoluted vertical requirements will bring that knowledge to the table.”

It’s not all about the Tech

And to the Magic?

The Gartner Magic (2) quadrant adds their not insignificant weight to this argument by indicating that there is ample opportunity for improvement in market execution and delivery of the solutions.

Referring again to the MS DynamicsWorld (1) report, their research confirmed that; “…technology alone, is not sufficient to deliver the insights that organizations believe they should get from their investment in core systems like ERP and CRM. Dynamics partners and customers will continue to be challenged to commit to the right policies, skills, and plans over the long term if they want to reap maximum value from their data.”

To VAR or not to VAR?

That is an easy question – it’s a yes from me! 

It is all about your choice of which VAR not if you should employ one as far as I am concerned.  And I would err on the side of hiring a VAR that can execute on all 3 phases of the BI project; the planning, the doing and the managing.

Choosing which VAR is an entirely new topic but in essence my recommendation would be to talk to their customers for reassurance that they can execute on all three phases.  Also invest in a proper pre-project workshop to assess, plan and set goals and metrics and responsibilities and put all that in writing.


Part 3

BI User Adoption? Why not?


But I can’t see the benefit! 

How many companies have installed a CRM system only to find that no-one uses it?  And how many of us have been given training that insisted we log pipeline, map sales stages and set activities?  Most of us in the IT industry have been “forced” at some stage to use a piece of software that we don’t like, can’t use and are blind to the value.  We use it reluctantly, kicking, screaming and resisting all the way.  If it takes us longer to be trained than to see the benefits, it is always going to be an uphill struggle.  And BI projects are probably more susceptible to this than most, with their high touch across the organisation and their need for accuracy – it can be a challenge.

So what are the issues?

Gartner (1) describes lacklustre BI adoption, compared to the level of investment, as a result of BI historically being all about a point of record; from data in to data out supporting consistency, stability and accuracy rather than a more modern concept of rapid prototyping and agile manipulation to enable information exploration and all that entails.

In discussions with Ed Paice (2) who manages large mission critical ERP and BI projects on behalf of clients he endorsed the change in attitude and expectation of BI projects;

“From a design and implementation perspective one can go either way; highly formal where one needs to provide exactly what client wants.  Or at the opposite end of the spectrum, one can go agile.  Now, I wouldn’t recommend agile for all projects, but for BI increasingly to drive user adoption you need to be reactive, flexible, responsive and even intuitive.  You need to give users what they “might” want! 

 “So a customer says they want ABC and you give them ABC, but then they realise that they also want D and in most instances the scope creeps and you try to accommodate.  But if you give them AB and talk about C or D then they can choose and then decide that wouldn’t it be nice to have XYZ as well.  You can bet that their requirements will change as they see more of the art of the possible.  If you spend all your time and allotted budget designing architecting and implementing exactly what they wanted at the outset, it will be an unhappy project close.  Letting them take ownership of their own knowledge requirements creates buy-in in itself!”

Vendor responsibilities

Vendors of the newer BI solutions are taking on board the technological advances that bring this flexibility as well as some of the changes in sales and marketing tactics that are also affecting user adoption.

  • Ease of use – the single most significant blocker to adoption is ease of use and this more recently needs to be extended to include agility, flexibility and integration to multiple data sources coming in and multiple reporting tools going out.
  • Results driven adoption – using the “land and expand” technique of more modern SaaS based apps where Fremium leads to pilot and the up-sell to more users as they witness the benefits.
  • Business driven not IT driven – as decisions are taken across the organisation in the above land and expand scenario, so the marketing approach of vendors exploits this opportunity to gain a foothold and traction
  • Proof of concept is another similar tactic that lends itself well to modern SaaS solutions as they do not require an on premise server and IT support.
  • Software costs have also been driven down which was an earlier barrier to entry and this can be seen through the per user or even per concurrent user pricing models
  • People power – never underestimate the power of the old-fashioned personal interaction be that demonstrable in pre-sales, support or training.
  • Enablement – scoring highly in this area makes a huge difference to adoption. Everything from training, online tutorials, documentation, user forums and community sites. In an increasingly social world it is critical for many users to see an active and vibrant community that shares and helps itself.

The BI and analytics market continues to grow (Gartner 1), with new product innovations, greater agility and new GTM (go to market) tactics as well as a significant change in buying decisions.  Expectations have moved on and now extend beyond a “point of record” system to encompass the key elements of consistency, stability and accuracy but with the added drivers of flexibility and integration as key requirements.  The need for data governance also increases and with all these variables to consider the importance of the partners with whom you choose to work becomes more and more significant.

So we have come full circle in our discovery.

For users to be empowered and “trust in the numbers” you need strong data governance and process and to help properly evaluate the growing number of options you need to choose the right trusted advisor and value added reseller who will guide you through the complexities and if you get all that right the people who need it, will use it.

And more importantly feel the value.

Thanks for your time. Your comments and suggestions are always welcomed.

Andrew Mennie



Discussion 1:

  1. MSDynamics released; Investment in BI and Reporting – by Jason Gumpert, published January 29, 2016 2. Gartner Magic Quadrant for Business Intelligence and Analytics Platforms; published 04 February 2016 | ID:G00275847 Analyst(s): Josh Parenteau, Rita L. Sallam, Cindi Howson, Joao Tapadinhas, Kurt Schlegel, Thomas W. Oestreich  here. 3. Edward Paice – Linkedin Page – Harness IT 4. The Business Intelligence Guide; here

Discusssion 2:

  1. MSDynamics released; Investment in BI and Reporting – by Jason Gumpert, published January 29, 2016
  2. Gartner Magic Quadrant for Business Intelligence and Analytics Platforms; published 04 February 2016 | ID:G00275847 Analyst(s): Josh Parenteau, Rita L. Sallam, Cindi Howson, Joao Tapadinhas, Kurt Schlegel, Thomas W. Oestreich
  3. Edward Paice – / / . for LinkedIn

Discussion 3:

  1. Gartner Magic Quadrant for Business Intelligence and Analytics Platforms; published 04 February 2016 | ID:G00275847 Analyst(s): Josh Parenteau, Rita L. Sallam, Cindi Howson, Joao Tapadinhas, Kurt Schlegel, Thomas W. Oestreich here
  2. Edward Paice – / / for LinkedIn