Emerging digital channels are driving an explosion of useful data: data available in every area, from sensors to social media, our customers to our machines. Yet, without the ability to analyse this information, it is of no value, as we cannot use it to derive actionable insight that will allow us to deliver what our customers want better than our competitors.

Information is a commodity that can be used as a differentiator. But in order to achieve this, businesses need to manage the biggest challenges – ensuring that their data is both available and trusted.

One of the challenges is an ongoing disconnect between business, which needs the data now, and IT, which is often too overburdened to locate, provision or share the data business users are looking for within a timeframe that allows business users to extract the information they need before it is no longer relevant. Although many companies have specialist BI practitioners, their effectiveness is often hampered by the data itself: its accessibility, quality and adequacy for the task at hand.

Self-service BI and analytics, which empower business users to pull their own reports and insights from available data without dependencies on BI specialists within the IT team, are becoming increasingly popular, as they enable users to make faster decisions based on insight for greater agility and enhanced competitiveness.

Moreover, they can reduce salary costs, since the business is not entirely dependent on specialists to provide data insight. Specialists can be freed up to add greater value by tackling the more complex analytical challenges.

However, in order to deliver value, data must be accessible when it is needed. Organisations need to have access to the right data at the right time from the right source. This may mean overnight loads into the data warehouse or data lake, the delivery of real time streams of transactional or IOT sources into streaming analytics engines, or any combination of these. The modern organisation must deliver a variety of different data pipelines, on premises or in the cloud, to serve various business needs.

A recent survey conducted by Syncsort reveals the extent of the challenge, with only nine percent of organisations claiming to be 'very effective' at getting value from their data. 48% indicated that they are 'somewhat effective', which means that almost half of businesses are unable to gain value from their data in any meaningful way.

This is no longer sufficient in today's business environment, where availability of data, and the ability to use it effectively may mean the difference between an industry leader and an also-ran. Improving access to data can also help organisations to deal with other priorities highlighted by the survey, namely increasing operational and workload efficiency, improving the customer experience and reducing costs.

Analytics and BI are unquestionably important for organisations in the Information Age, as they are central to delivering timely and valuable insight. Clearly, however, many companies still struggle, firstly, to access their data and secondly, to derive any value from it.

One of the most significant issues is that data still exists in siloes, with 68% of respondents to Syncsort's survey reporting that siloed data is a problem in their organisation. This means that finding the right data takes too long, and if it can be located, it is often the incorrect information, making self-service analytics impossible.

Data quality becomes increasingly important with the introduction of machine learning, as any inherent errors or data biases will affect the ability of artificial intelligence to deliver accurate insight.

To leverage self-service BI in conjunction with comprehensive analytics requires data to be removed from siloes, provisioned securely and timeously, and maintained in compliance with an organisation's data quality policies.

As business becomes more data-driven, it becomes important to leverage self-service analytics. This in turn requires the ability to deliver timely, accurate and trusted data.

Data engineering - the ability to deliver quality data pipelines - are the initiatives that underpin this ability and enable organisations to begin to gain competitive advantage from their information.

* By Gary Allemann, Managing Director of Master Data Management.