Data is at the heart of any digital transformation and that data must be “fit for purpose” to create a data-centric organisation that can drive digital transformation. Data needs active management and governance as one of the most valuable assets in any organization. This will become harder to achieve as the volume of data grows exponentially, as well as the number of data sources, and types of data available to an organisation.
The management of data can be divided into two key areas:
- Data Foundations
- Data Exploitation
Data Exploitation often gets the headlines, leadership attention and funding as these are the “smart applications” to turn useable and reliable data into decision-making aids. However, it’s the Data Foundations that enable this to happen.
Data Foundations are increasingly becoming recognised as a blocker to digital transformation and although the real data experts have known this for years, they now have the attention of senior leadership who have seen examples of AI, big data, or digital transformation projects failing to deliver a return on investment (ROI) despite the slick marketing and sales presentations. The solution is to start any data project with Data Foundations using recognised deep data experts.
No-one would start a multi-million pound construction project without proper foundations in place. Big data or other digital transformation projects are no different. Being “data-driven” means getting the Data Foundations right first and then building analytics, data science, AI and Machine learning applications.
To communicate the importance of Data Foundations to leadership, data experts often highlight four key features of Data Foundations that align to the challenges that every organisation needs to solve before pursuing a data analytics, AI or ML initiative:
- What data do I need to solve my business problem? As per the diagram above, Data Architecture aligns an organisation’s needs at the start of the process to the Data Foundations that follow. This should not be confused with IT solution architecture and the IT department.
- How can I ensure my data is usable by data analytics, AI & ML tools? Data Modelling is critical for interoperability and creates a common language for your data, and in a standardised format, which is central for future Machine-to-Machine (M2M) applications.
- How can I trust my data and rely on it for better, and predictive, decision making? Data Quality is critical for transparency and provides a consistent measurable standard for your data.
- How can I efficiently access my data from different silos and sources? A Data Catalogue or Warehouse provides a central place to access your data, with rich metadata, ideally in a way that is extensible and future-proofed to allow for new and different sources of data to be added at a later date, simply and without significant cost.
With this in place, decision-makers and applications can access the right data, at the right time, in the right format, in a cost-effective manner to drive insight and meet their organisation’s business objective. This meets the Government’s FAIR principles (findable, accessible, interoperable and reusable) as laid out in the 2021 National Data Strategy.
Without Data Foundations in place there is significant risk of:
- Getting a poor return on investment from technology expenditure. It doesn’t matter what technology an organisation has acquired, whether a CRM, ERP, or Asset Management Platform, with poor data and the “garbage in / garbage out” principle the outcome will still be sub-standard.
- Sub-optimal decision making. If there are multiple versions of the same data set, or missing data, poor quality data or data from an untrustworthy source (ie. misinformation), decisions based on data will be sub-optimal and the organisation’s performance will suffer in a data-centric market.
- Additional costs and time for finding, sorting and wrangling data. Data scientists are in short supply, and are an expensive resource to spend their time trying to sort data. Unfortunately, they spend up to 80% of their time preparing data rather than exploiting it. With data foundations in place, data engineers can spend their time enriching the data and allow data scientists to build the predictive models of the future.
- Failing to be future proof for Machine-to-Machine Applications. Currently there are humans in the decision-making process with data driving reports, dashboards and visualization tools. The next level of data maturity will be more automated decision-making based on predictive analytics. This needs good quality and trustworthy data and rich metadata.
Envitia works with our customers to first identify the value customers need to achieve from a data project and we identify a measurable return on those data investments. We then offer an end-to-end data capability but always start with Data Foundations to ensure that data is accessible and is trusted – these are the very basics, which do not require a significant financial investment but provide a great starting point.
Our heritage of 25 years of Applied Research projects in Data Modelling, Data Quality, Data Standards as well as our Envitia Data Discovery Platform based on Open Source Components means you can rely on our deep data expertise to deliver the right solution for your organisation.
