4 Core Aspects Of A Successful Data Integration Project
Table of Contents
Data integration is a fairly easy activity to define, as it’s simply the process of drawing together and combining data from different sources to provide the user with a unified and accurate view of the information.
Such a straightforward definition however masks the reality of how much complexity is actually involved. This is especially the case when it comes to supplier data integration projects. There can be more touch points for supplier data across an organization than for any other data type and, as supplier data is hugely operational, the requirements for precision and accuracy are also very high.
With stakes so high, it means that many supplier data integration projects simply do not achieve their set objective, and many end in failure. This can, however, be avoided.
Successful supplier data integration projects all have one common foundation. They all have clear answers to four key questions before the integration takes place:
- What exactly do you want to do?
- Why do you want to do it?
- Who are the stakeholders?
- How will you do it?
Let’s take a closer look at what these questions mean in practice.
What exactly are you doing? Is it operational or analytical data integration?
There are two main types of supplier data integration use case:
- Operational – integration of data from a number of operational, largely transactional systems
- Analytical – based on operational data, analytical systems collate information for analysis to provide insights and drive business decisions
It is vital to recognize what type of integration it is that you are undertaking, as this distinction will drive the decisions that need to be made when the data is being integrated – and the approach to be followed.
Operational data versus analytical data
For example, operational data integration, as it is invasive and can impact day-to-day operations in the business, will need to consider how to mitigate these risks (i.e. how will the integration take place without disruption?)
It requires also clear decisions to be taken around aspects such as which are the controller systems and which are worker systems – and how data flowing between these systems should be managed, whether that be one or two-way.
Analytical data integration, on the other hand, will require more planning around aspects such as the creation of common dimensional structures – this might be time series, categorizations and labels – that will be required within the integrated data sets.
Determining the context helps to identify the right data hierarchy at the design phase of the integration. It is important to know how each system currently maps to the hierarchy and to ‘design’ what the new hierarchy will look like. This step must be undertaken before any integration work can begin.
Why do you want to do it?
It is then important to understand and articulate the business objectives as part of your data integration project plan – the reasons why the data integration is required, what the end result should look like and the value that will be measured.
Common goals for supplier data integration projects
Common goals relating to supplier data integration might include:
- Enabling greater flexibility when acquiring future ProcureTech applications
- Removing data silos from the business created by disparate software systems and eradicating resulting redundancy or manual workarounds
- Reducing the amount of redundant data or the reliance on one specific or non-standard databases
- Improving data management and ability to govern processes
- Increasing the availability or visibility of data for different teams throughout the organization
- Supporting wider goals of digitalization or digital transformation across the business
The value is derived from the impact of achieving these goals and will vary depending on the objectives chosen.
However, measurable outcomes might include faster adoption of systems, reduction of workloads due to inefficiencies, reduction in enquiries, faster execution of processes or improved levels of data accuracy.
Who are the stakeholders in a supplier data integration project?
There are four categories of stakeholder to consider as part of your data integration project plan:
- Internal users
- Regional users
- Supplier users
- The system
These stakeholders have different requirements which will need to be taken into account as part of the project, based on the various required ‘views’ of the data, an example of which is given below.
Systems as a stakeholder
The key point to remember is that the systems (such as the ERPs) that you want to integrate must be considered as stakeholders.
Do not base your integration around one single transactional system if you can avoid it, as this will create restrictions on how the data can be structured. If there is no choice, you will have to consider how you are going to work around the limitations.
The whole point of creating integrations between different systems is to easily move information between them. When they are connected, data will be moving between them constantly.
The importance of data governance
This means that, before you get into the true detail of a data integration project, you need to have clear data governance processes and definitions in place for what will happen in certain situations – for instance, when data from each system tries to overwrite corresponding information from another.
There might be certain information being passed from the ERP to the supplier information management system (SIM), CRM or other systems that is only editable within the ERP itself. This could be something like an account ID or account number that is transferred across to the supplier information management platform, but can only be edited within the ERP.
Furthermore, governance happens (and therefore must be defined) at multiple levels. It is not just the governance of processes and systems, but it can come down to the detail of who owns which fields in a database, otherwise known as System of Record.
Remember that a centralized system doesn’t mean a system just for one team; rather it is a common system for many teams. The advice: don’t try to build the politics of the organization into the complexity of integrations. This is a frequent misstep which results in too much complexity. It is hard to maintain and ultimately impacts end users.
If you don’t tie up these issues, then integrations become problematic, with a much higher chance of failure. If you can’t establish a governance framework, then migrating and integrating data will be very difficult, there is a risk of orphaned data, and the resulting workflows are also going to end up extremely complicated.
On the other hand, having a robust design plan in place and being able to articulate the business goals and value drivers to the diverse stakeholders will encourage support and buy-in for the data integration project, as well as position the project for success.
How will you do it?
Integrating two systems and setting up a straightforward sync between the two of them is relatively simple, because there are only two possible directions in which information can travel – data goes in, and data is exported out. However, as soon as you start adding other systems into the mix, the situation can become infinitely more complex with each new application that is added.
Data integration strategy
This is why having a clear strategy for your integration projects in place from the outset should help you to determine which systems are going to be integrated, and in turn that means you can be proactive in implementing and planning out the overall data sharing model.
If you instead choose to take a reactive approach to implementing new systems, you run the risk of having to rebuild and redesign your integrations from the ground up each time, which is clearly a very costly and time-consuming approach.
In our view the most efficient long-term approach is to have a centralized style approach. It has the least amount of interface complexity and will deliver the lowest TCO and the most efficient processes.
The next steps
Once you have answers to the above questions and are clear on the decisions, you can further talk through these options and seek advice from your data integration partner who will be able to guide you on the next steps and flag any concerns. Your data partner can advise you throughout all the stages of the design and implementation of the integration project.
They will then be able to work with you on the necessary steps and determine the precise mechanisms for undertaking the data integration without disruption to the business, based on your requirements.
Article updated February 2023
