How to Manage Data Quality in an ERP Migration

In a recent ERP migration project, our team was knee-deep in helping a customer retain their legacy records that had been accumulated over ten years, spread across multiple systems and sources, containing thousands of duplicates without any unique identifier to tie them together. It was a challenge that delayed their migration process significantly as the team had not anticipated the extent of duplication and quality issues.

Though they had come to us for data cleaning, they soon realised they had a bigger challenge with unresolved entities. The conversation went from establishing cleaning rules to making critical decisions on which records to merge, purge, which values to retain, and how to handle new data against the current cleanup.

This isn’t the first time we’ve encountered a data quality migration challenge like this where the project has far more issues that originally anticipated. And that’s why we believe data is the biggest bottleneck you should plan around in an ERP migration. Unresolved data decisions can hold up several workstreams together, even if your data ‘looks fine,’ on the surface.

So if you’re planning an ERP migration anytime soon, this guide helps you work through the decisions that can hold up your migration, starting with what are some of the common data migration problems to watch out for and eventually how to safely resolve these issues without putting your sources at risk.

Investigating the Data for Deeper Quality Issues Before Migration

When you are managing an ERP migration alongside everyday responsibilities, finding time to investigate and optimise the data can be difficult. In our experience, that early review helps you understand the work ahead and prepares you for any rude surprises!

One of our customers in the energy sector discovered around 115,000 accounts were unfit for migration when they began profiling it – this was right in the middle of a very tight migration schedule, that resulted in process delays as well as financial loss.

So we recommend starting with data profiling to understand the condition of your migration data and do a deeper dive than just basic formatting or structure analysis. You might want to understand how much of your data is duplicated, how many columns have missing values, or are inconsistent. A deeper look at the quality of the data can help you build a stronger preparation plan that addresses key concerns like:

• Which issues are impacting data health the most (is it missing values, incomplete data, inconsistent formatting?)
• How many % of records need fixing or cleaning before it can be matched for consolidation
• What kind of data structures should be preserved during a cleaning process? (such as abbreviations, naming conventions etc)

At WinPure, we focus that review on areas that affect the cleaning and matching work ahead:

• Completeness of useful identifiers. Check how consistently fields such as supplier IDs, registration numbers and product codes are populated. Gaps can affect which attributes you can reliably use for matching.
• Consistency of formats and values. Examine field types, lengths and value distributions to identify mixed formats, unexpected codes and variations that need standardisation.
• Repeated values in fields expected to be unique. Investigate whether these indicate duplicate records, shared identifiers or a source convention that your matching rules need to accommodate.
• Patterns hidden by an overall quality score. Review findings by source and relevant record group. Missing identifiers concentrated in one legacy extract may require a different treatment from the rest of your data.

We use these findings to shape the preparation approach: which corrections you can apply through agreed rules, where matching needs further testing and which questions require business input. That gives your team a clearer basis for estimating the work before migration testing begins.

What are the common data migration problems in ERP?

Once you begin working through the data, you might run into a few common problems become more apparent when data from multiple sources are assessed together.

Some of the most common challenges we’ve encountered during a data migration are:

• Non-exact or fuzzy duplicates with confusing entries. Most deduplication work on closely-related or exact matches, but considering how data is structured these days, you may find one entity with not just 5 different emails, but also under abbreviated names, addresses or numbers. In terms of company data, you may also discover shared details can be misleading: where separate businesses may use the same address, even though they may not be related to each other. These duplicates can only be caught when you have a powerful match algorithm that goes beyond matching names or numbers and looks at your data holistically.

• Losing your organisation’s data conventions during cleaning. Companies have their own data conventions that they’d want to preserve during a data clean session. For example, when we clean supplier data, we need to determine if product codes should include leading zeros or prefixes that distinguish product ranges, and whether there are any internal conventions that must be preserved when matching supplier records across systems, so those differences do not cause us to overlook a genuine match or combine records that should remain separate. With WinPure’s editable Data Type Library, we have made it easier now for customers to build their custom data rules and adapt existing definitions to make the cleaning and matching effective.

• Unclear relationships between entities. As you work through potential matches, you may discover that records you initially considered duplicates actually belong to separate branches or subsidiaries – while apparently unrelated records represent the same supplier. Different names, addresses and source IDs can obscure those relationships, with orders and invoices attached to each version. At that point, you need to establish which records represent the same entity, which entities are related and which must remain separate before deciding what to consolidate.

• An unclear merge and purge process. Once you identify duplicates, you still need to decide which records to merge, which values to retain and which records can safely be removed. Those decisions become difficult when different departments rely on different versions of the same customer or supplier. Without agreed rules, your team can get stuck reviewing the same groups or discard details and source references that are still needed.

• No persistent identity for the master record. After consolidating records, you need a reliable way to associate subsequent extracts with the master you have already established. If identifiers change between systems or loads, the same supplier can reappear as a new record, creating duplicates and fragmenting its history again. A persistent ID helps maintain that connection, supporting the master record as your single source of truth as data changes throughout the migration.

These problems can turn a seemingly straightforward ERP cleanup for migration into a series of decisions that hold up preparation. This is why our team approaches every ERP migration project as a connected transformation process, carrying what we learn during cleaning through to matching and master record creation.

Transforming legacy data into records your new ERP can use

Addressing these problems requires deep investigative work, and while it may seem time consuming, it makes the subsequent work of building master records ready for migration more straightforward. Working with a data quality provider experienced in managing that transformation helps keep the cleaning, matching and consolidation connected as new findings emerge.

At WinPure, we support this work through our specialists and Clean&Match Enterprise, bringing the treatment of individual fields through to resolved entities and consolidated master records. Here is how that transformation takes place.

Here’s how you can transform legacy ERP data for a new platform:

🔷Clean the values while preserving their meaning

We start with the issues your profiling has exposed, applying corrections around your data conventions and the requirements of the target ERP. That might involve separating combined address fields, standardising telephone formats or replacing inconsistent abbreviations, while preserving meaningful prefixes in supplier and product codes.

Our data cleansing solution combines CleanMatrix™, where you control operations for each column, with CleanAI™, an intelligent built-in logic that auto maps your data types and the data quality issues it has in the matrix making it easy for you to clean data without manual effort.  You can adjust the proposed treatment and inspect the preview before applying it. Word, Pattern and Regex Managers also let you retain organisation specific formats, replacements and exceptions for subsequent extracts.

We also consider how the cleaned structure will support matching. For example, when our team works with customers on a project, we might decide on separating a building number from a street name, allowing us to accommodate spelling variations in the street while keeping the number exact. That gives us more precise comparison criteria than applying one similarity threshold to the entire address.

🔷Match records according to the information they contain

Once those inconsistencies have been addressed, records that previously looked different may become easier to connect. However, a supplier name with reordered words needs a different comparison from a registration number that should agree exactly.

With AdaptiveMatch™, we can combine comparison methods across individual fields. Its Data Type Library supports recommendations for algorithms and matching settings, including custom data types, while you retain control over thresholds, weights and conditions. Users can also test those settings against the resulting groups, investigating both unexpected matches and records that remain separate.

🔷Use AI assistance while retaining technical accountability

Your team can configure the treatment directly, work from AI recommendations or combine both approaches. This flexibility matters when a dataset contains exceptions that require more specific logic than the initial recommendations provide.

AI Connect™ adds support for exploring configuration questions through a compatible AI assistant. We treat those suggestions as inputs to investigation, checking their effect against the settings and record evidence. Your team retains responsibility for accepting changes, with match explanations and audit records supporting review of the resulting decisions.

🔷Resolve the entity behind the matched records

As we examine matched groups, the distinction between a duplicate and a legitimate business relationship becomes important. The same organisation may appear as both a customer and a supplier, with separate accounts serving different financial roles, therefore, recognising that shared identity is necessary to preserve the context of the data, which has a direct impact on downstream applications.

Our entity resolution solution supports investigation across the available attributes and sources, with explanations showing the evidence behind the groupings. Through MatchAI™, your team can examine possible duplicates and relationships before deciding which records belong to the same entity, allowing you to build a more complete understanding of the entity while retaining the accounts and relationships your ERP still needs.

🔷Consolidate the information you need to retain

When an identity is established, you can address the conflicting information within its records. The most complete record may still contain an outdated address, while a less populated source holds the current contact details. Master selection therefore needs to account for the information available across the group.

Within our golden record management solution, you can select the master and merge stronger values from other records in the group. SmartMaster AI™ supports master selection using completeness, consistency and reliability, alongside your configured rules and review. Merge and purge decisions can then follow the confirmed identity and retention logic, with the resulting master prepared for export.

🔷Keep that identity consistent across subsequent loads

The consolidated record also needs continuity when another extract arrives. Golden Record Identity™ assigns a persistent identity to the resolved entity, helping associate incoming records with the master already established. Your team can retain source references alongside that identity to support the mapping into the ERP.

We use this distinction throughout preparation: the master’s attributes may change as better information arrives, while its persistent identity maintains the connection across processing runs. That supports a consistent reference through migration testing and subsequent data updates.

🔷Maintaining a consistent clean routine post-migration

By the time your migration data is ready, you have established more than a clean dataset. You have defined how your organisation’s values should be standardised, which attributes identify a match and how information should survive consolidation. Those decisions remain valuable as new records enter the ERP.

Without continued treatment, incoming data can gradually reintroduce the variations and duplicates you have already resolved. We therefore build repeatability into the transformation, so your cleaning and matching configurations can continue supporting the data beyond the migration.

With WinPure’s automation, you can save tested configurations and schedule recurring cleaning and matching runs against data supplied from your new environment. The same conventions and comparison rules can be applied to subsequent batches, with editable settings and execution logs providing visibility into the work performed.

For incoming records, that means consistent formatting and comparison against established entities, helping your team maintain the master dataset rather than repeatedly rebuild it. New exceptions still need investigation, but familiar issues can follow the treatment you have already agreed. Scheduled cleaning, matching and master creation support that continuity as the dataset grows.

Practical tip: Retain your tested cleaning libraries and matching configurations after migration, then review their suitability when incoming sources or field structures change.

Your ERP migration data preparation checklist

As you work through your migration data, use this checklist to keep track of the findings and decisions that need to carry through to the final dataset.

✅ Data health investigated. You understand the extent of missing values, inconsistent formats, repeated identifiers and other issues affecting preparation.

✅ Business conventions preserved. Cleaning rules retain meaningful codes, prefixes, annotations and organisation specific formats, with reusable libraries saved for subsequent extracts.

✅ Matching criteria tested. Your comparisons account for spelling variations, incomplete attributes and differences between sources, with results reviewed for missed matches and incorrect groupings.

✅ Entities and relationships resolved. Confirmed duplicates are distinguished from related organisations, branches and other entities that need to remain separate.

✅ Merge and purge decisions established. You have defined which values survive consolidation, which records can be removed and which exceptions still require investigation.

✅ Persistent identities retained. Consolidated entities have stable identifiers, with source references preserved to support traceability and subsequent matching.

✅ Transformation settings documented. Your team can inspect and explain the cleaning rules, matching configuration and master selection logic, including any AI assisted recommendations it has accepted.

✅ Recurring cleaning and matching prepared. Tested configurations are saved for reuse, with scheduled runs and exception reviews suited to the data entering your new environment.

✅ Give your migration data the attention it needs early

We’ve repeatedly seen how teams start migration projects believing they only need to clean and dedupe the data, but the work soon extends into questions about identity, consolidation and what needed to survive from ten years of accumulated records. These questions take time because the answers depend on understanding the data – which depends on how deep you can go into assessing the data’s structure, its challenges. Most importantly though, the ability to transform the data and keep it clean requires a platform and a team that can help you achieve all of this without demanding additional time, complexity or infrastructure setup.

Why work with WinPure for ERP data migration?

Preparing migration data takes time, technical judgement and a close understanding of the records involved. We bring our specialists and Clean&Match Enterprise together to support that work, with the flexibility to fit your data, infrastructure and requirements. Some key features that can help you with the process includes:

🔷Keep sensitive data within your environment. Deploy locally on cloud, on server, or on desktop, with localised AI processing for cleaning, matching and master creation.

🔷Bring the transformation together. Profile, clean, match, resolve entities and create master records within one connected platform.

🔷Work with experienced specialists. Get support investigating your data, refining configurations and working through complex consolidation challenges.

🔷Make preparation easier to repeat. Reuse cleaning libraries, matching settings and automated schedules across migration extracts and ongoing maintenance.

🔷Retain technical control. Combine AI assistance with configurable rules, inspect proposed changes and review the evidence behind matches before approving the output.

If you are preparing for an ERP migration, speak to us about a free trial of WinPure Clean&Match Enterprise. Bring the data challenges you are working through, and we can discuss how our solution supports your cleaning, matching and master record preparation while keeping your team in control.

Written by

Farah Kim

Farah Kim is a human centric product marketer who specialises in making complex data management topics accessible to business and technical audiences. With a background in Computer Science, Linguistics, and Media Communications, she bridges the gap between technology and business by translating data quality, entity resolution, data matching, and governance challenges into practical, actionable insights. At WinPure, she works closely with product and customer teams to educate organisations on building trusted, high quality data for analytics, AI, compliance, and operational success.

View all posts›

Reviewed by

David Leivesley

David Leivesley is CEO of WinPure and has over 20 years of experience in enterprise data quality. He specialises in data matching, data cleansing, entity resolution, data migration, and master data management. His work helps organisations improve data accuracy, eliminate duplicates, and build trusted data for AI, analytics, and business-critical decision-making.

View all posts›