Data migration projects highlight serious data quality issues that too often lead to delayed projects and cost overruns. In extreme cases, poor data quality has led to the abandonment of new business initiatives. Environments where high volumes of data are entered into key business systems are a breeding ground for data quality problems. Lack of user knowledge, absence of stable and robust processes, and missing relationship linkages can lead to poor quality data in any system. The most common issues are:
1. Incomplete Data
Data can be missing partially or completely. For instance if some field values are null and during migration process these were not handled in elegant manner, could cause lot of problems in proper functioning of the migrated system.
2. Duplicate Data
Multiple instances of the same data is a big problem during data migration. It’s unlikely that conversion will ignore duplicate data records. Since the data format is different in each of the duplicate records, though the information is the same, it is difficult to narrow down and ignore duplicate data records.
3. Data Non-conformity
This is because the way the data in the database is formatted differs greatly from individual to individuals who have created the database structure. Hence data is not present in the standard format.
4. Inconsistent and Inaccurate Data
When merging various systems, the data can lack consistency and represent wrong information. Data deteriorates over time, which can cause a lot of difficulties during migration.
6. Data Integrity
Missing relationship linkages can drastically degrade the quality of data and pose problems during migration.
Prior to developing a conversion utility for data migration, it is worthwhile to research the type of data presented in the system. This is similar to the requirements gathering phase of any project. Listing all the types of data that need to be converted reduces the risk of errors.