Data migration projects highlight serious data quality issues that too often lead to delayed projects and cost overruns. In extreme cases, poor data quality has led to the abandonment of new business initiatives. Environments where high volumes of data are entered into key business systems are a breeding ground for data quality problems. Lack of user knowledge, absence of stable and robust processes, and missing relationship linkages can lead to poor quality data in any system. The most common issues are:
1. Incomplete Data
Data can be missing partially or completely. For instance if some field values are null and during migration process these were not handled in elegant manner, could cause lot of problems in proper functioning of the migrated system.
2. Duplicate Data
Multiple instances of the same data is a big problem during data migration. It’s unlikely that conversion will ignore duplicate data records. Since the data format is different in each of the duplicate records, though the information is the same, it is difficult to narrow down and ignore duplicate data records.
3. Data Non-conformity
This is because the way the data in the database is formatted differs greatly from individual to individuals who have created the database structure. Hence data is not present in the standard format.
4. Inconsistent and Inaccurate Data
When merging various systems, the data can lack consistency and represent wrong information. Data deteriorates over time, which can cause a lot of difficulties during migration.
6. Data Integrity
Missing relationship linkages can drastically degrade the quality of data and pose problems during migration.
Prior to developing a conversion utility for data migration, it is worthwhile to research the type of data presented in the system. This is similar to the requirements gathering phase of any project. Listing all the types of data that need to be converted reduces the risk of errors.
There are variety of tools available to use in the data migration process by both advanced and beginner developers. These tools are contingent upon the compatibility of the data environment and platform dependent.
Informatica Power Center – A functional programming tool for platform accessing, reporting and integrating data.
Ascential Data Stage – Provides OnDemnd data integration.
ESS Base Hyperion – Creates analytic and enterprise performance management applications.
Business Objects: Data Integrator – Builds data marts, ODS systems and data warehouses.
SAS Data Integration – Generates data streams, data marts and data warehouses.
Microsoft Data Transformation Services (DTS) – Aids data movement issues, and automates ETL processes through databases.
Oracle Warehouse Builder – Provides fully integrated relational and dimensional modeling and the management of data and metadata, among other functions.
For more tips on data migration read eCommerce Best Practices book
Walmart is stepping up its integration of its online and in-store properties by creating a proprietary search engine called Polaris. The search engine was developed by the retail giant’s in-house technology shop @WalmartLabs, which the company created following the April 2011 acquisition of Kosmix.
Polaris, announced on August 30 but launched three months ago on Walmart.com and one month ago on Walmart’s mobile platform, underscores the importance of search, particularly for e-commerce.
“The search engine is the crown jewel of any e-commerce platform,” says Sri Subramaniam, VP of @WalmartLabs. “And, typically, if you are a business you want to own [the] crown jewel and you want to actually get under the hood.” He adds that Walmart is “very, very, very serious about e-commerce” and views e-commerce and in-store shopping as part of an integrated experience. Read more
Technology giant Oracle is not letting off its acquisition spree. After a gap, followed by a series of acquisitions, Oracle Inc, on Tuesday, announced that it would buy Endeca Technologies for an undisclosed amount. Endeca is known for its robust enterprise search solution that could be integrated with ATG commerce, the commerce suite of ATG that Oracle acquired some time ago.
In a press release, Oracle reveals that “the combination of Oracle ATG Commerce with Endeca’s inFront web commerce solutions is expected to enhance cross-channel commerce, merchandising, and online customer experiences.” Read more
An appropriate combination of both Web services technology and the SOA approach addresses many of the issues of building an SOA based application integration environment. That is not to say that Web services and SOA are intrinsically linked, because they can be implemented separately. Read more
One of the biggest and most important decisions you need to make is what eCommerce system will meet your online sales needs. An eCommerce system or platform is basically what makes the website function correctly – in terms of searchandising, content management, sales, payment processing etc. It is the engine, so to speak, of your ecommerce site and you need to make sure that the engine is compatible. Read more