Data integration

Our approach to data integration is based on proven extraction-transformation-loading (ETL), data replication and service oriented architecture (SOA) solutions. Our consultants are experienced in commercial ETL tools (Informatica, DataStage, Oracle Warehouse Builder and others) as well as custom developed ELT tools (based on SQL, Unix shell, C, C++, PL/SQL).

When implementing ETL solutions we always pay particular attention to scalability, data quality and ease of maintenance. We usually recommend implementing atomic data stores. Such an approach enables easy addition of new analysis as business needs evolve.

In our solutions, we typically apply an architecture consisting of the following layers:

• Source systems
• Data staging
• Atomic data
• Aggregates

The data staging area prevents a creation of data loading stovepipes with inconsistent architecture. While data is loaded to the atomic data area, the following processes take place:

• Cleansing
• Transformation
• Historization

Atomic data provides the company’s central information warehouse. Aggregates enable a quick response to typical business analyses.

Selection of the right data integration method depends on the nature of the data (volatility, volume, contents) and is driven by the business needs. ETL/ELT mechanisms provide the most mature and stable technologies. Data replication is often used to build operational data stores. SOA architecture is applied successfully when integrating highly volatile data.