Has there been a marriage between EAI and ETL tools?
Two years ago, when we published the second edition of this survey, we saw that not many ETL tools had good, reliable functionality for real-time application integration (EAI) projects. Since then, many ETL tools have added tools for real-time extraction, transformation and integration, and there are signs of a convergence between ETL and EAI tools into a market which is being called Enterprise Information Management (EIM).
Best of both worlds
A marriage? At this stage it is difficult to tell, it is clear that the suppliers think that this is where the future lies, but when asked most will admit that they have very few customers yet who are using what amounts to real-time ETL. In the table below it is clear that real-time systems and traditional ETL come from very different backgrounds, but there are examples where combining the two produces the best of both worlds.
Unlikely to merge completely
It is clear that many of the suppliers have now integrated these tools, but ETL and EAI tools are unlikely to merge completely, each market will continue to exist on its own, and have its own user population mainly because that is what the majority of the customers still want. What we have seen is that these tools increasingly make use of a common metadata layer, a repository of transformations and business logic and standard adaptors to connect to a wide variety of systems.
Data is entered only once in the enterprise
Finally, the nature of ETL for Business Intelligence purposes, differs in many aspects from the nature of EAI. ETL is about moving, transforming and cleaning large amounts of data not more than a few times a day from many sources into one place - convergence. Whilst with EAI we often move realtively small amounts of data, transactions, spreading them across various systems - divergence. We use ETL for data migration, for one view of the truth or data integration to allow better decision making. In general we use EAI for process optimization and workflows and to make sure that data is entered only once in the enterprise, although some companies are moving towards real-time Business Intelligence and using EAI tools to keep the data warehouse up to date.
ETL and EAI have a strong relationship
Marriage? Maybe. From the vendors point of view, IBM, Microsoft, Oracle and other vendors in the EAI and ETL market, are producing a unified platform, which makes it look as if the products will at least be living together. Recently we have seen acquisitions where products from companies like BO, Cognos and Datamirror have been bought by platform vendors, with the consolidation in this marketplace many of the suppliers are no longer dependent on one product for their revenue, so as long as the customer keeps using their revenue generating product, which may be a database or a particular brand of front end software (like MS Office) then the costs of the ETL and the broker software will continue to be reduced.
In our opinion ETL and EAI have a strong relationship, should use the same business rules and definitions, but have a different architecture – perhaps it looks like a marriage after all!
|
Strengths of each: ETL versus EAI (source: Computerworld) EAI: Enterprise application integration systems |
> List of ETL tools studied
> Aspects and criteria explained
> Ranking of the ETL tools by aspect
> Download the ETL Survey Report
Back to the homepage of etltool.com
