What is data integration middleware?

Data integration middleware provides a layer of services that allow applications and development tools to view different underlying data structures and data sources as a single data source. This means that an application or a developer is not affected by physical implementation of data structure. In other words, a table can be in an Oracle, Sybase, or Informix database, but to the application or developer, it simply looks like a database table. In fact, data integration middleware can make multiple tables look as if they are a single table. Like middleware in general, data integration middleware abstracts the details of underlying constructs, in this case data constructs, to make it easier for the layer of software above it to use the constructs. Another type of data integration tools is known as a "SQL gateway." Much like a gateway used in networking to tie together disparate networks, an SQL gateway hides underlying physical differences to data sources from the data requestor, and manages database connections. Thus, the SQL Gateway:

SQL gateways are particularly useful when dealing with B2C^[1] applications that have a high number of temporarily connected users, and possess heterogeneous database sources. Many application servers now possess a similar form of SQL gateway functionality, such as shared connection pooling, and abstraction from underlying physical database implementations. SQL gateways were important in early CGI-based Web applications, but are slowly being replaced by application servers.

Data integration middleware is most often used to tie together two tier applications (user tier to database, or middle tier to database), and do not offer transactional capabilities. They do, however, simplify heterogeneous data environments.
The performance and reliability of data integration middleware has matured, and is now considered a viable alternative for virtually any solution versus native calls. Native calls are requests to a database engine that make use of the particular semantics of the database engine.

Most data integration middleware products now get OEMed into development tools (such as Visual Café and Inprise/Borland's JBuilder), and EAI suites (such as Vitria's and Tibco's). A few years ago, data integration middleware was a distinct market. Due to OEMing, it has virtually disappeared, though the technology lives on in other tools.
Please see the Resources section of the course to download a PDF file containing a complete list of all of the vendors and tools covered in this course, along with URLs to the vendor Websites.

The flexibility and ease of maintenance afforded by data integration middleware now usually outstrips the slight performance improvement available through native calls. Most e-Commerce solutions should use some kind of ODBC, JDBC, and/or other data integration layering technology, rather than programming for a specific physical database.

1. Open Data Base Connectivity (ODBC): Inserts a middle layer, better known as a database driver, making it possible to access any data from any application, regardless of which database management system handles the data. For this to occur, the application must be capable of issuing ODBC commands and the database management system must be capable of responding to them.
2. Java Database Connectivity (JDBC): Enables Java programs to interact with any SQL-compliant database and execute SQL statements. It is just like ODBC but is designed to work with Java.

[1]Business to Consumer (B2C): Commerce is conducted between a consumer, such as a home user on a PC and a business.