The changes made in the productive DL/I databases in daily production operation by online users and through batch operation are captured by tcVISION in real time and written to a special Db2 table - or, if Db2 should not be active, to a VSAM file. These changes are processed at regular intervals and applied to the Db2 databases.

Once the migration is complete, changes in Db2 are also replicated from tcVISION to DL/i in order to comply with legal archiving regulations.

The daily change volume is more than 200,000 changes, at the peak more than 500,000..

The synchronization scenario is based on changed data capturing on the mainframe in real-time. The Adabas extension of tcVISION captures the changes and these changes are then propagated to a LINUX system using a mainframe collector and data pool storage. The LINUX system hosts an Oracle database that acts as a mirror of the Adabas databases. tcVISION directly propagates the mainframe changes into the Oracle database.

Both platforms act as equals (master/master). The tcVISION DBMS extension for Adabas captures changes on the mainframe in real-time and replicates those changes into an Oracle mirror database, from which the changes are applied to the Oracle production database.

The replication path from Oracle to Adabas on the mainframe is maintained through Oracle and the JDBC component of tcACCESS.

A tcACCESS Stored Procedure developed in PL/I obtains the Adabas Internal Sequence Number (ISN) for new inserted records for the Oracle environment.

The bidirectional replication guarantees that changes already applied on one of the platforms are not replicated back via the capturing mechanisms (tcVISION Loopback processing).

Log files are created to document the correct replication scenario.

The production implementation of tcVISION supports more than 370 Db2 tables. The operational changes to mirrored Db2 tables are replicated on the Windows replication server. The replication processes are part of the daily IT operations. The online systems and batch applications are responsible for the changes applied to Db2. All tables that are supposed to be replicated have the attribute ‘DATA CAPTURE CHANGES’ and all the changes are kept in the Db2 log. The log files are sent to the replication servers using FTP processes as soon as the online logs are archived by Db2. Once received, tcVISION processes the log files on the replication server. To process the log files and capture the changes, tcVISION has to understand the structures of the Db2 tables. This metadata is stored in a central repository which is also stored in the Db2. tcVISION provides import functions for creating the metadata from the Db2 source database. This metadata is used for the initial load of the Db2 tables to the server as well as for the processing of the Db2 log files.

The initial load of the Db2 tables on the replication server is performed by the BULK TRANSFER function of tcVISION. The input is a Db2 imagecopy that is sent to the replication server via FTP. The copy is processed and loaded to Db2 with the help of a LOAD statement generated by tcVISION. Then, the tables are available on the replication server and changes can be applied via the log file processing.

A special challenge was presented by the Db2 Compression Dictionaries and the fact that the structures of the tables can be changed during the daily operations. The information about the compression algorithm in use is scanned by tcVISION while importing the metadata and stored in the central repository. Each table is stored in a table space of its own. The different dictionary versions and corresponding compression algorithm are kept in the repository, so tcVISION can always use the correct algorithms for the decompression when processing the log files or imagecopies. The metadata reflects the structure of the data at the time of the import. tcVISION historicizes this structural information during a new import after a structure change. This way, tcVISION recognises changes of the data structure and is able to apply the changes to the target tables in accordance with the structure in place at the time the change was performed. That is why all Db2 structural information is refreshed every day by an automated process. This automated process guarantees the correct metadata and compression information to be used. tcVISION provides tools and utilities to delete all the historical repository entries that are no longer required.

The daily processing of all the Db2 changes is nearly completely automated.

The distribution of the subscription data is performed by tcVISION. The subscribed data is captured in real-time from the active log of a shared Db2 system on a z/OS system. The capture mechanism of tcVISION uses the Instrumentation Facility Interface of Db2. The changed data is transferred to an Intel based Linux system via TCP/IP, where the distribution of the data records to the customers is performed based on data record attributes.

Technically this was realised using a bit string as an integer attribute for the relevant tables. A comparison is performed for every replicated record by using a logical bit compare with different comparison values to check whether the record should be distributed to the customer. If the bit comparison is not successful, the record is rejected.

In addition to the bit string logic, a comparison based on a Lookup SQL and character string comparisons are performed. To serve each individual customer based on his requirements, a subscription system has been developed that only transfers the stock exchange values to which the customer has subscribed.

All processing definitions are stored in a central tcVISION repository.