IT Architecture Integration – SOA Model With a Dedicated Integration Layer

The model with a dedicated integration layer is an alternative way of SOA implementation. Read this post to find out:

• the strengths and capabilities of this solution
• the weaknesses and limitations that you’ll have to deal with
• when to use SOA with a dedicated integration layer

SOA with a dedicated integration layer in the services area has similar characteristics to that of an SOA solution with direct communication. What distinguishes this variant from other solutions is the use of a dedicated component that physically manages data flows as an “intermediary” between systems/data sources. The model focuses on the maximum centralization of information and making it available across the organization’s IT architecture. Its integration component collects information on how all systems in the organization are interconnected, and comprehensively supports the exchange of data between them. With centralization and reusability, it is possible to ensure global logging, as well as error and transaction handling, which is a huge advantage of this approach compared to the integration concepts discussed earlier.

The most popular – albeit not the only – class of the component that currently fulfils this role is the ESB (Enterprise Service Bus), which provides a wide range of possibilities in various aspects of integration. ESB is a tool to implement the SOA concept by enabling communication between systems in the organization, with efficient use of services.

Strengths and capabilities of the solution

• Flexibility of the integration layer – the ability to implement business logic, combine data from various sources and modify data structures without having to modify the services which feed data from source systems.
• Reusability – once manufactured and placed on the integration platform, tested and configured, the components responsible for exchanging data between systems can be used multiple times by different recipients, which significantly reduces costs.
• Elimination of dependency on system suppliers – some changes on the side of data transfer mechanisms (e.g. transformations) can be made in the integration layer, without having to modify the interfaces provided by source systems.
• Effective implementation of integration mechanisms – proven platforms are based on popular technologies characterized by a large number of auxiliary libraries, which greatly reduces the time needed to create viable solutions.
• Easier infrastructure maintenance and support from users of the interfaces – by using common, standardized login policies, and monitoring tools available “out of the box”.
• Possibility to use ready-made tools supplied with the integration platform – e.g. connectors and libraries that can handle communication by means of specific protocols and data sources, or that can enable connection with known systems.
• “Lighter” and less complex systems in the architecture – they do not have to contain any sophisticated code created for the purpose of integration processes – the center of gravity is shifted towards the core platform.
• Application of standards and promotion of good practices for organization-wide integration – each recipient of data from a different system must use the well-described services available on the platform.

Weaknesses and limitations

• The implementation of an integration platform in the IT infrastructure implies the need to learn, develop and maintain the platform. This in turn implies the need for appropriate competences within and outside the organization.
• If there is a lot of work to be done on the integration layer, the team responsible for that task may have to deal with a “bottleneck” in the organization, i.e. the time needed to deliver the components connecting the systems (even though both systems are ready, work needs to be done on the integration platform).
• Change handling on the platform and regression –- each modification made in the integration mechanisms should trigger the testing of all data flows, which may generate extra costs. To prevent this, great emphasis should be put on the use of CI (continuous integration) tools and ensuring that the mechanisms are covered with automatic tests.
• Platform = SPOF – the integration platform may become a single point of failure in the organization’s infrastructure. This means that, if a failure happens, integration processes will bust. To prevent this from happening, you should use solutions that ensure a high-availability of the integration layer. With this capability, if one instance of the platform fails, the second instance will be automatically activated. This will ensure continuity of inter-system data exchange.
• Given stringent performance requirements, adding an intermediary to convey a message from the source system to the target system can become a problem (time overhead). However, ESB class solutions are very well scalable, and in most cases the selection of appropriate hardware parameters is sufficient to ensure the desired performance.

Is this a solution for my organization?

A cost-benefit analysis for using SOA alongside the dedicated integration component shows that this is the right way to develop IT architecture. However, there are two scenarios in which this solution will not do the job:

This concept does not appear to be appropriate only in the following two scenarios:

• Scenario 1: The organization has a small number of systems that are supposed to communicate with each other using simple mechanisms, and over the next few years there will be no need to expand the inter-system connections (adding new systems, introducing more complex mechanisms);
• Scenario 2: The infrastructure needs an advanced integration platform divided into layers. Depending on the granularity and level of detail of data from the source system, and characteristics of the data recipient (system, application, user), the layers will be made available in a safe and appropriate manner.