Page History

...

Data integration is the amalgamation of a systematic series of operations used to combine data from disparate applications into valuable information.

Data integration occurs when a variety of data sources are blended into a single database, offering users of that database efficient access to the information they need. DI aims at providing an integrated and consistent view of data coming from internal and external data sources.

There are several ways to integrate data that depend on the size of the business, the need is fulfilled and the resources available.
Manual data integration A process by which an individual user manually collects necessary data from various sources by accessing interfaces directly then cleans it up as needed and combines it into one warehouse. This is highly inefficient and inconsistent and makes little sense for all but the smallest of organizations with minimal data resources.

Middleware data integration is an integrated approach where a middleware application acts as a mediator, helping to normalize data and bring it into the master data pool. Middleware comes into play when a data integration system is unable to access data from one of these applications on its own.

Application-based integration is an approach to integration wherein software applications locate, retrieve, and integrate data. During integration, the software must make data from different systems compatible with one another so they can be transmitted from one source to another.

Uniform access integration is a type of data integration that focuses on creating a front end that makes data appear consistent when accessed from different sources. The data, however, is left within the original source. Using this method, object-oriented database management systems can be used to create the appearance of uniformity between unlike databases.

Common storage integration is the most frequently used approach to storage within data integration. A copy of data from the original source is kept in the integrated system and processed for a unified view. This is opposed to uniform access, which leaves data in the source. The common storage approach is the underlying principle behind the traditional data warehousing solution.

According to Gartner, iPaaS Integration Platform as a Service (iPaaS) is a suite of cloud services . It enables enabling development, execution , and governance of integration flow by flows connecting any combination of On-Premise, or on premises and cloud-based processes, services, applications , and data , within an individual organization or , across multiple organizations.

...

Functional Components of iPaaS

Integration platform typically contains a set of functional components, such as: 

• Message bus for enabling reliable messaging between enterprise applications.
• Adapters to transform messages from, and to, application's proprietary protocol. Adapters often offer connectivity via common standards, like FTP, SFTP or format support, like EDI.
• Transformation engine and visualized data mapping to transform messages or files from one format to another.
• Metadata repository for storing information separated from processes - like a business party.
• Process Orchestration Engine for orchestration design and execution. In this, context orchestration is a technical workflow that represents a business process or part of it.
• Technical dashboard for tracking messages in a message bus and viewing execution history of orchestrations.
• Scheduler for scheduling orchestrations.
• Batch engine for controlling large file transfers, batch jobs, execution of external scripts and, other non-messaging based tasks.

• Communication protocol connectors (FTP, HTTP, AMQP, MQTT, Kafka, AS1/2/3/4, etc.).
• Application connectors/adapters for SaaS and on-premises packaged applications.
• Data formats (XML, JSON, ASN.1, etc.).
• Data standards (EDIFACT, HL7, SWIFT, etc.).
• Data mapping and transformation.
• Data quality.
• Routing and orchestration.
• Integration flow development and lifecycle management tools.
• Integration flow operational monitoring and management.
• Full lifecycle API management.

s