DBSync Cloud Workflow (iPaaS)

2022

Page tree

Versions Compared

Old Version 1

changes.mady.by.user Anil Bhimineni

Saved on

compared with

New Version Current

changes.mady.by.user Tushar Singh

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Integration platform as a service Service (iPaaS) is a suite of multiple cloud services aimed at addressing a wide range of cloud, B2B, and on-premises integration and governance scenarios. An iPaaS consolidates multiple cloud services in a suite aimed at the . The suite aims at integration and governance of any combination of an on-premises and off-premises applications, SOA (various combinations of services - including On-Premise and On-Cloud applications, Service oriented architecture (SOA) and , cloud services, processes , and data, within or across organizations. An iPaaS complement the application-development- and hosting-oriented application platform as a service(PaaS). An iPaaS The iPaaS suite is an evolution of integration Integration as a service, which Service and has been widely adopted for B2B and cloud services integrations.integration. It complements the application-development and hosting-oriented application Platform as a Service (PaaS).

Fundamentals Of Integration

The integration approach can be either point-to-point or middleware or Middleware, iPaaS or EAI, . All of underlying concepts and principles around integrations are translated to into Extraction, transformation, and loading also known as ETL. Transformation and Load (ETL).

ETL refers to the methods involved in accessing and manipulating source data and loading it into target database. 

The first step in ETL process is mapping comprises of three general steps: 

  1. Mapping the data between source and target database.
The second step is cleansing of
  2. Cleansing source data in staging area.
The third step is transforming
  2. Loading cleansed source data
and then loading
  1. into the target system.
However
  1. It is worth noting that, the
second
  1. 2nd step may not be necessarily be the same for all the ETL tools
as their
  1. . This is because a company's internal system architecture
might vary internally
  1. may vary from that of other companies.



Basics

The DBSync's iPaaS can be mostly related

with the following

to ETL's concepts/terminology.

While reading in the next article you should be able to relate the core concepts of ETL with that of DBSync iPaaS.

 

Source System : 

A database, application, file or other storage facility from which the data in a warehouse is derived.

Mapping: 

The definition of the relationship and data flow between source and target objects.

Metadata:

Data that describes data and other structures , - such as objects, business rules , and processes. For example, the schema design of a data warehouse is typically stored in a repository as metadata, which . The metadata is then used to generate scripts used to build and populate the data warehouse. A repository contains metadata.

Staging Area:

A place where data is processed before entering the warehouse.

Cleansing:

The process of resolving inconsistencies and fixing the anomalies in source data , typically as - a typical part of the ETL process.

Transformation: 

The process of manipulating data . Any manipulation, beyond copying, is a transformation. Examples include cleansing, aggregating, and integrating data from multiple sources.

Transportation:

The process of moving copied, or transformed, data from a source to a data warehouse.

Target System:

A database, application, file, or other storage facility to which the "transformed source data " is loaded in a data warehouse.