Replicate Salesforce data into Redshift Using Datapipeline:

Performance tuning for Loading data into Redshift

Anchor
OLE_LINK7 OLE_LINK7

Anchor
OLE_LINK8 OLE_LINK8

Launch DBSync Cloud Replication for

...

The goal of this white paper is to help you to understand how to increase the download performance into redshift by using an intermediate AWS RDS instance and push the whole load using AWS data pipeline.

We review in detail how to set up an AWS Data pipeline and steps to achieve clean download maintaining scalability and performance. After reading this whitepaper you will be able to make an educated decision and choose the solution that best fits your needs. 

Setting up AWS Datapipeline for RDS MySQL to RedShift

• We shall make use of an intermediate RDS instance to load data into Redshift. To do so we need to first connect our Salesforce and any Staging Databases like RDS MySQL on AWS.

• Please find the below link on setting up Dbsync Cloud replication to connect to Salesforce and any AWS RDS instance like MySQL , Sql Server, Oracle etc. 
  https://help.mydbsync.com/docs/pages/viewpage.action?pageId=53903406

• In your amazone console goto Analytics and select datapipeline. As per your requirement you can select the template from the "source" button in my case i need move data from my MySQL RDS to RedShift. Also based on your requirements you will be able to create deploy your custom templates for deploying datapipeline.

Image Removed

• As per your requirement you will be able to schedule through the console. I have scheduled for every week starting from 17-11-2015 also specified a convenient time so that we can make sure that there is no much load on the server.Image Removed

• Set your database parameter and additional setting on the "parameter" section also you will be able to change the data type conversion etc. You will be allowed to modify your requirement in templates. 

Image Removed

• Next step is "Pipeline Configuration" you can store you logs in s3 bucket also you have an option to disable to logs in it. Specify your IAM roles on the console.

• You can check and edit your configuration in "Edit in Architect" 

Image Removed

• In Architect you have option to modify update and change your template properties. You have "Add activity" tab to add any type of activity you need to attach with pipeline configuration. Add Data Node for adding more source and targets into your pipeline. "Export" option used for exporting your template code

• Next step is you need to save the pipeline and Activate it. 

Image Removed

• After activation you can monitor the logs while you click on the data-pipeline. 

Image Removed

Salesforce from AWS Marketplace with 1-Click

DBSync Cloud Replication can be launched into AWS with 1-Click, thus pay only for what you use, by the hour or by the month.

Login to https://aws.amazon.com with your login credentials. Search AWS Marketplace for Cloud Replication for Salesforce. Select Cloud Replication for Salesforce sold by DBSync.

Image Added

Review Pricing Details for the required region and click on Continue.

Image Added

1-Click Lauch tab will be highlighted. Select appropriate Version, Region, EC2 Instance Type, VPC Settings, Security Group, Key Pair. If you want to Launch DBSync Cloud Replication with EC2 console, click on Launch with EC2 Console and follow the instructions.

Image Added

You get to choose the required version of DBSync Cloud Replication when multiple versions are available.

Image Added

Select the required region.

Image Added

Review specifications and Pricing details of different types of available EC2 instances. Select the required EC2 Instance Type.

Image Added

You may choose to launch into EC2 or VPC by selcting the required VPC Settings.

Image Added

To select the Security Group, Select Create new based on seller settings. This will open only 8080 port. You may also select your own Security Group. 

Image Added

Select the appropriate Key Pair.

Image Added

Review Pricing Details. Pricing under Software is for DBSync Cloud Replication for Salesforce.

Image Added

Review Monthly Estimate. Software Charges are for DBSync Cloud Replication charges.

Image Added


Click on Accept Terms and Launch with 1-Click.
Image Added
Use the IP address of this newly launched instance to access DBSync Cloud Replication Console in your browser.

The URL should be https://<ipaddress>:8080/dbsync

Online wiki in the home page of DBSync Cloud Replication console will lead you to home page of DBSync wiki for Cloud Replication.

Image Added
Please refer the below DBSync wiki link to learn more about Navigation & using DBSync Cloud Replication.
http://help.mydbsync.com/docs/display/CloudReplication/Navigation