Streaming is a method used to transfer data over a network as a continuous flow. One of its main benefits is that it allows the data recipients to use it in real time, without having to wait until the complete data set has been uploaded on their computers.
Salesforce has a set of data APIs that can be used to manipulate the App’s data. Among them, the Streaming API is used to set notifications triggered by data changes. The notifications are sent from a server to a client.
The Salesforce Streaming API follows the Publish – Subscribe model. In it, users subscribe to available channels that broadcast specific data changes. These data changes are defined by SOQL queries. For example, a channel can notify its users about changes in Account’s Emails.
The main advantage of this model is that it eliminates pooling and therefore, it is useful for communicating very frequent changes - practically producing real time integration.
Additionally, the API’s use of long polling or Comet Programming technique, reduces the number of requests that return no-data.
Salesforce API has many important characteristics that define its behavior. Among them:
It is within the context of the Salesforce Streaming API that, DBSync developed a connector that can be used specifically for real time integration. The connector is known as Salesforce Streaming Connector.
There are two important points to consider when using the Salesforce Streaming Connector. First, the process should run as continuously as possible, without much interruption. Second, how to schedule the process. We will explain the use of this connector with an example.
In this example we will connect Salesforce table Accounts to a MySQL table Accounts. The steps are:
As the steps are similar to any DBSync process creation, we will concentrate on the steps with specific differences, namely steps 2, 4 and 5.
In order to create a Salesforce Streaming connector, simply press the button Create New Connector, and select the type Salesforce Streaming Connector.
As we mentioned earlier, we want to create a process that runs as continuously as possible. However, DBSync cannot detect if the process has stopped for any reason; and thus, it cannot restart it automatically.
In order to solve this problem, we need to set up the schedule at every one minute interval. The cron expression for this is “0/1 0/1 1/1 ?*”. Alternatively, we can simply select minutes and specify “1”.
As we are transferring data from Salesforce to MySQL, we need to create a trigger that contains a query. This query must include the ID of the Salesforce table - in our case Accounts. The query must not include joins or aggregate functions. Besides, the WHERE clause must not include lookup fields.
When we create a new rule, we must make sure to map the ID from Salesforce to the MySQL table’s ID.
The advantages of the Salesforce Streaming Connector can be easily seen by changing the value of, for example, the website of an account: the change is almost immediately reflected in our MySQL database.
DBSync’s Salesforce Streaming Connector allows us to link Salesforce to another application in almost real time.
However, there are few aspects to consider - namely, the setting up of the schedule with a repeating frequency of one minute; the limitations on the trigger’s query; and, the mapping of IDs.