Streaming API

What is the force.com streaming API?

• Stream API exposes a near real-time stream of data from the force.com platform.
• Using the API, notifications can be sent to Pages in salesforce application, Application servers outside salesforce or to external client.
• Streaming API use for Applications that need to poll against Salesforce data frequently near real-time
• Streaming API enables you to reduce the number of API calls and improve performance. For example- Applications like (Mulesoft) that have constant polling action against the Salesforce, consuming unnecessary API calls and processing time would benefit from Streaming API because it reduces the number of requests that return no data.

What is the basic technology behind the streaming API?

• It uses Push Technology or Publish/Subscribe Model

What are the underlying mechanisms for the streaming API

To enable the publish/subscribe model, an active connection must be maintained between Salesforce and each client to enable the publish/subscribe model. Steaming API relies on the method Long Polling. Streaming API uses the Bayeux protocol and CometD for long polling.

Below are the three steps to connect the client:

• CometD sends a handshake request.
• The client subscribe to the channel
• CometD maintains the connection by using long polling.

You can visualize the concept. Let's go to the workbench, when you go to Streaming push topic from the queries tab in workbench. 1^st you see the Handshake success message and then you need to select a topic push topic to subscribe and if you will make any change in CRM associate your push topic, you will see notifications in workbench.

What is PushTopic?

PushTopic is a record that

• Defines a channel
• Determine what event will cause the notification
• Describes the data the notification will contain.

An event is a modification of a field while creating, updating, deleting or undeleting the record whenever an event occurs it sent the notification. However, updates performed by the BULK API won’t generate the notifications, since such updates could flood a channel.

When multiple PushTopic notifications are generated for the same record within about one millisecond and in the same transaction, only the last notification is sent. 

A notification is data sent as a result of an event to the clients who subscribe to the associate channel. 

What is the channel name?

• Subscribers identify a channel using the name of assigned to the PushTopic.
• In the above example pushTopic.Name='pushTopicForAccount';. The channel name will be '/topic/pushTopicForAccount'
• The Channel name must match the PushTopic name exactly, including the casing of the letters.
• If a Push Topic name is changed, live subscribers are not affected however, new subscribes must use the latest Push topic name. 

When will an event generate the notification?

Notifications are generated for record events based on how you configure your PushTopic. 

• The Streaming API matching logic uses the NotifyForOperationCreate, NotifyForOperationUpdate, NotifyForOperationDelete, NotifyForOperationUndelete, and NotifyForFields fields in a PushTopic record to determine whether to generate a notification.
• Clients must connect using the cometd/29.0 (or later) Streaming API endpoint to receive delete and undelete event notifications.

How to define what data to include in notification?

Data in the notification is determined by:

• The select clause of the PushTopic
• The Access the user has to data

How does security work with PushTopics?

To receive a record notification, the logged-in user must have:

• API Enabled permission.
• Streaming API permission.
• Read access to the object specified in the query.
• Field-level security access for the fields in query
• Access to record by sharing rules
• Data returned by the query will be limited to the fields that the user has permission to access 

What are the unsupported queries in PushTopics?

The following SOQL statements are not supported in PushTopic queries:

• Queries without an Id in the selected fields list
• Semi-joins and anti-joins
• Example query: SELECT Id, Name FROM Account WHERE Id IN (SELECT AccountId FROM Contact WHERE Title = 'CEO')
• Error message: INVALID_FIELD, semi/anti join sub-selects are not supported
• Aggregate queries (queries that use AVG, MAX, MIN, and SUM
• Example query: SELECT Id, AVG(AnnualRevenue) FROM Account
• Error message: INVALID_FIELD, Aggregate queries are not supported
•  COUNT
• Example query: SELECT Id, Industry, Count(Name) FROM Account
• Error message: INVALID_FIELD, Aggregate queries are not supported
• LIMIT
• Example query: SELECT Id, Name FROM Contact LIMIT 1
• Error message: INVALID_FIELD, 'LIMIT' is not allowed
• Relationships aren’t supported, but you can reference an ID:
• Example query: SELECT Id, Contact.Account.Name FROM Contact
• Error message: INVALID_FIELD, relationships are not supported
• Searching for values in Text Area fields
• ORDER BY
• Example query: SELECT Id, Name FROM Account ORDER BY Name
• Error message: INVALID_FIELD, 'ORDER BY' clause is not allowed
• GROUP BY
• Example query: SELECT Id, AccountId FROM Contact GROUP BY AccountId
• Error message: INVALID_FIELD, 'Aggregate queries are not supported'
• Formula fields in WHERE clauses (formula fields are supported in SELECT clauses though.)
• NOT
• Example query: SELECT Id FROM Account WHERE NOT Name = 'Salesforce.com
• Error message: INVALID_FIELD, 'NOT' is not supported
• To make this a valid query, change it to SELECT Id FROM Account WHERE Name != 'Salesforce.com'

What is returned in streaming API Notifications?

The response containing the notification is encoded in JSON.

How can you deactivate a PUSHTOPIC?

To deactivate a PushTopic:

• Determine the ID of the PushTopic
• Create Apex Code to deactivate the PushTopic as below:

Where can PUSHTOPIC code be executed?

PushTopic can be executed in two ways:

1. Execute code using the Anonymous Window of the developer console
2. Load the PushTopic data using Data loader

You can also create apex class passing all required parameters for pushtopic and be reused in different places within the org.

How do we handle error in Streaming API? 

There are different types of error in streaming API integration:

401 Authentication Errors-Client authentication can sometimes become invalid, for example, when the OAuth token is revoked or a Salesforce admin revokes the Salesforce session. An admin can revoke an OAuth token or delete a Salesforce session to prevent a client from receiving events. Sometimes a client can inadvertently invalidate its authentication by logging out from a Salesforce session. Streaming API regularly validates the OAuth token or session ID while the client is connected. If client authentication is not valid, the client is notified with an error. A Bayeux message is sent on the /meta/connect channel with an error value of 401:: Authentication invalid and an advice field containing reconnect=none. After receiving the error notification in the channel listener, the client must reauthenticate and reconnect to receive new events.

1. 403 Unknown Client Error-If a long-lived connection is lost due to unexpected network disruption, the CometD server times out the client and deletes the client state. The CometD client attempts to reconnect but the connection is rejected with the 403:: Unknown client error because the client state doesn't exist anymore. The error response returned when the client attempts to reconnect after a timeout looks similar to the following message

2. 503 Server Too Busy Error-If the Salesforce servers don’t have available resources to process your Streaming API request, a 503 error is returned in the ext/sfdc/failureReason field. This error is returned for a handshake or a connection request. For example, this response shows the 503 error on the /meta/connect channel.

To know more about the security aspect of integration, pls Click here

To know more other integration API at a glance, pls Click here