Amazon SQS

The Airbyte SQS destination allows you to sync data to Amazon SQS. It currently supports sending all streams to a single Queue.

All streams will be output into a single SQS Queue.
Amazon SQS messages can only contain JSON, XML or text, and this connector supports writing messages in all three formats. See the Writing Text or XML messages section for more info.

Feature
Supported?(Yes/No)
Notes
Full Refresh Sync
No
Incremental - Append Sync
Yes
Incremental - Deduped History
No
Namespaces
No

  • AWS IAM Access Key
  • AWS IAM Secret Key
  • AWS SQS Queue

If the target SQS Queue is not public, you will need the following permissions on the Queue:
  • sqs:SendMessage

Required properties are 'Queue URL' and 'AWS Region' as noted in bold below.
  • Queue URL (STRING)
    • The full AWS endpoint URL of the queue e.g. https://sqs.eu-west-1.amazonaws.com/1234567890/example-queue-url
  • AWS Region (STRING)
    • The region code for the SQS Queue e.g. eu-west-1
  • Message Delay (INT)
    • Time in seconds that this message should be hidden from consumers.
    • See the AWS SQS documentation for more detail.
  • AWS IAM Access Key ID (STRING)
    • The Access Key for the IAM User with permissions on this Queue
    • Permission sqs:SendMessage is required
  • AWS IAM Secret Key (STRING)
    • The Secret Key for the IAM User with permissions on this Queue
  • Message Body Key (STRING)
    • Rather than sending the entire Record as the Message Body, use this property to reference a Key in the Record to use as the message body. The value of this property should be the Key name in the input Record. The key must be at the top level of the Record, nested Keys are not supported.
  • Message Group Id (STRING)

This property allows you to reference a Key within the input Record as using that properties Value as the SQS Message Body.
For example, with the input Record:
{
"parent_with_child": {
"child": "child_value"
},
"parent": "parent_value"
}
To send only the parent_with_child object, we can set Message Body Key to parent_with_child. Giving an output SQS Message of:
{
"child": "child_value"
}

To output Text or XML, the data must be contained within a String field in the input data, and then referenced by setting the Message Body Key property.
For example, with an input Record as:
{
"my_xml_field": "<something>value</something>"
}
To send a pure XML message, you would set the Message Body Key to my_xml_field.
The output SQS message would contain:
<something>value</something>

Version
Date
Pull Request
Subject
0.1.0
2021-10-27
#0000
Initial version
Copy link
On this page
Overview
Sync overview
Getting started
Requirements
Properties
Setup guide
CHANGELOG