AWS SQS

4min
Write events directly to an Amberflo supplied and secured AWS SQS queue
Amberflo provides you with an AWS SQS queue (FIFO) with access rights and controls to write meters.
It automatically picks up the meters for processing as they arrive into the queue.
Please contact us to get the SQS queue and S3 bucket provisioned for your account.
Format
The meter records you send to the SQS queue should be of the same standardized format as accepted by the ingest API﻿. Here are some examples:
JSON
[{
     "customerId": "customer-123",
     "meterApiName": "ComputeHours",
     "meterValue": 5,
     "meterTimeInMillis": 1619445706909,
     "dimensions": {
        "region": "us-west-2",
        "az": "az1"
     }
}]
[{
     "customerId": "customer-123",
     "meterApiName": "ComputeHours",
     "meterValue": 5,
     "meterTimeInMillis": 1619445706909,
     "dimensions": {
        "region": "us-west-2",
        "az": "az1"
     }
}]
﻿
We also support NDJSON format (JSON separated by a newline)
JSON
{ "customerId": "customer-123", "meterApiName": "ComputeHours", "meterValue": 5, "meterTimeInMillis": 1619445706909 }
{ "customerId": "customer-321", "meterApiName": "ComputeHours", "meterValue": 4, "meterTimeInMillis": 1619445712341 }
{ "customerId": "customer-123", "meterApiName": "ComputeHours", "meterValue": 1, "meterTimeInMillis": 1619445783456 }
{ "customerId": "customer-123", "meterApiName": "ComputeHours", "meterValue": 5, "meterTimeInMillis": 1619445706909 }
{ "customerId": "customer-321", "meterApiName": "ComputeHours", "meterValue": 4, "meterTimeInMillis": 1619445712341 }
{ "customerId": "customer-123", "meterApiName": "ComputeHours", "meterValue": 1, "meterTimeInMillis": 1619445783456 }
﻿
Code Example
Python
import json
from uuid import uuid1
from datetime import date

import boto3

records_to_send = [{
     'customerId': 'customer-123',
     'meterApiName': 'ComputeHours',
     'meterValue': 5,
     'meterTimeInMillis': 1619445706909,
     'dimensions': {
        'region': 'us-west-2',
        'az': 'az1'
     }
}]

queue_url = 'https://sqs.us-west-2.amazonaws.com/624335419252/62-ingest.fifo'

sqs = boto3.client('sqs')

sqs.send_message(
    QueueUrl=queue_url,
    MessageBody=json.dumps(records_to_send),
    MessageGroupId=str(uuid1()),
    MessageDeduplicationId=str(uuid1()),
)
import json
from uuid import uuid1
from datetime import date

import boto3

records_to_send = [{
     'customerId': 'customer-123',
     'meterApiName': 'ComputeHours',
     'meterValue': 5,
     'meterTimeInMillis': 1619445706909,
     'dimensions': {
        'region': 'us-west-2',
        'az': 'az1'
     }
}]

queue_url = 'https://sqs.us-west-2.amazonaws.com/624335419252/62-ingest.fifo'

sqs = boto3.client('sqs')

sqs.send_message(
    QueueUrl=queue_url,
    MessageBody=json.dumps(records_to_send),
    MessageGroupId=str(uuid1()),
    MessageDeduplicationId=str(uuid1()),
)
﻿
Troubleshooting
The records sent to the queue will be saved in an S3 bucket before being processed (see the AWS S3 integration﻿).
So troubleshooting just follows the same pattern, i.e. a file will be created in S3 with the failure reason.
We also set up a dead-letter queue in case there are issues writing to the S3 bucket.
Updated 05 Nov 2024
Did this page help you?
AWS CloudWatch
Elastic Logstash