---
title: AWS S3
slug: 3jaLAaGSXNt9lD6y-PPxb
docTags: 
createdAt: 2024-07-11T15:09:39.357Z
---

# Write events directly to an Amberflo supplied and secured AWS S3 bucket

Amberflo provisions an **AWS S3 bucket** for your account with the necessary **access rights and permissions** to write meter data.

- **Meter files are automatically picked up** for processing as soon as they arrive in the bucket.
- If processing fails, **error reports are written back** into the same bucket for review.

There are several ways to write data to S3, including:

- **AWS S3 SDK**
- **AWS Glue**
- **Logstash**
- **Fluentd**
- And other ingestion tools

To get your S3 bucket provisioned, please contact us.

## Format

The **meter records** you send to the S3 bucket should be of the same standardized format as accepted by the ingestion API. Here are some examples:

```json
[{
     "customerId": "customer-123",
     "meterApiName": "ComputeHours",
     "meterValue": 5,
     "meterTimeInMillis": 1619445706909,
     "dimensions": {
        "region": "us-west-2",
        "az": "az1"
     }
}]
```

We also support [NDJSON](https://github.com/ndjson/ndjson-spec) format (JSON separated by a newline)

```json
{ "customerId": "customer-123", "meterApiName": "ComputeHours", "meterValue": 5, "meterTimeInMillis": 1619445706909 }
{ "customerId": "customer-321", "meterApiName": "ComputeHours", "meterValue": 4, "meterTimeInMillis": 1619445712341 }
{ "customerId": "customer-123", "meterApiName": "ComputeHours", "meterValue": 1, "meterTimeInMillis": 1619445783456 }
```

## Code Example

The S3 object key should have the date to allow easier troubleshooting (e.g.: `/ingest/amberdata/06-07-2022/0000a2e4-e6ad-11ec-8293-6a8da1c4f9f0`);

```python
import json
from uuid import uuid1
from datetime import date

import boto3

records_to_send = [{
     'customerId': 'customer-123',
     'meterApiName': 'ComputeHours',
     'meterValue': 5,
     'meterTimeInMillis': 1619445706909,
     'dimensions': {
        'region': 'us-west-2',
        'az': 'az1'
     }
}]

bucket_name = '183-amberflo'

date = datetime.now().strftime(r'%m-%d-%Y')

object_key = 'ingest/amberdata/' + date + '/' + str(uuid1())

s3 = boto3.resource('s3')

s3.Object(bucket_name, object_key).put(Body=json.dumps(records_to_send), ACL='bucket-owner-full-control')
```

## Troubleshooting

If you encounter issues while ingesting data through S3, Amberflo will generate a **failure report **in the following S3 path:

`s3://<bucket_name>/failed-requests/<date>/<original uri>.reason.txt`

This file contains the reason for the failure and can help you diagnose and resolve the issue.

## Compression (gzip support)

Amberflo supports ingesting **gzip-compressed files**.

- Simply upload files with the **.gz extension**
- Amberflo will **automatically decompress** these files during ingestion