> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sawmills.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# CloudWatch Metrics (S3/SQS) Source

> Receive CloudWatch Metric Streams JSON from S3 through SQS notifications, using the Sawmills Collector's AWS identity.

## Supported Data Types

📈 **Metrics**

Select **CloudWatch Metrics** in the Sawmills source picker to ingest CloudWatch metrics delivered to Amazon S3. The collector polls an Amazon SQS queue for object notifications, downloads the objects, and decodes their CloudWatch Metric Streams JSON records into metrics.

```text theme={null}
CloudWatch Metric Stream (JSON)
  → Amazon Data Firehose (Direct PUT, S3 destination, uncompressed)
    → S3 object creation notification → SQS
      → Sawmills Collector → pipeline → metrics destinations
```

The collector makes outbound requests using its own AWS identity, such as an IAM role for a Kubernetes service account (IRSA). This source needs no inbound endpoint, listener port, or shared access key. The pipeline must include a [destination that accepts metrics](/docs/pipeline-destinations).

For AWS resources, IAM policies, and verification steps, see [How to send CloudWatch metrics via S3](/docs/send-metrics-via-cloudwatch-s3).

## Configuration

| Name          | Required | Description                                                                                                                                |
| :------------ | :------- | :----------------------------------------------------------------------------------------------------------------------------------------- |
| Name          | Yes      | Unique source name within Sawmills.                                                                                                        |
| Region        | Yes      | AWS Region of the bucket and queue, for example `us-west-2`. It must match the Region in the queue URL.                                    |
| S3 Bucket     | Yes      | Bucket name, for example `example-cloudwatch-metrics`. Use the name without `s3://` or a path.                                             |
| SQS Queue URL | Yes      | Full queue URL, for example `https://sqs.us-west-2.amazonaws.com/123456789012/example-cloudwatch-metrics`. Use the URL, not the queue ARN. |
| Key Prefix    | No       | Literal beginning of the object key, for example `cwmetrics/`. Empty means no prefix filter. This is not a wildcard or an S3 URI.          |

The encoding is fixed to **CloudWatch Metric Streams JSON** (`cwmetrics`); there is no format selector. Logs, traces, OTLP files, and arbitrary JSON are not supported by this source.

The equivalent source configuration data is:

```yaml theme={null}
region: us-west-2
s3_bucket: example-cloudwatch-metrics
sqs_queue_url: https://sqs.us-west-2.amazonaws.com/123456789012/example-cloudwatch-metrics
encoding: cwmetrics
s3_prefix: cwmetrics/ # Optional; omit for no prefix filter.
```

<Warning>
  Objects outside the configured bucket or prefix are skipped, and their SQS
  notifications are deleted once all records in the message are processed or
  skipped. A wrong bucket or prefix can silently lose metrics even while the
  queue drains. Prefixes are literal: `cwmetrics/` and `/cwmetrics/` match
  different keys. Correcting the source later does not replay deleted
  notifications.
</Warning>

Use a dedicated queue for this source. An unrelated consumer, or another source with a different bucket or prefix, can consume and delete notifications before the intended source receives them.

## AWS permissions

Grant the AWS role used by the collector pods running this source:

* `s3:GetObject` on the bucket's metric objects, scoped to the configured prefix when possible.
* `sqs:ReceiveMessage` and `sqs:DeleteMessage` on the notification queue.

S3 also needs permission to publish notifications to the queue. See the [setup guide](/docs/send-metrics-via-cloudwatch-s3#step-2-allow-s3-to-notify-sqs) for the queue policy, collector policy, and encryption considerations.

## Processing, retries, and dropped data

Use **Sawmills Collector v1.1120.0 or later** for the behavior below and its drop counters.

The receiver accepts native S3 object notifications, either directly or inside an SNS notification. EventBridge message bodies are not supported. A notification is deleted only when all its records have been processed or skipped; a retryable failure retains the whole message.

| Condition                                                                                        | Behavior                                                                                                             |
| :----------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------- |
| Metrics accepted by the next pipeline component                                                  | Marks the object as processed.                                                                                       |
| Object outside the configured bucket or prefix                                                   | Skips the object.                                                                                                    |
| Invalid SQS JSON                                                                                 | Deletes the message; reason `invalid_json`.                                                                          |
| JSON that is not an S3 or SNS notification, including EventBridge                                | Deletes the message; reason `not_s3_notification`.                                                                   |
| Invalid SNS payload or no embedded S3 event                                                      | Deletes the message; reason `invalid_sns_message`.                                                                   |
| No matching decoder, decompression failure, or decoding failure                                  | Skips the object; reason `unsupported_format`, `decompress_failed`, or `decode_failed`.                              |
| S3 download failure, other than `NoSuchKey`, or an error returned by the next pipeline component | Keeps the entire message for retry after the SQS visibility timeout, including records already processed or skipped. |
| Object no longer exists (`NoSuchKey`)                                                            | Treats the record as done.                                                                                           |

Monitor these collector counters, grouped by their `reason` label:

* `otelcol_receiver_awss3_sqs_messages_dropped`: unreadable or unsupported notifications.
* `otelcol_receiver_awss3_objects_dropped`: unsupported object formats, decompression failures, and decoding failures.

Both counters increase only after SQS message deletion succeeds. Bucket and prefix mismatches are not included in these counters, so verify actual metrics arrival as well as queue consumption. See [Monitor the Collector with Prometheus](/docs/guide-how-to-monitor-with-prometheus).

Retries can process an object again. Even for successfully decoded objects, queue deletion only confirms that the next pipeline component accepted the data; it is not a guarantee that a remote destination has stored it. Check destination health and collector export errors as part of verification.

## S3 object retention

The receiver **never deletes S3 objects**. Set an S3 lifecycle rule on the metrics prefix to expire objects according to your retention policy. Allow enough time for queue backlog, outages, and retries before expiration. An object that expires before the collector downloads it cannot be ingested from its notification.