Skip to main content

​
Supported Data Types

📘 Logs | 📈 Metrics | 🚦 Traces

​
Configuration

FieldTypeDefaultRequiredDescription
NameStringnonetrueUnique identifier within Sawmills.
RegionString”us-east-1”trueAWS region.
S3 BucketStringnonetrueS3 bucket name.
Role ARNStringnonefalseThe Role ARN to be assumed.
File PrefixStringnonefalsePrefix for the S3 key (root directory inside the bucket).
MarshalerStringotlp_jsonfalseMarshaler used to produce output data (see Marshaler options below).

​
Advanced Options

FieldTypeDefaultRequiredDescription
REST API EndpointStringNonefalseOverrides the endpoint, instead of constructing it from region and s3_bucket.
S3 Force Path StyleBooleanfalsefalseSet this to true to force the request to use path-style addressing.
Use SSLBooleantruefalseSet this to false to disable SSL when sending requests.
CompressionStringnonefalseShould the file be compressed.

​
Marshaler

The Marshaler determines the format of the data sent to AWS S3.
  • otlp_json (default): The OpenTelemetry Protocol format represented as JSON.
  • otlp_proto: The OpenTelemetry Protocol format represented as Protocol Buffers. A single protobuf message is written into each object.
  • sumo_ic: The Sumo Logic Installed Collector Archive format (supported only for logs).
  • body: Exports the log body as a string (applies only for logs).
  • datadog: Datadog rehydrate logs format represented as JSON.
  • datadog_parquet: Datadog logs format represented as Parquet files.

​
Snowflake Table Schema for Datadog Parquet

When using the datadog_parquet Marshaler, you can create a Snowflake table with the following schema: 
CREATE TABLE PARQUET_LOGS (
    Date TIMESTAMP_LTZ,
    Service VARCHAR,
    Source VARCHAR,
    Host VARCHAR,
    Status VARCHAR,
    Message VARCHAR,
    Attributes OBJECT,
    Tags OBJECT
);

​
AWS Credential Configuration

Sawmills collector is running on a Kubernetes cluster and uses helm charts. To provide AWS credentials, pass extraEnvs in values.yaml as shown below: 
extraEnvs:
  - name: AWS_ACCESS_KEY_ID
    value: "<YOUR AWS ACCESS KEY>"
  - name: AWS_SECRET_ACCESS_KEY
    value: "<YOUR AWS SECRET ACCESS KEY>"