S3 Sink¶

varTrack writes config values to S3 as objects. S3 is ideal for audit storage, batch pipelines, and config files read by many services.

Configuration¶

datasources: [{
  s3: {
    tag:               ""
    bucket:            "my-config-bucket"
    region:            "us-east-1"
    access_key_id:      "AKIA..."
    secret_access_key:  "..."
    endpoint:          ""    // optional — set for MinIO or other S3-compatible stores
  }
}]

MinIO / S3-compatible¶

datasources: [{
  s3: {
    bucket:            "configs"
    endpoint:          "http://minio:9000"
    access_key_id:      "minioadmin"
    secret_access_key:  "minioadmin"
    region:            "us-east-1"
  }
}]

IAM role (no credentials)¶

For services running on AWS with an instance role or IRSA:

datasources: [{
  s3: {
    bucket:       "my-config-bucket"
    region:       "us-east-1"
    use_iam_role: true   // no access_key_id / secret_access_key needed
  }
}]

Destination template¶

The destination_template sets the S3 key prefix:

rules: [{
  platform:             "github"
  datasource:           "s3"
  destination_template: "{tenant}/{env}/"
}]

Config keys are stored as objects under the prefix:

s3://my-config-bucket/acme/production/database.host
s3://my-config-bucket/acme/production/max_connections
s3://my-config-bucket/acme/production/feature.dark_mode

One file per key vs one object per key¶

By default each config key becomes a separate S3 object. Set one_file_per_key: false to write everything into a single JSON object:

datasources: [{
  s3: {
    bucket:          "my-config-bucket"
    one_file_per_key: false   // write one JSON blob instead of per-key objects
  }
}]

Server-side encryption¶

datasources: [{
  s3: {
    bucket:               "my-config-bucket"
    sse_algorithm:        "AES256"     // or "aws:kms"
    // sse_kms_key_id:    "arn:aws:kms:..."  // only for aws:kms
  }
}]

Algorithm	Description
`AES256`	S3-managed keys (SSE-S3)
`aws:kms`	AWS KMS key (SSE-KMS) — specify `sse_kms_key_id`

Object versioning¶

datasources: [{
  s3: {
    bucket:            "my-config-bucket"
    enable_versioning: true   // enable S3 bucket versioning
  }
}]

When enabled, each write creates a new object version, preserving full config history.

Drift detection¶

The watcher lists all objects under the configured prefix and compares their ETags against the baseline. Objects are only re-downloaded on ETag mismatch, keeping bandwidth low.

rules: [{
  platform:   "github"
  datasource: "s3"
  self_heal:  true
}]