S3 Sink¶

varTrack writes config values to S3 as objects. S3 is ideal for audit storage, batch pipelines, and config files read by many services.

Configuration¶

datasources: [{
  s3: {
    tag:               ""
    bucket:            "my-config-bucket"
    region:            "us-east-1"
    access_key_id:      "AKIA..."
    secret_access_key:  "..."
    endpoint:          ""    // optional — set for MinIO or other S3-compatible stores
  }
}]

MinIO / S3-compatible¶

datasources: [{
  s3: {
    bucket:            "configs"
    endpoint:          "http://minio:9000"
    access_key_id:      "minioadmin"
    secret_access_key:  "minioadmin"
    region:            "us-east-1"
  }
}]

IAM role (no credentials)¶

For services running on AWS with an instance role or IRSA, leave out access_key_id and secret_access_key: the sink then uses the AWS default credential chain.

datasources: [{
  s3: {
    bucket:       "my-config-bucket"
    region:       "us-east-1"
    use_iam_role: true   // documents the intent; no keys needed
  }
}]

access_key_id and secret_access_key take any credential form: a string, {file: …} or {ref: …} from Vault. prefix sets a key prefix used when the rule has no destination_template.

Destination template¶

The destination_template sets the S3 key prefix:

rules: [{
  platform:             "github"
  datasource:           "s3"
  file_name:           "app-settings.yaml"
  branch_map:           {main: "production"}
  destination_template: "{tenant}/{env}/"
}]

Write strategies¶

The datasource's update_strategy decides what is written under the prefix:

datasources: [{
  s3: {
    bucket:          "my-config-bucket"
    update_strategy: "STRATEGY_DOCUMENT"   // or "STRATEGY_KEY_VALUE", "STRATEGY_FILE"
  }
}]

`update_strategy`	Objects under the prefix
`STRATEGY_KEY_VALUE`	One object per key: `acme/production/database.host` holds `mongo.prod.internal`
`STRATEGY_DOCUMENT`	`config.json`, always nested: `{"database": {"host": "mongo.prod.internal"}, "max_connections": 50}`
`STRATEGY_FILE`	`config.json`, in the shape the file parsed to: nested for YAML, JSON, TOML…, flat for `.env` and `.properties`
unset	One object per key with `one_file_per_key: true`; otherwise `config.json` with the keys flat: `{"database.host": "mongo.prod.internal", "max_connections": "50"}`

STRATEGY_DOCUMENT and STRATEGY_FILE keep the types and arrays the file has. A flat file under STRATEGY_DOCUMENT is nested by the dots in its keys; one where a key is both a value and the parent of other keys fails to sync. Use STRATEGY_FILE for that file.

Encryption, versioning and connection¶

datasources: [{
  s3: {
    bucket:                 "my-config-bucket"
    server_side_encryption: "aws:kms"            // or "AES256"
    kms_key_id:             "arn:aws:kms:us-east-1:123456789012:key/…"
    enable_versioning:      true
    session_token:          {file: "/run/secrets/aws-session-token"}
    force_path_style:       true                 // MinIO and other S3-compatible stores
    timeout:                "30s"
    ssl_ca_cert:            "/etc/ssl/minio-ca.pem"
    content_type:           "application/json"
    file_pattern:           "settings-{env}.json"
  }
}]

Field	Effect
`server_side_encryption`, `kms_key_id`	Sent with every object written
`enable_versioning`	Versioning is turned on for the bucket when the sink first writes to it; without the permission to, the write goes on and the log says why
`session_token`	Temporary credentials, with `access_key_id` and `secret_access_key`
`force_path_style`	Path-style addressing (`endpoint/bucket/key`)
`timeout`	Connect and read timeout
`insecure_skip_verify: true`	Turns certificate checks off. They are on unless this is set; `ssl_ca_cert` is the path of a CA bundle to check against instead. (Written `verify_ssl: false` in older configs, which still loads.)
`content_type`	The objects' `Content-Type`; otherwise JSON for `config.json` and a guess per key
`file_pattern`	The name of the single object (`config.json` by default); `{env}` is the environment

version_id is not used when writing.

Drift detection¶

The watcher asks the orchestrator each poll; the orchestrator reads the objects varTrack wrote and compares their keys with what it last synced from Git — a changed or missing key is drift (Drift Detection).

rules: [{
  platform:   "github"
  datasource: "s3"
  file_name: "app-settings.yaml"
  branch_map: {main: "production"}
  self_heal:  true
}]