Skip to content

The adapter ecosystem

One store, many sources. YAML files are the default config reads out of the box, but they are not the boundary. A file in another format, or configuration that lives in a remote system rather than a file at all, joins the store as an ordinary layer — with the same precedence, the same per-key provenance, the same coherent snapshots and the same fail-closed reload every other layer gets.

That is the difference that matters. A library that treats "remote config" as a bolt-on special case gives you a value and little else; here there is no special case. Consul, a parameter store or a secrets manager takes part in the merge exactly as /etc/app.yaml does, and Explain("server.port") will name it as the source. This is what the previous Viper-shaped world could not do.

Every adapter is its own sibling module, depended on only by the consumers who use it. Your dependency graph carries the one integration you reached for and nothing else: a consumer reading TOML never compiles the XML parser, and a consumer configuring from Consul never pulls a cloud SDK it does not touch.

What every adapter inherits

An adapter only teaches the store how to read (and, where it makes sense, write and watch) one kind of source. Everything else is the core's job, so it is identical across the whole family:

  • Precedence — the adapter's layer sits wherever you place it in the source order.
  • ProvenanceExplain and Origin name the adapter as the source of a value, and flag where it shadows or is shadowed.
  • Coherent reads — a View is pinned to one snapshot regardless of which adapters fed it.
  • Write fidelity — where an adapter supports writes, the change lands in the layer that owns the key, and structure (comments, order, quoting) is preserved.
  • Fail-closed reload — a source that will not parse or fails your schema is rejected; last-known-good stays live.

File & format adapters

Available now, each published and versioned. The adapter name links to its how-to guide.

Adapter Handles Reads Writes Source
config-json JSON & JSON Lines (structure-preserving) repo · API
config-toml TOML (structure-preserving) repo · API
config-hcl HCL (as a config format, not Terraform) repo · API
config-xml XML repo · API
config-dotenv dotenv (.env) repo · API
config-ini INI repo · API
config-properties Java .properties repo · API

The read-only format adapters (dotenv, ini, properties, xml) add no third-party dependency — they parse their format in-module. And if the format you need is not here, write a format adapter: a codec is a Decode/Encode pair, and the store handles the rest.

Filesystem adapters

A format adapter decides how a config file is parsed; a filesystem adapter decides where it lives. The two compose — you can read TOML out of an S3 bucket — because config imposes no filesystem of its own: config.FS is six methods you can satisfy over the real disk (config.OS()), a rooted directory (config.Dir), an embedded filesystem, a remote host or a cloud object store. A filesystem adapter reads a configuration file that happens to live somewhere other than local disk — which is what distinguishes it from a dynamic backend, that maps a remote key–value namespace instead.

Two implementations ship in the core — config.OS() and config.Dir(path) — so you only reach for an adapter when the file lives somewhere neither covers.

Adapter Where the file lives Reads Writes Source
config-afero an existing afero filesystem repo · API
config-iofs any io/fs.FSembed.FS, zip, tar repo · API
config-billy a go-billy filesystem (go-git) repo · API
config-sftp a remote host over SSH repo · API

config-iofs is read-only because io/fs is read-only by design, and adds no third-party dependency. How reload, read-only capability and the object-store commit work across the family is How filesystem adapters work; the filesystem adapters spec is the umbrella that governs it.

Cloud object stores

The three cloud object stores share one model — a config file that lives in a bucket, watched by polling, committed with the store's best atomic primitive — and differ only where the services do (GCS has a native atomic move; S3 and Azure Blob copy-then-delete). Each is its own approved spec, built and released independently.

Adapter Store Commit Status Source
config-aws-s3 AWS S3 copy-then-delete v0.1.0 repo · API
config-gcp-gcs GCP Cloud Storage native Move v0.1.0 repo · API
config-azure-blob Azure Blob Storage copy-then-delete v0.1.0 repo · API

All three are read+write, poll at a 60-second default (a poll is a billed object read; WithPollInterval overrides), and are proven against a real emulator — LocalStack, fake-gcs-server and Azurite — in a Docker-in-Docker job.

Dynamic backends

The next chapter, and the one that pulls furthest ahead of a file-only tool: configuration fetched at runtime from a remote system, given full precedence, provenance and hot-reload exactly as a file is. The seam already exists and is proven — WithBackend takes anything satisfying a three-method Backend, with writes and native watch as opt-in capabilities. The dynamic backend adapters spec is the umbrella that governs the whole family.

config-consul — the first

config-consul — released at v0.1.0 (API) — reads and writes configuration from HashiCorp Consul through config. You build and configure the Consul client — every address, token, TLS and datacenter decision stays yours — and hand it in with a prefix that scopes and is stripped from the keys:

import (
    capi "github.com/hashicorp/consul/api"
    "gitlab.com/phpboyscout/go/config"
    configconsul "gitlab.com/phpboyscout/go/config-consul"
)

client, _ := capi.NewClient(capi.DefaultConfig())

store, err := config.NewStore(ctx,
    config.WithFiles(fsys, "/etc/app.yaml"),                     // YAML defaults
    config.WithBackend(configconsul.FromClient(client, "app/")), // Consul outranks them
)

A Consul layer takes part in precedence, per-key merge, provenance and hot-reload exactly as a file does — and Explain will tell you when a value came from Consul rather than the file beneath it. It is the reference implementation for everything that follows. Learn it by building one in the Configure from Consul tutorial, reach for a specific operation in the Read & write Consul how-to, or read How the Consul backend works for the data, conflict and watch models behind it.

Parameter stores — released

Consul's siblings, the cloud parameter stores, are all released at v0.1.0. They share Consul's shape — injected client, prefix-scoped nested tree, values decoded through an injected codec — and differ where the systems do, which is what How dynamic backends work explains. In short: they poll rather than watch natively, and they split on compare-and-swap.

  • config-aws-ssm — AWS SSM Parameter Store. Read-only (SSM has no compare-and-swap); SecureString values read decrypted and mark the layer sensitive so the leak guard protects them.
  • config-azure-appconfig — Azure App Configuration. Read and write on per-key ETag compare-and-swap; scoped by a label.
  • config-gcp-parameter — GCP Parameter Manager. Read-only, in two shapes: one parameter as a whole document, or a prefix of many parameters.

Roadmap

Each adapter below carries its own SDK, its own authentication and its own consistency and watch semantics, so — by the umbrella's headline rule — each gets its own approved spec before it is built. The grouping is a planned order, not a commitment date.

Adapter System Phase Status
config-consul HashiCorp Consul A — reference & parameter stores Released · v0.1.0
config-aws-ssm AWS SSM Parameter Store A Released · v0.1.0 (read-only)
config-azure-appconfig Azure App Configuration A Released · v0.1.0
config-gcp-parameter GCP Parameter Manager A Released · v0.1.0 (read-only)
config-vault HashiCorp Vault B — secrets managers Planned (read-only by default)
config-aws-secrets AWS Secrets Manager B Planned (read-only by default)
config-azure-keyvault Azure Key Vault B Planned (read-only by default)
config-gcp-secret GCP Secret Manager B Planned (read-only by default)
config-etcd etcd C — cloud-native key–value Planned (native watch)
config-k8s Kubernetes ConfigMaps C Planned (native watch)

Secrets managers ship read-only by default — a config tool writing a secret is a rarer and riskier thing than reading one, so write support for those is opt-in and specified per adapter. Feature-flag systems are deliberately out of scope.

Build your own

Nothing here is a closed set. The same two seams the family is built on are yours to use:

  • Write a custom backend — make any remote system (a secrets manager, an HTTP endpoint, an internal service) an ordinary layer, walked end to end against a Consul-shaped example.
  • Support a new file format — a codec is a Decode/Encode pair; add one and every store feature comes with it for free.