The adapter ecosystem¶
One store, many sources. YAML files are the default config reads out of the box, but
they are not the boundary. A file in another format, or configuration that lives in a remote
system rather than a file at all, joins the store as an ordinary layer — with the same
precedence, the same per-key provenance, the same coherent snapshots and the same fail-closed
reload every other layer gets.
That is the difference that matters. A library that treats "remote config" as a bolt-on
special case gives you a value and little else; here there is no special case. Consul, a
parameter store or a secrets manager takes part in the merge exactly as /etc/app.yaml does,
and Explain("server.port") will name it as the source. This is what the previous
Viper-shaped world could not do.
Every adapter is its own sibling module, depended on only by the consumers who use it. Your dependency graph carries the one integration you reached for and nothing else: a consumer reading TOML never compiles the XML parser, and a consumer configuring from Consul never pulls a cloud SDK it does not touch.
What every adapter inherits¶
An adapter only teaches the store how to read (and, where it makes sense, write and watch) one kind of source. Everything else is the core's job, so it is identical across the whole family:
- Precedence — the adapter's layer sits wherever you place it in the source order.
- Provenance —
ExplainandOriginname the adapter as the source of a value, and flag where it shadows or is shadowed. - Coherent reads — a
Viewis pinned to one snapshot regardless of which adapters fed it. - Write fidelity — where an adapter supports writes, the change lands in the layer that owns the key, and structure (comments, order, quoting) is preserved.
- Fail-closed reload — a source that will not parse or fails your schema is rejected; last-known-good stays live.
File & format adapters¶
Available now, each published and versioned. The adapter name links to its how-to guide.
| Adapter | Handles | Reads | Writes | Source |
|---|---|---|---|---|
config-json |
JSON & JSON Lines | ✓ | ✓ (structure-preserving) | repo · API |
config-toml |
TOML | ✓ | ✓ (structure-preserving) | repo · API |
config-hcl |
HCL (as a config format, not Terraform) | ✓ | ✓ | repo · API |
config-xml |
XML | ✓ | — | repo · API |
config-dotenv |
dotenv (.env) |
✓ | — | repo · API |
config-ini |
INI | ✓ | — | repo · API |
config-properties |
Java .properties |
✓ | — | repo · API |
The read-only format adapters (dotenv, ini, properties, xml) add no third-party
dependency — they parse their format in-module. And if the format you need is not here,
write a format adapter: a codec is a Decode/Encode pair, and
the store handles the rest.
Filesystem adapters¶
A format adapter decides how a config file is parsed; a filesystem adapter decides where it
lives. The two compose — you can read TOML out of an S3 bucket — because config imposes no
filesystem of its own: config.FS is six methods you can satisfy over the
real disk (config.OS()), a rooted directory (config.Dir), an embedded filesystem, a remote host
or a cloud object store. A filesystem adapter reads a configuration file that happens to live
somewhere other than local disk — which is what distinguishes it from a dynamic backend,
that maps a remote key–value namespace instead.
Two implementations ship in the core — config.OS() and config.Dir(path) — so you only reach for
an adapter when the file lives somewhere neither covers.
| Adapter | Where the file lives | Reads | Writes | Source |
|---|---|---|---|---|
config-afero |
an existing afero filesystem | ✓ | ✓ | repo · API |
config-iofs |
any io/fs.FS — embed.FS, zip, tar |
✓ | — | repo · API |
config-billy |
a go-billy filesystem (go-git) | ✓ | ✓ | repo · API |
config-sftp |
a remote host over SSH | ✓ | ✓ | repo · API |
config-iofs is read-only because io/fs is read-only by design, and adds no third-party
dependency. How reload, read-only capability and the object-store commit work across the family is
How filesystem adapters work; the
filesystem adapters spec is the umbrella
that governs it.
Cloud object stores¶
The three cloud object stores share one model — a config file that lives in a bucket, watched by polling, committed with the store's best atomic primitive — and differ only where the services do (GCS has a native atomic move; S3 and Azure Blob copy-then-delete). Each is its own approved spec, built and released independently.
| Adapter | Store | Commit | Status | Source |
|---|---|---|---|---|
config-aws-s3 |
AWS S3 | copy-then-delete | v0.1.0 | repo · API |
config-gcp-gcs |
GCP Cloud Storage | native Move |
v0.1.0 | repo · API |
config-azure-blob |
Azure Blob Storage | copy-then-delete | v0.1.0 | repo · API |
All three are read+write, poll at a 60-second default (a poll is a billed object read;
WithPollInterval overrides), and are proven against a real emulator — LocalStack, fake-gcs-server
and Azurite — in a Docker-in-Docker job.
Dynamic backends¶
The next chapter, and the one that pulls furthest ahead of a file-only tool: configuration
fetched at runtime from a remote system, given full precedence, provenance and hot-reload
exactly as a file is. The seam already exists and is proven —
WithBackend takes anything satisfying a three-method Backend,
with writes and native watch as opt-in capabilities. The
dynamic backend adapters spec is the
umbrella that governs the whole family.
config-consul — the first¶
config-consul — released at
v0.1.0 (API) — reads and writes
configuration from HashiCorp Consul through config. You build and
configure the Consul client — every address, token, TLS and datacenter decision stays yours —
and hand it in with a prefix that scopes and is stripped from the keys:
import (
capi "github.com/hashicorp/consul/api"
"gitlab.com/phpboyscout/go/config"
configconsul "gitlab.com/phpboyscout/go/config-consul"
)
client, _ := capi.NewClient(capi.DefaultConfig())
store, err := config.NewStore(ctx,
config.WithFiles(fsys, "/etc/app.yaml"), // YAML defaults
config.WithBackend(configconsul.FromClient(client, "app/")), // Consul outranks them
)
A Consul layer takes part in precedence, per-key merge, provenance and hot-reload exactly as a
file does — and Explain will tell you when a value came from Consul rather than the file
beneath it. It is the reference implementation for everything that follows. Learn it by building
one in the Configure from Consul tutorial, reach for a specific operation in
the Read & write Consul how-to, or read How the Consul backend
works for the data, conflict and watch models behind it.
Parameter stores — released¶
Consul's siblings, the cloud parameter stores, are all released at v0.1.0. They share Consul's shape — injected client, prefix-scoped nested tree, values decoded through an injected codec — and differ where the systems do, which is what How dynamic backends work explains. In short: they poll rather than watch natively, and they split on compare-and-swap.
config-aws-ssm— AWS SSM Parameter Store. Read-only (SSM has no compare-and-swap);SecureStringvalues read decrypted and mark the layer sensitive so the leak guard protects them.config-azure-appconfig— Azure App Configuration. Read and write on per-key ETag compare-and-swap; scoped by a label.config-gcp-parameter— GCP Parameter Manager. Read-only, in two shapes: one parameter as a whole document, or a prefix of many parameters.
Roadmap¶
Each adapter below carries its own SDK, its own authentication and its own consistency and watch semantics, so — by the umbrella's headline rule — each gets its own approved spec before it is built. The grouping is a planned order, not a commitment date.
| Adapter | System | Phase | Status |
|---|---|---|---|
config-consul |
HashiCorp Consul | A — reference & parameter stores | Released · v0.1.0 |
config-aws-ssm |
AWS SSM Parameter Store | A | Released · v0.1.0 (read-only) |
config-azure-appconfig |
Azure App Configuration | A | Released · v0.1.0 |
config-gcp-parameter |
GCP Parameter Manager | A | Released · v0.1.0 (read-only) |
config-vault |
HashiCorp Vault | B — secrets managers | Planned (read-only by default) |
config-aws-secrets |
AWS Secrets Manager | B | Planned (read-only by default) |
config-azure-keyvault |
Azure Key Vault | B | Planned (read-only by default) |
config-gcp-secret |
GCP Secret Manager | B | Planned (read-only by default) |
config-etcd |
etcd | C — cloud-native key–value | Planned (native watch) |
config-k8s |
Kubernetes ConfigMaps | C | Planned (native watch) |
Secrets managers ship read-only by default — a config tool writing a secret is a rarer and riskier thing than reading one, so write support for those is opt-in and specified per adapter. Feature-flag systems are deliberately out of scope.
Build your own¶
Nothing here is a closed set. The same two seams the family is built on are yours to use:
- Write a custom backend — make any remote system (a secrets manager, an HTTP endpoint, an internal service) an ordinary layer, walked end to end against a Consul-shaped example.
- Support a new file format — a codec is a
Decode/Encodepair; add one and every store feature comes with it for free.