Read secrets from Vault¶

The core reads and writes files. HashiCorp Vault is a remote secrets backend, provided by a sibling module, config-vault, so a consumer who reads secrets from Vault takes it — and its Vault SDK — and one who does not pays nothing.

go get gitlab.com/phpboyscout/go/config-vault

You build and authenticate the Vault client — that is where every auth method, address, namespace and TLS decision lives — and hand it in with the KV v2 mount and the secret path:

import (
    vaultapi "github.com/hashicorp/vault/api"
    "gitlab.com/phpboyscout/go/config"
    configvault "gitlab.com/phpboyscout/go/config-vault"
)

client, _ := vaultapi.NewClient(vaultapi.DefaultConfig())
client.SetToken(token)

store, err := config.NewStore(ctx,
    config.WithFiles(fsys, "/etc/app.yaml"),                             // YAML defaults
    config.WithBackend(configvault.FromClient(client, "secret", "app")), // Vault outranks them
)

A Vault layer takes part in precedence, per-key merge, provenance and hot-reload exactly as a file does. The secret's fields become the layer, keeping the structure Vault stores them with:

secret/app = {                          store.View().GetString("db.host") // "db.internal"
  "db": { "host": "db.internal",   →    store.View().GetInt("db.port")    // 5432
          "port": 5432 } }

Unlike the byte-valued stores — Consul, the parameter stores — Vault returns already-structured JSON, so nested maps, slices, booleans and nulls all survive the round trip. There is no value codec here and none is needed.

When you do not need this¶

If your platform already delivers the secret — an injected environment variable, or a file mounted by an agent or sidecar — then WithEnv, WithFiles or config-filekv read it with no Vault SDK and no token to manage.

Reach for config-vault when your process should authenticate and fetch for itself: short-lived credentials, values that rotate under a running process, or an audit trail that has to name your application rather than the agent.

Authenticate however you like — the adapter never does¶

config-vault has no auth method, no address and no credential anywhere in its API. It uses the client you give it, which is what lets every Vault auth method work without the adapter knowing any of them.

Every method ends the same way — an authenticated *api.Client goes into FromClient:

// AppRole — the common service-to-service path.
appRole, _ := approle.NewAppRoleAuth(roleID, &approle.SecretID{FromEnv: "APPROLE_SECRET_ID"})
client.Auth().Login(ctx, appRole)

// Kubernetes — the common in-cluster path.
k8s, _ := kubernetes.NewKubernetesAuth("your-role")
client.Auth().Login(ctx, k8s)

// …then, either way:
config.WithBackend(configvault.FromClient(client, "secret", "app"))

The auth helper packages (vault/api/auth/approle, .../kubernetes) are separate modules — add whichever you use; config-vault depends on none of them.

watcher, _ := client.NewLifetimeWatcher(&vaultapi.LifetimeWatcherInput{Secret: login})
go watcher.Start()
defer watcher.Stop()

One secret, or a tree of them¶

FromClient reads one secret — the common shape, one request, one read grant in your policy. When configuration genuinely spans a tree, FromClientPrefix walks a prefix recursively, stripping it and nesting each secret's fields under its remaining path segments:

secret/app        = { "name": "checkout" }         name: checkout
secret/app/db     = { "user": "admin" }       →    db:    { user: admin }
secret/app/cache  = { "url": "redis://…" }         cache: { url: "redis://…" }

config.WithBackend(configvault.FromClientPrefix(client, "secret", "app"))

Prefix mode is opt-in because its cost is real: Vault has no recursive list, so a walk is one request per directory plus one per secret, and it needs the list capability on every directory it descends — which least-privilege policies often withhold. Each watch poll pays that same cost again.

A field and a child secret cannot share a name¶

In Vault a path is both a secret and a directory — listing a prefix returns app and app/ for the same name. In prefix mode that means a secret's own fields merge with its child secrets into one node, and they can collide:

secret/app     = { "db": "postgres://…" }     ← a field named "db"
secret/app/db  = { "host": "db.internal" }    ← a child secret also named "db"

Both claim db. There is no correct silent answer — either resolution throws away a value you can see in Vault — so the load is refused:

_, err := config.NewStore(ctx,
    config.WithBackend(configvault.FromClientPrefix(client, "secret", "app")))

// errors.Is(err, configvault.ErrFieldCollision)
// configvault: secret field collides with a child secret: "app" holds a field "db" and a child secret "app/db"

Fix it in Vault — rename the field, or move the child secret. The error names the path and the segment so you know which two to look at. This is the same choice config-xml makes for an attribute colliding with a child element, and the reasoning is in How dynamic backends work.

Single-secret mode cannot produce this, having only one flat map of fields — a further reason it is the default.

Writing a Vault-provided key is refused¶

Every value in Vault is a secret, so this backend declares itself Sensitive, and it is read-only. Those two facts combine into a guard worth meeting here rather than in production.

Because the layer is read-only, a write to a key it provides cannot land in Vault — so it would otherwise fall through to the next writable layer, typically a plain YAML file on disk. The core refuses that write:

_, err := store.Apply(ctx, config.Set("db.password", "rotated"))
// errors.Is(err, config.ErrSensitiveLeak) — refused, and app.yaml is untouched

That is the core stopping a secret being written into a plaintext file. If a key needs to be writable, do not source it from Vault. See sensitive read-only backends for the full reasoning.

Watching for change¶

Vault's Go client offers no change feed, so watching is polling, at 60 seconds by default — deliberately slower than the file watcher, because every poll is an authenticated read that lands in Vault's audit log.

config.WithBackend(configvault.FromClient(client, "secret", "app",
    configvault.WithPollInterval(5*time.Minute)))

A poll compares the secret's KV v2 version, so another client writing it is noticed and reaches your observers. Vault being briefly unreachable — sealed, restarting, mid-renewal — backs off and retries rather than ending the watch.

Two limits worth knowing¶

Vault rounds integers above 2^53. Vault decodes submitted JSON numbers through a float, so an integer larger than 9007199254740992 is rounded on write, before this adapter sees it:

kv.Put(ctx, "app", map[string]any{"id": 9007199254740993})
store.View().Get("id")  // 9007199254740992

If you keep large identifiers or nanosecond timestamps in Vault, store them as strings, which round-trip exactly. The adapter converts integers as int64 rather than through a float, so it adds no further loss — but it cannot recover what Vault has already discarded.

KV v2 only. Vault's default and recommended secrets engine. KV v1 is legacy, has no version metadata for the watch to compare, and lists through a different path.

Vault Enterprise namespaces are set on the client (client.SetNamespace(…)), so the adapter is namespace-agnostic and takes no parameter for it.

Getting a client¶

Building the client yourself is the default. Two further rungs exist, each in both shapes:

// You assembled the config; the adapter builds the client.
b, err := configvault.FromConfig(cfg, "secret", "app/config")
b, err := configvault.FromConfigPrefix(cfg, "secret", "app/")

// Nothing at all: VAULT_ADDR, VAULT_TOKEN and friends.
b, err := configvault.Default("secret", "app/config")
b, err := configvault.DefaultPrefix("secret", "app/")

Default inherits Vault's own documented https://127.0.0.1:8200, not one this adapter invented.

Every rung checks something the SDK makes easy to miss: vaultapi.DefaultConfig reports failure by populating the Error field on the config it returns rather than by returning an error, so a config that looks fine can already be broken.

To share one Vault client across this adapter, go/signing and go/encryption, use go/vaultclient and hand the client to FromClient.

The token is read once, and no rung renews it¶

vaultapi.NewClient reads VAULT_TOKEN a single time and holds it for the life of the client. Nothing re-reads the environment and nothing renews the lease, so a backend carries whatever token was set when it was built for as long as your Store holds it.

For a command that is fine — it rarely outlives its token. For a long-lived process reloading configuration it is not: once the TTL expires every subsequent Load fails with a permission error, and there is no path back, because the backend cannot learn that its credential lapsed.

This is a property of Vault's client rather than of any one rung — FromClient has always behaved the same way. But Default is where it is easiest to meet without noticing, precisely because you never handle the token yourself. If the process outlives the TTL, build the client, keep a vaultapi.LifetimeWatcher renewing it, and pass it to FromClient:

client, err := vaultapi.NewClient(vaultapi.DefaultConfig())
// ... start a LifetimeWatcher renewing client's token ...
b := configvault.FromClient(client, "secret", "app/config")

What it costs¶

A backend adapter carries its system's client, and the Vault SDK is the largest thing here. That cost is pinned by an allowlist test in the module, so a version bump that widens the graph fails its build rather than arriving quietly.

Related¶

• How dynamic backends work — the mechanics every remote backend shares
• The adapter ecosystem — every adapter, status and roadmap
• Backends and capabilities — the interface split all backends implement
• Read and write Consul — the reference backend, read + write + native watch