Read secrets from GCP Secret Manager¶

The core reads and writes files. GCP Secret Manager is a remote secrets backend, provided by a sibling module, config-gcp-secret, so a consumer who reads secrets from GCP takes it — and its SDK — and one who does not pays nothing.

go get gitlab.com/phpboyscout/go/config-gcp-secret

You build the client — Application Default Credentials, a service account, workload identity — and hand it in with the project it should read:

import (
    secretmanager "cloud.google.com/go/secretmanager/apiv1"
    "gitlab.com/phpboyscout/go/config"
    configgcpsecret "gitlab.com/phpboyscout/go/config-gcp-secret"
)

client, _ := secretmanager.NewClient(ctx)

store, err := config.NewStore(ctx,
    config.WithFiles(fsys, "/etc/app.yaml"),                                   // YAML defaults
    config.WithBackend(configgcpsecret.FromClient(client, "my-project", "")),  // secrets outrank them
)

The empty third argument is the location: "" reads the project-global parent, and a region name reads a regional one. The adapter never authenticates — every GCP credential mechanism works because it knows about none of them.

When you do not need this¶

If the secret already arrives as an environment variable or a mounted file, WithEnv or WithFiles read it with no SDK.

Reach for Secret Manager when your process should fetch for itself: a version that moves under a running process, or an audit trail naming your application.

Names are keys, verbatim¶

Secret IDs allow only letters, digits, hyphens and underscores — no dots, no slashes. There is no hierarchy to map, so an ID becomes a config key exactly as it appears in the console:

db-password  = "s3cr3t"        store.View().GetString("db-password")
api_key      = "k-123"    →    store.View().GetString("api_key")

Hyphens and underscores are not separators, for the reason Azure Key Vault gives: both are legal in ordinary IDs with no character left to escape with, so treating either as a separator would silently restructure every secret you own.

Structure comes from a document instead:

import configjson "gitlab.com/phpboyscout/go/config-json"

config.WithBackend(configgcpsecret.FromClientSecret(client, "my-project", "", "app-config", configjson.Codec{}))

The codec is a parameter, not an option — a payload is one opaque blob, so without something to decode it there is no tree to contribute. In flat mode WithValueCodec is optional and decodes any secret holding a document into a subtree under its own key.

Scoping what gets read¶

Reading a whole project is one listing plus one access per secret, so scoping matters:

configgcpsecret.FromClient(client, "my-project", "",
    configgcpsecret.WithLabel("app", "checkout"),      // server-side
    configgcpsecret.WithNamePrefix("checkout-"))       // hint + client-side re-check

WithLabel filters server-side and is the one to reach for — the service does the work and nothing else is fetched.

WithNamePrefix is weaker than it looks and the adapter compensates. Secret Manager's name: filter is case-insensitive substring containment, not a prefix match — name:app also matches legacy-app-token. So the prefix is sent as a hint to narrow the listing, and then re-checked client-side, because the service's answer is broader than the word "prefix" suggests.

Version states, and the one behaviour that will surprise you¶

latest in Secret Manager means the most recently created version — not the most recently enabled. So disabling a freshly-created version does not hand latest back to the one before it; it makes latest unreadable.

This adapter falls back to the newest ENABLED version in that case, so the application keeps running through a half-completed rotation.

configgcpsecret.WithFallbackObserver(func(f configgcpsecret.Fallback) {
    log.Warn("serving an older secret version",
        "secret", f.ID, "latest", f.LatestVersion, "state", f.LatestState, "served", f.ServedVersion)
})

A secret with no enabled version contributes no key in flat mode, and fails the load in document mode — there is nothing to build a tree from. WithVersion pins an explicit version, and a pinned version never falls back: if you named it and it is unusable, that is an error rather than a substitution.

Payload integrity¶

Secret Manager returns a CRC32C checksum with each payload, and the adapter verifies it. A mismatch fails the read with ErrChecksumMismatch rather than serving bytes that did not survive the trip. A payload with no checksum is served unchecked — the service generates one only when the writer did not supply their own.

Payloads are arbitrary bytes with nothing declaring text versus binary, so a payload that is not valid UTF-8 is skipped in flat mode and fails the load in document mode.

Watching for change¶

No first-class client watch, so this polls, at five minutes by default.

The poll deliberately reads version metadata rather than payloads. That matters for audit: AccessSecretVersion is logged as DATA_READ, while GetSecretVersion is ADMIN_READ, so a quiet poll stays out of the data-access stream your security team watches. It costs no more in the steady state, and only accesses what actually moved.

A poll notices a rotation, a secret appearing or disappearing — and a fallback appearing, which is the case worth naming: a failed rotation does not move the served version at all, so change detection compares the fallback set too.

Writing a secrets-provided key is refused¶

Every value here is a secret, so this backend declares itself Sensitive, and it is read-only. A write to a key it provides would otherwise fall through to the next writable layer — typically a plain YAML file. The core refuses that write:

_, err := store.Apply(ctx, config.Set("db-password", "rotated"))
// errors.Is(err, config.ErrSensitiveLeak) — refused, and app.yaml is untouched

Getting a client¶

Building the client yourself is the default. Two further rungs exist:

// You hold client options — an emulator endpoint, an explicit credentials file,
// or a credential resolved once with go/gcpclient and shared across adapters.
b, err := configgcpsecret.FromOptions(ctx, "my-project", "", opts)

// Application Default Credentials.
b, err := configgcpsecret.Default(ctx, "my-project", "")
defer b.Close()

To detect Application Default Credentials once and feed several GCP adapters from it, resolve with go/gcpclient and pass the options it yields to FromOptions. It hands out client options rather than a client precisely because the three GCP adapters need three different client types.

Both return *OwnedBackend, which you should Close. secretmanager.NewClient opens a gRPC connection its own documentation says must be closed, and config.Backend has no Close — so a client the adapter built for itself would otherwise live for the whole process. Nothing for a one-shot CLI; a leak for a long-lived service.

The project is still required. Application Default Credentials authenticate a principal — they do not name the project whose secrets to read, and guessing would read another project's. ErrNoProject refuses at construction.

What it costs¶

This is the heaviest adapter in the toolkit — roughly five times AWS Secrets Manager's five modules or Azure Key Vault's six, because the Google API client stack is large and shared. Worth knowing before you add it to a small binary. The figure is pinned by an allowlist test in both directions.

Related¶

• How dynamic backends work — the mechanics every remote backend shares
• Read secrets from Azure Key Vault — the other flat-namespace secrets store
• Read secrets from Vault and from AWS Secrets Manager
• The adapter ecosystem — every adapter, status and roadmap