Administrative guide

Deploy, secure, and operate Triage

Everything a platform owner needs to stand up a self-hosted Triage instance and keep it running: deployment targets, the configuration surface, authentication and RBAC, multi-instance federation, data retention and payload encryption, and licensing.

What Triage needs

Runtime

A single Go binary with the React UI embedded — no sidecars, no reverse proxy required. Ships as a Docker image or a standalone binary.

Database

PostgreSQL 16 — stores the audit log, replay rules, users, transforms, and configuration.

Cache

Redis 7 — queue-depth time-series and worker/leader coordination.

Host

Any Linux server. 1 vCPU / 1 GB RAM is enough to start; scale up for high replay volume.

Provider

Credentials for exactly one message broker per instance. Storage (S3/GCS/Azure Blob) is optional and configured independently.

Clients

Users reach it from any modern browser. Nothing is installed on user machines.

All configuration is via environment variables — Triage reads them with os.Getenv and has no opinion about where they come from. Secret storage and injection are the platform’s job (Secrets Manager, SSM, K8s Secrets, or sops + age for Compose); Triage just reads the resulting env.

Deployment targets

Triage is one artifact — the same image runs everywhere. Pick the target that matches your platform.

Evaluation one-liner

Request an eval license from the site, then run the install script. It seeds Postgres and Redis, pulls the published image, and brings the stack up with a provider profile. The fastest path to a running instance.

curl -fsSL https://.../install.sh | sh
# seeds Postgres + Redis, pulls the image, starts Triage

Docker (single container)

Pull the published image (pin a release tag in production) and run it against your own Postgres and Redis, passing configuration as environment variables.

docker run -p 8443:8443 \
  -e PROVIDER_TYPE=sqs -e PROVIDER_NAME=sqs-prod \
  -e DATABASE_URL=... -e REDIS_URL=... \
  -e JWT_SECRET=... -e LICENSE_KEY=... \
  ghcr.io/.../triage:vX.Y.Z

Docker Compose

Bring up Triage with bundled Postgres and Redis. Provider credentials come from your environment; DB/Redis URLs are wired automatically. Provider-specific overrides and a Prometheus profile are available.

docker compose up -d
# provider creds from the host env (sops exec-env recommended)

ECS / Fargate

Run the image as a Fargate task. Secrets are supplied through the task definition’s secrets: block, referencing AWS Secrets Manager or SSM Parameter Store. TLS is typically terminated at the load balancer (set TLS_DISABLED=true).

taskDefinition.secrets:
  - JWT_SECRET  -> secretsmanager:.../jwt
  - LICENSE_KEY -> secretsmanager:.../license

Kubernetes / Helm

Deploy with the provided Helm chart. Create a Kubernetes Secret out of band and reference it with existingSecret — the chart injects every key via envFrom.secretRef. The API runs active-active behind a Service.

helm install triage ./helm/triage \
  --set existingSecret=triage-secrets \
  --set provider.type=anypointmq

Configuration reference

Copy .env.example as a starting point. The essentials:

Required for every deployment

PROVIDER_TYPE	Queue backend: sqs · rabbitmq · activemq · anypointmq · azservicebus · googlepubsub
PROVIDER_NAME	Logical label stamped on every audit record (e.g. sqs-prod)
DATABASE_URL	Postgres connection string — use sslmode=require in production
REDIS_URL	Redis connection string — rediss:// for TLS
JWT_SECRET	Session-token signing key. You generate it (openssl rand -base64 32); Triage refuses to boot without it
LICENSE_KEY	Your issued license key

Server & TLS

PORT	Listen port (default 8443)
BASE_URL	Public URL of the instance — required for OAuth/OIDC redirects
POLL_INTERVAL	Queue-depth polling cadence (default 30s)
TLS_CERT_FILE / TLS_KEY_FILE	PEM cert + key. If neither these nor TLS_DISABLED are set, a self-signed cert is generated at startup
TLS_DISABLED	Set true when TLS is terminated upstream (load balancer / ingress)

About `JWT_SECRET`

Treat it as one long-lived secret per environment. Keep it stable across releases — the same secret means existing sessions survive upgrades. Every replica must get the identical value (the API runs active-active, so a token minted by one replica must verify on another). Rotate it only on suspected compromise — changing it invalidates every outstanding session at once. Session tokens themselves are ephemeral, minted per login and re-checked against the user’s current state on every request, so disabling a user or changing a password revokes their tokens immediately rather than waiting for expiry (TOKEN_EXPIRY, default 8h).

Queue providers

One instance manages one provider. Set PROVIDER_TYPE and the matching credentials.

Provider	PROVIDER_TYPE	Key variables
AWS SQS	sqs	AWS_REGION, AWS_ROLE_ARN (or an IAM instance role). Supports cross-account role assumption.
RabbitMQ	rabbitmq	RABBITMQ_URL (amqps:// for TLS), RABBITMQ_MGMT_URL — vhost-aware via the HTTP Management API.
ActiveMQ / Amazon MQ	activemq	ACTIVEMQ_JOLOKIA_URL, ACTIVEMQ_USER, ACTIVEMQ_PASSWORD — Classic broker over Jolokia with auto-discovery.
Azure Service Bus	azservicebus	AZURE_SERVICEBUS_CONNECTION_STRING, or AZURE_SERVICEBUS_NAMESPACE with Managed Identity.
Google Pub/Sub	googlepubsub	GOOGLEPUBSUB_PROJECT_ID, GOOGLEPUBSUB_CREDENTIALS_FILE (empty = Application Default Credentials).
MuleSoft Anypoint MQ	anypointmq	ANYPOINT_CLIENT_ID/SECRET, ANYPOINT_ORG_ID, ANYPOINT_ENV_ID, ANYPOINT_REGION. OAuth2 client-credentials — no Mule runtime license needed.

Storage provider (optional)

Set STORAGE_PROVIDER_TYPE to enable poison-object remediation (empty disables it). The identity Triage runs as needs list + read + write + delete on the bucket — read-only roles are not sufficient, and the grant must be on the bucket itself.

The s3 type targets any S3-wire-compatible store, not just AWS. Point S3_ENDPOINT at Cloudflare R2, Wasabi, Backblaze B2, DigitalOcean Spaces, or a self-hosted MinIO / Ceph gateway and supply S3_ACCESS_KEY_ID / S3_SECRET_ACCESS_KEY; Triage recognises the endpoint and labels the store by vendor across the Storage page, instance-info dialog, and Architecture view. Leave the endpoint empty for byte-identical AWS behaviour.

Type	Key variables	Required IAM
s3	S3_BUCKET + AWS credentials (role/instance profile or keys). For an S3-compatible store (Cloudflare R2, Wasabi, Backblaze B2, DigitalOcean Spaces, MinIO): add S3_ENDPOINT, S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEY, and S3_USE_PATH_STYLE for gateways that need it.	s3:ListBucket on the bucket, plus GetObject / PutObject / DeleteObject on its objects. On S3-compatible stores, the equivalent list/read/write/delete grant on the bucket.
gcs	GCS_BUCKET, GCS_CREDENTIALS_FILE (empty = ADC / Workload Identity)	roles/storage.objectAdmin (or objectUser) at the bucket level — objectViewer is not enough
azureblob	AZURE_STORAGE_CONTAINER + connection string or account	Storage Blob Data Contributor on the container

Authentication

Configure one strategy. All of them mint the same session tokens once a user is signed in.

Strategy	Variables	Notes
Local users	USERS (bcrypt hashes), PASSWORD_REGEX	Built-in — no external dependency.
GitHub OAuth	OAUTH_GITHUB_CLIENT_ID / _SECRET, BASE_URL	Social/enterprise GitHub sign-in.
Google OAuth	OAUTH_GOOGLE_CLIENT_ID / _SECRET, BASE_URL	Google Workspace sign-in.
OIDC (Okta, Dex, …)	OIDC_ISSUER_URL, OIDC_CLIENT_ID / _SECRET, BASE_URL	Any standards-compliant IdP.

OIDC role mapping (optional)

By default a new OIDC user is created as viewer and promoted by hand. Set OIDC_ROLE_CLAIM (the claim to read — e.g. groups) and OIDC_ROLE_MAP (comma-separated claimValue=triageRole pairs, roles being admin/operator/developer/viewer) to derive the role from the ID token. It’s re-evaluated on every login, so your IdP stays authoritative.

OIDC_ROLE_CLAIM=groups
OIDC_ROLE_MAP=platform-admins=admin,platform-ops=operator,app-engineers=developer,triage-users=viewer

Users, roles & API tokens

Four roles

Viewer inspects but never touches. Operator edits, replays, and applies transforms and rules. Admin adds user management, provider config, tokens, and the Architecture export.

Developer is not a rung on that ladder — it authors transform scripts and nothing else, so you can let an application engineer save a script without handing over purge, users, tokens, and replay rules. Message payloads are outside it: peeks, the replay log, and traces are all closed to a developer, which is also why Triage shows it only the Transforms page.

Managing users

Admins create and disable users and assign roles from the admin console. Disabling a user or editing their account revokes their live sessions immediately — no wait for token expiry.

API tokens

For CI and automation, issue a scoped, revocable API token under Settings → API Tokens rather than hand-crafting a JWT. Service accounts authenticate this way.

Multi-instance federation

One instance manages one provider. To run a mixed-provider estate from a unified view, deploy several instances and connect them over mTLS federation — enabling cross-provider replay routing, end-to-end traces across instances, and combined dashboard/metrics.

Shared Postgres & Redis

Instances can share one Postgres and one Redis. Give each a distinct PROVIDER_NAME and its own port; audit rows, retention scrubbing, and payload encryption are all scoped to the provider names an instance owns, so one instance never touches a co-tenant’s data in the shared database. A shared-database group is also one key domain for envelope encryption — every instance in it must carry the same key material.

Peer health is observable

The dashboard shows peer status, and /metrics exposes triage_federation_peer_up, _peer_latency_seconds, and _peer_cert_expiry_seconds per peer — so an expiring federation certificate surfaces before it breaks the mesh.

Data retention & payload encryption

The only sensitive payload Triage stores at rest is the before/after body snapshot on each replay and storage remediation. You choose how it’s protected. These controls compose — you can encrypt bodies and time-box them.

default

Store bodies, keep the diff

By default Triage retains before/after body snapshots indefinitely so the replay diff always works. Protect them with envelope encryption (below) if the database is in scope for a breach.

SNAPSHOT_CAPTURE=false

Never persist payloads

Bodies are dropped at insert time and never touch the database. You keep the full "who remediated what, when, on which queue" audit trail and give up only the before/after diff.

SNAPSHOT_RETENTION_DAYS=N

Scrub bodies after N days

An hourly, leader-gated job blanks body columns older than N days while keeping the metadata row intact — capping breach blast radius to the last N days. Scoped to the provider names each instance owns.

envelope encryption

Encrypt against a live-DB breach

Stored bodies are written as AES-256-GCM ciphertext. The data key lives only in memory, wrapped at rest by a raw master key or a cloud KMS (AWS / GCP / Azure). The only option that keeps the diff and defends a live-DB breach. Mint key material with the built-in keygen command.

At-rest / RDS / volume encryption protects a stolen disk or backup, but gives nothing against a live DB breach — Postgres decrypts transparently for anyone who can connect. Application-layer envelope encryption is the only option that both keeps the diff working (bodies decrypt at render time) and defends a live-DB breach. The data key is unwrapped once at startup and pinned in memory, so there’s no KMS call on the per-message path. Cloud KMS requires the kms license feature; the raw master-key provider is always available. Mint key material with the keygen command.

Alerting

Queue-depth thresholds

Operators set per-queue depth thresholds with business-context labels. Triage fires when a queue breaches — and again on recovery — to Slack, PagerDuty, email, webhooks, AWS SNS, or OpenTelemetry. Depth-history charts and an alert event timeline live on each queue.

Replay-failure alerts

Automated replay rules run unattended, so a silent failure is the exact mode Triage exists to eliminate. Rule runs alert on exceptions only — one notification per run, with actionable per-failure detail (which message, which destination, which stage, what error) — through the same sink transports. Successes stay silent.

Observability

Prometheus metrics

A built-in /metrics endpoint publishes triage_queue_depth, triage_replays_total, triage_alert_firing, plus federation peer gauges and Go runtime metrics. Point any Prometheus-compatible scraper at it.

OpenTelemetry

Set OTEL_EXPORTER_OTLP_ENDPOINT to ship replay-path spans to Datadog, Grafana, or Honeycomb. Triage injects the W3C traceparent header so upstream traces continue through it.

Health & logs

A /health diagnostics endpoint and structured JSON logs. An admin-only /api/v1/export/archimate endpoint emits a live ArchiMate 3.1/3.2 model of the instance and its peers.

MCP server — AI agent access

Triage mounts a read-only Model Context Protocol server at /api/v1/mcp, so an SRE can point any MCP client at an instance and ask why a queue is filling up. It is enabled by default and needs no configuration: there is no environment variable to set and no separate process to run. When federation is configured, the surface reaches past the local instance: any token can describe the whole mesh with get_instance_info, and an operator or admin token adds live peer-workload comparison — to find which instance a backlog is actually on and confirm a cross-instance replay destination — all from the one endpoint it is pointed at. Where object storage is configured and licensed, an operator or admin token also sees the archive: which buckets exist and are reachable, what has been written to them, and the x-triage-* lineage on each object that links it back to the message trace that produced it.

How clients connect

Stateless streamable HTTP at https://<triage-host>/api/v1/mcp, authenticated with a normal bearer token — issue a scoped API token to the person or agent that needs it. Because there is no session state to pin, replicas stay interchangeable behind a load balancer with no sticky routing.

Access is the caller's own

The endpoint sits inside the authenticated group and every tool runs with the claims of the token that invoked it — a viewer token yields a strictly read-only agent. Every account role is accepted; architecture share links are rejected outright. Role also scopes the surface itself: a viewer sees ten tools, while an operator or admin token adds the two live cross-peer federation tools and, where storage is configured and licensed, the four storage tools — matching the roles the /storage REST routes already require. Tool calls are audited exactly as the equivalent REST calls are.

Read-only by construction

Replay, purge, and delete are absent, not gated — remediation stays human-authored. There is no tool to peek at messages still sitting in a queue either: peeking is a real receive on most brokers, so an agent looping it would redrive healthy messages into the DLQ. And no tool returns a message body — not queued messages, not the before/after snapshots of replayed ones. The surface is metadata only, so a payload cannot leave through the agent even for a message a human could open in the UI; list_replays still reports whether a body was modified, without revealing either version. The same holds on the storage side: object bodies are never transferred, and neither the sender's own message attributes nor an object's user-defined metadata are returned — only the x-triage-* lineage Triage itself writes. Paginated tools clamp to 100 results to protect the model's context window.

Tool surface

Each tool dispatches into the same handler as the REST endpoint it mirrors, so RBAC, audit, error mapping, and provider-capability fallbacks are identical by construction — a tool cannot drift from its endpoint.

list_queues	Queues visible to the instance with depth, DLQ status, and per-queue capabilities. Filter by kind or name.
get_queue_config	Broker-level config for one queue: redrive policy, retention, visibility timeout, TTLs — why messages are arriving.
get_provider_info	Which broker this instance is attached to and which operations it supports, so a suggested remediation is possible here.
get_instance_info	Identify this instance — id, version, broker and enabled features — and list its federation peers with each peer's web URL, health, broker, version, and group. Served from cached health polling, so it makes no live call-out.
get_stats	Instance-wide totals: replay counts and success rate over a rolling window, live depth totals, transform and alert-rule counts.
get_depth_history	Time-series depth readings for one queue — separates a queue filling for days from one that spiked ten minutes ago.
get_alert_status	Current state of every alert rule: alerting, ok, or never evaluated.
list_alert_events	Alert state transitions, newest first — correlate queue behaviour with a deploy or incident window.
list_replays	Past replay attempts with filters — whether these messages have already been retried, by whom, and whether it worked. Requires the audit license feature.
get_trace	One message's full journey by triage message id or root id: every hop, replay attempt, and transform, including federated instances.
list_federation_peers	Live workload on every instance in the federation group, self first — DLQ counts, total depth, and firing alerts per instance. peerCount and unknownPeers reveal whether the group is fully connected; sharedTokenStore says whether the calling credential is accepted on each peer or needs its own. Requires an operator or admin token.
get_peer_queues	Queues on one federated peer that can receive a replay from here, and whether each is a DLQ. Reaches the peer live over mTLS, so it also confirms a cross-instance replay destination exists before an operator attempts it. Requires an operator or admin token.
list_storage_buckets	Object storage buckets this instance archives failed messages to — backend type, vendor label, endpoint, and reachability, so a misconfigured bucket is never mistaken for an empty one. Call it first: the other storage tools take a bucket name whenever more than one is configured. Requires an operator or admin token.
list_storage_objects	Archived objects by prefix, key search, or extension — where to look when a queue is empty but messages are unaccounted for. Cursor-paged: a non-empty nextCursor means more remain, however few the window matched. Requires an operator or admin token.
get_storage_object_metadata	One object's size, content type, and storage class, plus the x-triage-* lineage archived with it — which yields a root id to hand straight to get_trace. Object bodies, sender message attributes, and user-defined object metadata are all withheld. Requires an operator or admin token.
list_storage_events	Edits, republishes, moves, deletes, and restores against archived objects, newest first, with actor and timestamp — the storage counterpart to list_replays. A delete records a trashKey, so the object was retained and can be restored. Requires an operator or admin token.

Withholding the storage tools — `MCP_STORAGE_TOOLS=false`

Every other field the surface returns is either operator-defined naming — queue, bucket, rule, and transform names — or withheld outright. Object keys are the exception: your own pipeline writes them per message, so on some estates they are opaque uuids and on others they carry customer identifiers. Redaction is not an option, since keys are how every storage tool is addressed and an operator acting on an agent's finding needs the real one. So if those keys must not reach a third-party model, set MCP_STORAGE_TOOLS=false to withhold that group of four entirely. Storage itself, the UI, the REST API, and the rest of the MCP surface are unaffected.

Licensing

A license is an offline, ECDSA-signed key scoped by queue count, feature set, and expiry date. Feature flags gate capabilities such as federation, automated replay rules, alerting, AI diagnosis, and cloud KMS.

•Offline validation. The key is verified at startup against a public key compiled into the binary — no phone-home, no dependency on any SaaS being reachable.
•Activate by setting LICENSE_KEY in the environment. Verify before you deploy with liccheck (below) to confirm the customer, expiry, queue count, and features.
•Expiry is non-disruptive. An expired license logs at ERROR and the server keeps running; renewals are arranged ahead of the expiry date rather than enforced by a hard cut-off.

Operator CLI

The server binary doubles as a small operator CLI — the tools ship inside the image you already run, so there is no Go toolchain, source tree, or extra artifact to distribute. Pass a subcommand as the first argument; with none, the binary runs the server. Help follows the cobra/kubectl convention: --help lists commands, and <command> --help shows that command’s flags.

triage --help

$ docker run --rm <triage-image> --help
Triage — dead-letter triage and replay for message queues.
 
Usage:
  triage [command]
  triage            run the server (no command)
 
Available Commands:
  keygen            Mint snapshot-encryption key material (raw master key or cloud KMS)
  federation-cert   Mint federation mTLS certificates (mesh CA and per-instance leaves)
  passwd            Hash a password with bcrypt for the USERS seeding env var
  liccheck          Verify a license key and print what it grants
  version           Print version, commit, and build time
  server            Run the Triage server (default when no command)
  help              Help about any command
 
Flags:
  -h, --help   help for triage
 
Use "triage [command] --help" for more information about a command.

One binary, every subcommand

Pass a subcommand to the same artifact you already run — the published image or the standalone binary. No extra tool to fetch, build, or sign:

docker run --rm <triage-image> <command> [flags]
./triage <command> [flags]              # standalone binary

Customer-safe by design. keygen never emits the plaintext data key, and liccheck only verifies against the embedded public key. The license signing tool (licensegen) holds the private key and is deliberately not in the image.

keygen

— Mint snapshot-encryption key material

Generates the key material for snapshot-body encryption and prints the environment variables to set. The plaintext data key is created in memory, wrapped, and wiped — only the wrapped DEK (and, on the raw path, the master key) is ever printed, so it is safe for you to run: Triage never sees your keys. Pairs with the payload-encryption controls above.

Cloud-KMS wrapping needs the kms license feature and the matching cloud credentials mounted into the container; the raw master-key path is always available.

Flag	Default	Purpose
-key-id=<1-255>	1	Data key id stamped on encrypted rows. Use a new id to rotate to a fresh DEK.
-master=<base64>	generated	Raw path: reuse an existing master key instead of generating one (DEK rotation).
-kms-provider=<aws\|gcp\|azure>	raw key	Wrap the DEK with a cloud KMS instead of a raw master key.
-kms-key-id=<id>	—	KMS key: ARN/alias (AWS), cryptoKey resource (GCP), or key URL (Azure).
-kms-region=<region>	default chain	AWS only — region to reach KMS when it is not implied by the ARN.

Raw master key — the default. Store the master key in a secrets manager; set both variables on the instance.

triage keygen

$ docker run --rm <triage-image> keygen
# Triage snapshot-body encryption key material (raw master key)
# key id 1
# store the master key in your secrets manager; do NOT commit it
SNAPSHOT_ENCRYPTION_MASTER_KEY=3Qp8x1oR…base64…c0=
SNAPSHOT_ENCRYPTION_WRAPPED_DEK=AAAAAQ…base64…

Cloud KMS — no master key is emitted; the DEK is wrapped by your cloud key.

triage keygen — AWS KMS

$ docker run --rm <triage-image> keygen \
$     -kms-provider=aws -kms-key-id=arn:aws:kms:eu-west-1:…:key/1234
# Triage snapshot-body encryption key material (aws-kms)
# key id 1 — the DEK is wrapped by the cloud key; no master key is emitted
SNAPSHOT_ENCRYPTION_KEY_PROVIDER=aws
SNAPSHOT_ENCRYPTION_KMS_KEY_ID=arn:aws:kms:eu-west-1:…:key/1234
SNAPSHOT_ENCRYPTION_WRAPPED_DEK=AAAAAQ…base64…

federation-cert

— Mint federation mTLS certificates

Federation between instances is authenticated with mTLS: each ingest listener requires and verifies a client certificate signed by a shared CA (INGEST_TLS_CERT / INGEST_TLS_KEY / INGEST_TLS_CA). This command mints that material with Go’s crypto/x509 — no openssl on the host — and ships in the image. Keys are ECDSA P-256, matching the TLS 1.3 floor the listener enforces.

Action	What it does
init-ca	Mint the mesh CA. Run once; keep ca.key offline — it never goes on an instance.
issue	Sign a per-instance leaf from the CA, with that instance’s hostname(s) as SANs.
renew	Re-sign a leaf before expiry, reusing its SANs. Rekeys by default.
rotate-ca	Mint a successor CA + old/new trust bundle for a zero-downtime cutover.

Flag	Default	Purpose
-ca=<dir>	certs	Directory holding ca.key / ca.crt to sign from.
-out=<dir>	same as -ca	Where to write output (rotate-ca defaults to certs-next; must differ from -ca).
-cn=<name>	required	Instance name / Common Name — required for issue and renew.
-san=<list>	CN fallback	Comma-separated SANs, each optionally prefixed dns: or ip:.
-days=<n>	825 leaf · 3650 CA	Certificate validity in days.
-keep-key	false	renew only — reuse the existing private key instead of rekeying.
-force	false	Overwrite existing key/cert files (init-ca, issue, rotate-ca).

Stand up a mesh — mint the CA once, then one leaf per instance. Each issue prints the env vars to set on that instance; ca.crt is shared by all of them.

triage federation-cert init-ca / issue

$ docker run --rm -v $PWD/certs:/certs <triage-image> \
$     federation-cert init-ca -out /certs
# Federation mesh CA written to /certs/
#   /certs/ca.key   (private — keep offline, never distribute)
#   /certs/ca.crt   (public  — distribute to every instance as INGEST_TLS_CA)
# CA valid until 2036-07-18
# Next: triage federation-cert issue -cn <instance> -san dns:<host>
 
$ docker run --rm -v $PWD/certs:/certs <triage-image> \
$     federation-cert issue -ca /certs -cn instance-a -san dns:a.mesh.internal,ip:10.0.0.7
# Leaf certificate for "instance-a" written to /certs/ (valid until 2028-10-21)
#   DNS: a.mesh.internal
#   IP:  10.0.0.7
# Set on this instance (copy ca.crt alongside; it is shared by every instance):
INGEST_TLS_CERT=/certs/instance-a.crt
INGEST_TLS_KEY=/certs/instance-a.key
INGEST_TLS_CA=/certs/ca.crt

Renew a leaf before it expires — recovers the SANs from the current cert and rekeys, then tells you to restart. Triage surfaces expiry ahead of time via the peer_health alert rule and the /metrics cert-expiry gauges.

triage federation-cert renew

$ docker run --rm -v $PWD/certs:/certs <triage-image> \
$     federation-cert renew -ca /certs -cn instance-a
# Renewed leaf for "instance-a" (rekeyed), valid until 2028-10-21
#   DNS: a.mesh.internal
#   IP:  10.0.0.7
INGEST_TLS_CERT=/certs/instance-a.crt
INGEST_TLS_KEY=/certs/instance-a.key
INGEST_TLS_CA=/certs/ca.crt
# Restart this instance to pick up the renewed certificate.

Rotate the CA with zero downtime — mints a successor plus an old+new trust bundle and prints the cutover runbook.

triage federation-cert rotate-ca

$ docker run --rm -v $PWD/certs:/certs -v $PWD/certs-next:/certs-next \
$     <triage-image> federation-cert rotate-ca -ca /certs -out /certs-next
# Successor CA written to /certs-next/ (valid until 2036-07-18)
#   /certs-next/ca.key        (new CA private key — keep offline)
#   /certs-next/ca.crt        (new CA cert)
#   /certs-next/ca-bundle.crt (old + new — trust both during cutover)
#
# Zero-downtime cutover:
#   1. Set INGEST_TLS_CA=/certs-next/ca-bundle.crt on every instance and restart.
#   2. Re-issue each leaf from the new CA, one instance at a time:
#        triage federation-cert renew -ca /certs-next -out <leaf-dir> -cn <instance>
#   3. Once every leaf is re-issued, set INGEST_TLS_CA=/certs-next/ca.crt and restart.
#   4. Retire the old CA key.

passwd

— Hash a password for the USERS seed

Bcrypt-hashes a password so it can go into the USERS env var (username:hash[:role]) that seeds local accounts. Stateless — it never touches the database — and uses the same bcrypt cost the server applies, so seeded and self-service passwords are indistinguishable. It prompts with no echo and confirms; pass --stdin to pipe one for scripting. Enforces the same PASSWORD_REGEX policy as the server.

Flag	Default	Purpose
-role=<admin\|operator\|developer\|viewer>	admin	Role baked into the USERS entry (only used when a username is given).
-cost=<n>	10	bcrypt cost.
-stdin	false	Read the password from stdin (one line) instead of prompting — for scripting.
-no-policy	false	Skip the configurable PASSWORD_REGEX check (the hard complexity floor still applies).

With a username it prints a ready-to-paste USERS entry (the Password: / Confirm: prompts and the note go to stderr).

triage passwd

$ docker run --rm -it <triage-image> passwd -role=operator alice
Password:
Confirm:
USERS=alice:$2a$10$Nf1c…bcrypt…q3W:operator
note: single-quote the value so the '$' in the hash survives your shell / compose file.

liccheck

— Verify a license key

Validates a license against the embedded public key and prints what it grants — handy to confirm a key before deploying. Verify-only: it holds no signing key and cannot mint or alter a license. Takes the license key as its one positional argument; exits non-zero with invalid license: … on a bad signature, wrong format, or expiry, so it is safe in a preflight check.

A valid key prints OK plus the grant; an expired-but-signed key prints EXPIRED and the same detail.

triage liccheck

$ docker run --rm <triage-image> liccheck TRIAGE-eyJ…signed-key…
OK: customer=acme expires=2027-01-01 maxQueues=50 features=[replay storage federation alerting kms]

A bad key fails verification and the process exits non-zero.

triage liccheck — rejected

$ docker run --rm <triage-image> liccheck bogus-key
invalid license: verify signature: bad key format
$ echo $?
1

version

— Print build metadata

Reports the version, git commit, and build time stamped into the binary, plus the Go toolchain. Available both as a subcommand and as the top-level --version flag.

triage version

$ docker run --rm <triage-image> version
triage version 1.2.0
  commit:  df363d4
  built:   2026-07-11T00:00:00Z
  go:      go1.26.2 (linux/amd64)
  Copyright (c) jcrstack.com 2026

Operating notes

Upgrades

Pull a newer image tag and restart. Keep the same JWT_SECRET and live sessions survive. Pin a release tag in production rather than tracking latest. Schema migrations run on start-up.

Backup

Postgres is the source of truth — the audit log, rules, users, and transforms all live there. Back it up as you would any production database. Redis holds only ephemeral depth series and coordination state and can be rebuilt.

Storage-object retention

When Triage deletes or moves a storage object, recoverability is your bucket’s job — enable S3 versioning, GCS soft delete, or Azure Blob soft delete on the bucket so a mistaken delete can be restored.

Key custody

If you use raw-master-key encryption, keep the master key in a secrets manager. Losing it — or disabling the cloud KMS key — makes existing encrypted bodies unrecoverable. Every instance in a shared-DB group needs the same key material.

Onboarding your team?

The user guide walks operators through inspecting, editing, replaying, transforming, and automating failed messages.

User guide →