opsPlugin

import { opsPlugin } from '@routecraft/routecraft'

Serves the operational surface: liveness, readiness, and operational health over HTTP. Health is its first capability; the /ops namespace is reserved for the action surface (taking a route offline, resetting a breaker) and answers 404 until that ships.

Most of what it reports needs no application code. Route lifecycle, circuit-breaker position, and the context's own serving state are all derived from events the framework already emits. Indicators are the one thing an app adds by hand, for dependencies the framework cannot see.

ops is a first-class core config key, so the common path is defineConfig({ ops: {...} }) rather than plugins: [opsPlugin(...)]. The factory is exported for programmatic composition.

import { defineConfig } from '@routecraft/routecraft'

export const craftConfig = defineConfig({
  servers: { default: { host: '0.0.0.0', port: 8080 } },
  ops: {},
})

The surface is a mount on a named server, not a listener of its own. An internal-only ops port is a second named server this mount points at:

export const craftConfig = defineConfig({
  servers: {
    default: { host: '0.0.0.0', port: 8080 },
    internal: { host: '0.0.0.0', port: 9090 },
  },
  http: {},                  // the app, on "default"
  ops: { server: 'internal' },
})

Three signals, not one health check

The distinction that matters is not what a check inspects, it is what acting on the answer does.

PathQuestionConsumerEffect of failureSees dependencies
/health/liveShould I restart this?Docker, KubernetesProcess killed and restartedNever
/health/readyShould I route traffic here?Reverse proxies, KubernetesInstance leaves the poolOnly instance-local ones
/healthIs everything working?Uptime monitors, humansA human is pagedAlways

Liveness excluding dependencies is the important one: restarting cannot fix someone else's outage, and a dependency blip pointed at a restart trigger would restart every replica at once. No configuration can make a component reach /health/live.

Readiness has two parts, and the first is the one people forget. It answers "can this instance serve right now", and the usual reasons it cannot are its own: still starting, or shutting down and draining. That is what makes readiness worth having at a single replica, because it is what makes a deploy zero-downtime.

The second part is which failures may also refuse traffic, which is decided by failure domain (see below).

What decides a status

SituationStatusWhy
Source gave up producingdownThe route is not running, so nothing it exists to serve is being served
Circuit breaker open or half-opendegradedLive, but holding calls off, and expected to close again on its own
Route deliberately taken offlinedegradedDeliberate reduced capability, visible without paging
Route disabled by its .enabled() predicateinactiveA capability whose credentials were never supplied is a configuration state the operator chose, not an incident. Listed with its reason, excluded from aggregation
Route running normallyup
One-shot finished, or route stopped cleanlyinactiveExcluded from aggregation entirely
An exchange failedno changeA refused caller, a validation error, or a business rule saying no is a healthy route behaving correctly

Disabled is not offline, and neither is failed. The three are separate readings of separate situations: failed is a route that should be running and is not, offline is a running deployment having capability taken away from it, and disabled is capability that was never configured. Collapsing any pair would either page on a dormant route or stop paging on a derotated one. A disabled route reports its reason in details.reason, so an operator reading /ops sees

"mail-inbound": {
  "status": "inactive",
  "domain": "deployment",
  "details": {
    "lifecycle": "disabled",
    "reason": "MAIL_USER and MAIL_APP_PASSWORD are not set"
  }
}

and knows immediately that nothing is broken. Overall health is unaffected: a deliberate configuration state is not an open circuit.

That last row is the rule the surface is built around. Exchange errors are route issues, not health issues. A failing exchange never changes any status, whatever the streak, because many errors are entirely expected and paging on them turns every scope gap and every malformed request into an incident. Repeated failure reaches health through exactly two doors, both deliberate: a circuit breaker, which knows which errors it counts and actually stops the route serving, or the app catching the error and pushing an indicator down itself. The consecutive-failure count still appears in details as context for a human; it is never an input to status.

Failure domains

Every component declares how widely its failure is felt.

  • instance: this process or replica specifically. Peers may be fine, so taking this instance out of the pool can genuinely help.
  • deployment (the default): every replica alike. Revoked credentials, a dead upstream, bad config baked into the image.

Readiness carries only instance-domain components. A deployment-wide failure must page but must never move traffic: instance one derotates, traffic shifts to instance two, which fails identically and derotates too, until the pool is empty and a degraded service has become a total outage.

Routes are always deployment. Indicators declare their own domain, and default to deployment because the two mistakes are not symmetric: wrongly admitting a component to readiness can empty the pool, while wrongly excluding one only misses a derotation that the operational signal still catches.

Options

OptionTypeDefaultDescription
serverstring'default'Named server to mount the health paths on.
authHttpAuth | falseinherits servers.<name>.authValidator for the details gate and for the management tiers. Never a wall over /health: those paths answer every probe without a credential whatever this says.
health.details'always' | 'when-authenticated' | 'never''when-authenticated'Exposure of per-component details.
tiers.introspectionfalse | true | stringfalseAdmission to GET /ops/routes and GET /ops/routes/{id}. See Management API.
tiers.dispatchfalse | true | stringfalseAdmission to POST /ops/routes/{id}/exchanges.
tiers.eventsfalse | true | stringfalseAdmission to GET /ops/events, the live event tail. See Tailing events.
indicatorsIndicator[][]Indicators to register. See defineIndicator.

The mount claims the /health and /ops prefixes on its server, so a collision with another surface fails at start rather than being decided by dispatch order.

Do not bind that server to 127.0.0.1 and expect probes to reach it. A Kubernetes httpGet probe comes from the kubelet against the pod IP and a reverse-proxy check comes from the proxy, so neither reaches loopback. Only a Docker HEALTHCHECK, which runs inside the container, does. Bind 0.0.0.0 and keep the port private at the network layer.

Endpoints

GET /health

The operational aggregate: every component, whatever its domain. 200 when nothing is down, 503 otherwise.

{
  "view": "all",
  "status": "up",
  "context": { "status": "up", "domain": "instance", "details": { "state": "started" } },
  "routes": {
    "leave-approval": { "status": "up", "domain": "deployment", "details": { "lifecycle": "running" } }
  },
  "indicators": {
    "mail": { "status": "up", "domain": "deployment", "ageMs": 42133 }
  }
}

Aggregation runs over the non-inactive components of the requested view:

  • status is down if any component is down, else degraded if any is degraded, else up.
  • The status code is 503 when status is down, and 200 otherwise.
  • A view whose components are all inactive is up, 200: nothing has said this instance cannot serve.
  • view names which components the report covers, so a consumer holding one can tell the operational aggregate from the routing signal without remembering which path produced it.

There is deliberately no separate ready flag. It would be exactly status !== "down" on whichever view produced the report, and a derivable field called ready on the operational aggregate invites the one mistake this design exists to prevent: routing traffic on the deployment-wide signal. Read status, and read it from /health/ready when the answer decides where traffic goes.

No response body ever carries an error message. details holds structural facts only: lifecycle, circuit position, counters, staleness. What went wrong and why belongs to logs and telemetry, which are not readable by everyone who can reach this port.

GET /health/live

{ "status": "up", "uptime": 812.4 }

200 while the process serves, including during shutdown. No component can reach it and no configuration can change that.

GET /health/ready

The same shape as /health, with view set to readiness and the components filtered to the context component plus any instance-domain component.

During boot the listener has not bound yet, so probes get connection refused, which every orchestrator reads as not-ready. At context:stopping readiness flips to 503 immediately while the listener stays up through the drain, which is the window that lets a load balancer stop sending work while in-flight exchanges finish.

GET /health/routes/<id> and GET /health/indicators/<name>

One component with its own status code: 503 when it is down, 200 otherwise, 404 for an unknown name.

Management API

Four resources on the same mount, under /ops, for driving a running instance: list its routes, describe one, dispatch work to one, and watch what it is doing. craft exec and the route commands in craft ops are clients of exactly this API, and so is anything you point at it with curl. The rest of that family (health, ready, indicators) reads the /health resources above instead, which is the group-by-task split the CLI reference explains.

Method and pathTierWhat
GET /ops/routesintrospectionList routes, filtered and paginated
GET /ops/routes/{id}introspectionDescribe one route
POST /ops/routes/{id}/exchangesdispatchDispatch work and return the outcome
GET /ops/eventseventsTail the context event bus as Server-Sent Events
GET /ops/{resource} and GET /ops/{resource}/{segment...}introspectionA resource another package contributed

Dispatching creates an exchange, so the exchange is a sub-resource of the route it runs on rather than a verb of its own.

Exposure tiers

Every tier is disabled unless you name it, so an app that configures nothing serves health and answers 404 on every /ops path. Each tier takes one of three values:

ValueMeaning
false, or unsetDisabled. Its paths answer 404
trueOpen. No credential is required for this tier
'some:scope'The caller's principal must carry that scope
ops: {
  auth: jwt({ secret: process.env.OPS_SECRET!, issuer, audience }),
  tiers: {
    introspection: 'ops:introspection',
    dispatch: 'ops:dispatch',
    events: 'ops:events',
  },
}

ops:introspection, ops:dispatch and ops:events are the names this project documents, not defaults: every tier defaults to false, and the value is any string your tokens carry. ops:operations is reserved for the operations tier when it ships. Dispatch is always its own scope and never rides along with introspection: a dashboard token that reads the route inventory must not also be able to run work.

The mount's auth decides who the caller is (its own validator, else the named server's, per mount auth inheritance); the tier value decides what that identity must carry. Because tiers are path-and-method rules, an operator can enforce the same split at a proxy that the framework enforces internally.

A disabled tier answers 404, not 403. An unconfigured instance should disclose neither its route inventory nor that a management surface exists at all, and 404 is what a proxy in front of it would answer anyway. A tier naming a scope where the mount declares no wall fails the boot rather than guessing: admitting everyone and refusing everyone are opposites, and neither is what was written.

The refusal on a missing scope is 403 with insufficient_scope, and it carries an RFC 6750 WWW-Authenticate challenge only for a bearer caller. An api-key client is told which scope it lacks in the body, without being pointed at a bearer ceremony it cannot perform.

Warning

`true` is an open door

A tier set to true needs no credential, and the ops surface mounts on the default server unless told otherwise, which on a typical app is the public listener. dispatch: true on a publicly bound server lets anyone who can reach the port run any dispatchable route. events: true is the quieter version of the same hazard: the tail carries failure events with their messages and stack traces, so filesystem paths, internal hostnames and upstream response text reach anyone who can reach the port. Use either on a listener the public cannot reach, or give the tier a scope.

Tailing events

GET /ops/events streams the context event bus as Server-Sent Events: every event the app emits, as it emits it. It is the live counterpart to the route listing, which is why it is its own tier rather than a corner of introspection. A route inventory describes an app's shape; the tail carries exchange failures, auth decisions and health transitions as they happen, and an operator who wants one is not thereby asking for the other.

curl -N -H "authorization: Bearer $TOKEN" 'http://127.0.0.1:8080/ops/events'
: open

event: route:exchange:failed
data: {"event":"route:exchange:failed","ts":"2026-08-27T21:04:11.522Z","contextId":"...","details":{"routeId":"orders","error":{"name":"Error","message":"upstream refused"}}}

: keep-alive

The bus event name rides the SSE event field, so an EventSource can subscribe to one kind, and repeats inside data so a reader consuming the body with fetch gets it without parsing fields. Payloads are rendered defensively: errors keep their name, message and stack, and an exchange, route or context collapses to its identifier rather than dragging its object graph onto the wire.

The tail omits _snapshot payloads. Several events, the agent events among them, carry request and response bodies under that key; the telemetry sink already gates them behind captureSnapshots because they are payload data rather than metadata. Opening a tail is not that opt-in, so a watcher of route:agent:tool:result sees the event without _snapshot.output. Read payloads through telemetry, which was asked for them.

Two frames are the tail's own and not bus events. : keep-alive is an SSE comment sent when the app has been quiet, so an intermediary does not reap a healthy connection. ops:events:dropped reports how many events a reader missed: the bus emits on the route's own hot path and can never wait for a network, so the tail buffers a bounded backlog and drops the oldest past that, then says so rather than leaving a silent gap. The backlog is bounded twice, by count (256 events) and by size (1 MiB of rendered frames), because one event carrying a large error or body is worth many small ones.

The tail closes when the context begins stopping, so a shutdown is not held open by a watcher.

Listing routes

curl -H "authorization: Bearer $TOKEN" 'http://127.0.0.1:8080/ops/routes'
{
  "items": [
    {
      "id": "greet",
      "dispatchable": true,
      "enabled": true,
      "sources": ["direct"],
      "requiresPrincipal": false,
      "title": "Greeter",
      "description": "Says hello to whoever asks"
    },
    {
      "id": "nightly",
      "dispatchable": false,
      "enabled": false,
      "sources": ["cron"],
      "requiresPrincipal": false
    }
  ],
  "nextCursor": "eyJhZnRlciI6Im5pZ2h0bHkiLCJmaWx0ZXIiOiJbbnVsbCxudWxsLG51bGxdIn0"
}

dispatchable and enabled are different questions. dispatchable asks whether the route has a door at all; enabled asks whether its .enabled() predicate currently says yes. A disabled route stays in this listing, is absent from the agent tool surface, and reports disabled on the health endpoint rather than failed, so an operator can tell "switched off on purpose" from "broken".

Not every route is dispatchable. A direct() ingress makes the route id into a door; a cron-, mail- or http-sourced route runs from its own trigger and has none. That is a fact about the route, so it rides on the representation rather than splitting the collection in two, and a dispatch against a route without a door is refused distinguishably with RC5060.

An imported route names its origin. A route another instance exposes and remotes made dispatchable here is listed beside the local ones, under its local endpoint (lab:hello, or bare for the default remote), as dispatchable: true with sources: ["remote"] and a remote field naming the instance it came from; ?source=remote lists exactly those. Dispatching one through this door runs it on the remote, which is the intentional re-exposure the remotes page describes.

A route can close its door on purpose. direct({ internal: true }) keeps the in-process endpoint (other routes compose against it unchanged) and skips the capability registry, so the route shows dispatchable: false here and a dispatch refuses with RC5060 naming the declaration rather than advising a direct source the route already has. It is the opt-out for a trusting subroutine whose boundary route carries .input(), .description() and .authorize(); dispatch the boundary route instead.

Query parameters narrow the one collection:

ParameterDescription
dispatchabletrue or false. Only routes that do, or do not, have a dispatch door
idExact route id. Not a prefix
sourceRoutes carrying a source of this kind (direct, cron, mail, ...); remote for routes imported from another instance
limitPage size. A positive integer, at most 200. Defaults to 50
afterThe nextCursor from the previous page

Paging

Every collection response is an object carrying items and an optional nextCursor, never a bare array. The contract, from the first release: if nextCursor is present there is more, and a correct client follows it. A client written against "this endpoint never paginates" is a client that silently reads page one forever the day it does.

The cursors are keyset, matching the deferral store's existing idiom rather than inventing a second one. That is not consistency for its own sake: the collections this API will grow are live and mutating while an operator reads them, and offset paging over a mutating set silently skips rows and repeats others, which surfaces as missing data rather than as an error.

A cursor is valid only for the filter that produced it. Replaying one under a different filter would return a page of a different result set with nothing on the wire to signal it, so it is refused with RC5059. A malformed limit is refused for the same reason rather than clamped: a caller silently handed a bounded page cannot tell a truncated answer from a complete one.

Dispatching

curl -X POST -H "authorization: Bearer $TOKEN" \
  -H 'content-type: application/json' \
  -d '{"name":"world"}' \
  'http://127.0.0.1:8080/ops/routes/greet/exchanges'
{ "outcome": "completed", "body": "hello world" }

Three outcomes, told apart because they need different answers:

outcomeStatusMeaning
completed200The route ran and produced body
deferred202The route deferred; deferral carries the acknowledgment
dropped200A filter rejected the exchange, so there is no result body

A route that fails answers 500 carrying the framework's error code, which is what lets a client tell an .authorize() refusal from a broken step.

No bypass, and no synthetic operator principal. A dispatch runs the full pre-from chain, and the principal is the one the mount's validator minted from the presented credential. A downstream .authorize() cannot distinguish an operator dispatch from any other authenticated caller, which is the point: the management door is built from the framework's own primitives rather than beside them.

A deferral is an outcome and not an error. The response is the standard Deferred acknowledgment every other surface returns, carrying the deferral id, the single-use resume token and the expiry. There is no resume verb here: who may resume a deferred run is the application's policy, expressed with .resume({ authorize }), and a management resume door would be the framework building the bypass it deliberately refused to build.

Contributed resources

Core knows routes, exchanges and events. What an ecosystem package keeps alongside them is listable through the same door and the same tier without core learning what it is: a plugin registers a read-only resource from its apply(), and the mount serves it at GET /ops/{name} (the collection, with the query string handed to the contributor as its filter) and GET /ops/{name}/{segment...} (one item, addressed by the decoded path segments). Admission is the introspection tier's, exactly as for the route listing, and a write answers 405.

import { registerOpsResource } from '@routecraft/routecraft'

registerOpsResource(ctx, {
  name: 'widgets',
  list: async (query) => ({ items: await widgets.list(query.colour) }),
  describe: async ([id]) => widgets.get(id),
})

A collection answer is the same { items, nextCursor? } envelope as the route listing, and the contributor pages it with the same helpers the mount uses: parsePageQuery(query) reads limit and after (refusing a bad limit with RC5059), decodeCursor(after, scope) checks a cursor against the contributor's own filter scope, and takePage(sorted, scope, limit, after) takes the page and mints the next cursor, all exported from @routecraft/routecraft. A throw from either read is answered by the mount: RC5059 as a 400 carrying the message, since it is the caller's; anything else as a 500 carrying the resource name and the error's code, never its message. routes and events are the mount's own names and refused, as is a second registration of one name. Registration needs no ops plugin to be present: it is inert until a mount exists.

deferrals is registered by core's own deferral plugin, so an instance that configured deferral has it and one that did not has no such resource rather than an empty one. It answers what the instance is waiting on: one row per deferral with its id, the routeId it will re-enter, its state (waiting or settled), what it is waitingFor (resume today, and rendered rather than assumed so a second kind lands without a listing change), whether a delivery claimed is outstanding, deferredAt, expiresAt when it has a deadline, and the outcome of a settled one. Never the resume token, which is the credential, and never the stored exchange, the step state or the resume-payload schema: the projection drops them in the store, so they are not read out of the database on the way to being dropped.

Waiting is the default, because the question this is opened with is what is still owed an answer; ?state=settled asks for the history and ?state=all for both, and ?route= narrows to one route. The collection is ordered oldest first, by (deferredAt, id) rather than by id: a deferral id is {exchangeId}#{sequence} over a uuid, so id order says nothing about time. limit and after page it, and the cursor is bound to the state and route filters it was minted under. GET /ops/deferrals/{id} answers one row. A store supplied through deferral: { store } that does not implement DeferralStore.list is refused with RC5066 naming the member, rather than answering an empty page: an empty listing and a listing the store cannot produce look identical to whoever is reading them.

curl -H "authorization: Bearer $TOKEN" 'http://127.0.0.1:8080/ops/deferrals?route=payout'
{
  "items": [
    { "id": "0f3a9c1e-4b2d-4a77-9c31-2b6a5e0d18f4#0", "routeId": "payout", "state": "waiting", "waitingFor": "resume", "claimed": false, "deferredAt": "2026-09-01T09:14:22.108Z", "expiresAt": "2026-09-04T09:14:22.108Z" }
  ]
}

craft ops deferrals reads the same thing from a terminal, with --state, --route, --limit and --after. It shows one page rather than walking the collection: an instance holds every exchange it has deferred until retention takes it, so the rendering names the cursor for the next page instead of loading an unbounded set to print the first screen of it.

agent-sessions is the other one shipped: @routecraft/ai registers it, and it lists every named agent session with the person it belongs to (owner, null for a session no principal opened), the directory it is bound to (cwd, absent unless it was opened with one), its human-readable title, its turn state (idle, running, or stale for a turn a restart cut short), the depth of its inbox, the background calls it is waiting on, whether a continuation is stored for them (deferred), its transcript length and its turn count. ?agent= narrows the collection to one agent, limit and after page it exactly as the route listing (the cursor is bound to the agent filter), and GET /ops/agent-sessions/{session} answers one summary, because a conversation is named by its id alone. A page costs one store read per session the filter considers, because ownership is applied before the page is sliced: a cursor is minted from the last row of a page and is reversible, so slicing first would hand a caller an identifier the filter exists to withhold. A store with an owner-aware listing could do better, and the contract does not have one. A context with no deferral store lists nothing, which the collection cannot tell apart from a store with no sessions.

This resource reads every session, whoever owns it. That is the deliberate operator privilege: owner gates who may list and read a conversation over a protocol surface such as acpPlugin(), and the management credential is what sees past it. It is what makes "who owns this conversation and which directory is it bound to" an answerable question.

curl -H "authorization: Bearer $TOKEN" 'http://127.0.0.1:8080/ops/agent-sessions?agent=aria'
{
  "items": [
    { "agent": "aria", "session": "feature-login", "owner": "operator", "cwd": "/work/acme", "title": "Feature login", "turn": "idle", "inbox": 0, "background": 1, "deferred": true, "messages": 14, "turns": 6, "updatedAt": "2026-09-04T21:40:11.522Z" }
  ]
}

Not built here

The API is shaped to accept these without a second grammar or a rename, and ships none of them: the operations tier (taking a route offline, resetting a breaker), reads over exchanges for the console, and GET /ops/config.

defineIndicator

An indicator is a named dependency whose health the framework cannot see for itself: a mailbox, a third-party API, a licence server.

import { defineIndicator } from '@routecraft/routecraft'

export const mailHealth = defineIndicator({
  name: 'mail',
  maxAge: 15 * 60_000,
  domain: 'deployment',
  route: 'probe-mail',
})
OptionTypeDefaultDescription
namestringrequiredUnique name. Becomes the report key and the path segment.
maxAgeDurationnoneNo report inside this window makes the indicator down (stale). Omit it for an indicator with no cadence.
domain'instance' | 'deployment''deployment'Whether readiness may carry this failure.
routestringnoneBind to a route's exchange outcomes.

Register it so a push has somewhere to land:

export const craftConfig = defineConfig({
  servers: { ops: { host: '0.0.0.0', port: 9090 } },
  ops: { server: 'ops', indicators: [mailHealth] },
})

Route-bound: no code in the route

With route set, a completed exchange on that route reports up, a failed exchange reports down, and no exchange inside maxAge goes stale. The probe route carries no health code at all:

export default craft()
  .id('probe-mail')
  .retry({ maxAttempts: 2 })
  .timeout(10_000)
  .from(cron('*/5 * * * *'))
  .enrich(mail({ folder: 'INBOX', limit: 1, markSeen: false }))
  .to(noop())

The .retry() is the hysteresis, and it is why the binding needs no threshold of its own: a probe that fails once and succeeds on the second attempt emits route:exchange:completed and never touches the indicator. Only a probe that fails every attempt flips it.

Note the placement. .retry() and .timeout() before .from() are route-scope: they sit above the pipeline in the filter chain and wrap every step. The same calls after .from() are step-scope and wrap only the step that follows, which is not what you want here, because a probe that grows a second step would quietly stop being covered.

Bind probe routes only. For a probe the exchange is the health check by construction. Binding a business route would make every expected refusal (a caller without the scope, a validation error, a business rule saying no) report the dependency down, which is the escalation this design exists to prevent.

What moves a bound indicator

A bound indicator and its route are separate components that move on different rules, so it is worth seeing both at once. A single failed exchange is enough to report the dependency down, with no threshold: on a probe the exchange is the check, and a probe that failed has answered the question.

Event on the bound routeIndicatorRouteAggregate
Exchange completesupup200
Exchange failsdownup, details.failures climbs503
Retry recovers inside the exchangeno changeup200
Exchange droppedno changeup200
Breaker open, no fallbackdowndegraded503
Breaker open, with fallbackupdegraded200
No exchange within maxAgedown, reason: 'stale'no change503
Source gave up producingno change until staledown503

Three rows earn an explanation.

A dropped exchange reports nothing. A drop is a terminal state that suppresses completed, but it is not evidence either way: the exchange may have been filtered out before the dependency was ever reached. A verdict read from it would be invented. A bound indicator on a route that only ever drops goes stale, which is the truthful answer.

An open breaker holds the indicator down. Each rejected call fails with RC5025 before the protected work runs, and that rejection is a failed exchange like any other. So the indicator stays down for the whole cooldown even though nothing is touching the dependency any more, which is the reading you want: the breaker is open precisely because the dependency was failing.

A breaker with a fallback does not. The fallback resolves the rejected exchange successfully, so it emits route:exchange:completed and reports the indicator up while the route still reads degraded. On a business route that is correct, because it is serving. On a probe it is a lie, because the fallback answers without ever reaching the dependency. Do not put a fallback on a breaker on a route you have bound an indicator to.

Source death and staleness stay independent on purpose. A dead scheduler takes the route down at once and lets the indicator go stale on its own clock, so "the probe stopped running" and "the dependency is unreachable" never get confused for one another.

Contributed by a plugin

defineIndicator is the app's API: a handle declared in source and named in ops.indicators. A plugin that watches a dependency the app never wrote a line for has no handle to give the app, so it contributes the declaration from its apply() and reports through the function the contribution returns. This plugin binds it at start(), whatever order the two were applied in, into every ops ledger on the context; a report made before that is kept and replayed at its own time when the ledger binds, so a maxAge window measures from the report. In a context with no ops plugin a report goes nowhere, for the same reason a push through an unbound handle is inert. A name that is also listed in ops.indicators is refused at start, because indicator names are the keys of the report.

import { contributeOpsIndicator } from '@routecraft/routecraft'

const report = contributeOpsIndicator(ctx, { name: 'remote.lab', domain: 'deployment' })
report({ status: 'down', details: { routes: 0 } })

remotes is the one contributor shipped: each remote reports as remote.<name>, down while its inventory cannot be read and up once it has been, with the number of imported routes in details.

Pushed by hand

For a probe checking several subsystems, or a business route flagging a dependency mid-flight, push through the handle from any .process() or .error() step:

mailHealth.down({ subsystem: 'imap' })
mailHealth.up()
mailHealth.inactive()   // deferred for maintenance: reports inactive, never pages

Details must stay structural (enums, numbers, booleans). Log the error where it happened; it does not belong on the endpoint.

Pushing through a handle no context has registered is inert on all three methods. Health instrumentation must not be able to kill the code it instruments: an exchange draining during shutdown, after teardown released the binding, would otherwise carry an error out of its .error() handler and into the route. The mistakes that a push would only report late are caught earlier instead, where the message can name the fix: an indicator naming a route no route declares fails context start, two indicators sharing a name fail the build, and an object that defineIndicator() did not produce is refused with RC5053.

Securing it

/health is read-only and may admit unauthenticated callers, which is what lets a probe reach it without a credential. The /ops management API is the opposite and fails closed: every tier is disabled until it is named.

health.details gates the per-component details maps. It withholds diagnostics and nothing else: it does not hide topology, because component names are the keys of the always-served status maps and the per-component paths answer 404 for an unknown id in every mode. An app that must not disclose its route inventory puts the ops listener somewhere the public cannot reach.

when-authenticated resolves the caller's credential through the mount's effective validator: ops.auth when set, else the server's (servers.<name>.auth). An anonymous probe still gets 200 with every status, and an operator holding an admitted credential additionally gets the details maps. With no validator in scope the gate fails closed: written explicitly it refuses the boot, because the operator asked for a gate and there is nothing to gate with, and as the unwritten default it collapses to never with a startup warning naming the ways out. Set details: 'never' to withhold regardless, or details: 'always' to serve details to every caller.

ops.auth is not a wall over /health. The ingress never authenticates on its own, and the health handler calls authenticate() only to decide whether to serve details, so those paths stay reachable without a credential, which is the only way an orchestrator can use them.

It is also the validator the management tiers identify their caller with. The two uses do not conflict, because a tier's own value decides what that identity must carry while health asks only whether there is one.

auth: false carries the server plugin's meaning unchanged: this mount declares no wall, while the inherited validator stays reachable for anything that pulls identity rather than demanding it. So the details gate still admits an authenticated caller, and a scope-gated tier, which does demand one, is refused at boot as a contradiction rather than quietly enforced through a validator the mount opted out of.

// One listener, no server-wide validator: the gate is the ops mount's own.
ops: {
  auth: { kind: 'apiKey', in: 'header', name: 'x-ops-key', keys: [process.env.OPS_KEY!] },
  health: { details: 'when-authenticated' },
}

The credential ladder

Four rungs, one decision. Every rung verifies through the same auth options this page already described; none of them makes Routecraft mint or store anything. The full recipes, each a complete craft.config.ts, live on Securing capabilities.

  1. No auth. Nothing configured, tiers set to true. craft exec needs no flags. For local work and trusted-network listeners.
  2. A static key. A { validator } comparing one secret from the environment with timingSafeStringEqual (exported from @routecraft/routecraft; never ===, which leaks timing). Scope-gate the tiers and put the scopes on the principal the validator returns.
  3. A self-signed JWT. jwt({ secret, issuer, audience }) on the server; mint tokens with your own tooling (any HS256 signer). Buys expiry and per-token scopes without an IdP.
  4. A real IdP. jwks() at the IdP's keys. Discovery tells a refused caller where to go.

Discovery on refusals (RFC 9728)

A caller refused at the management door can find out what would satisfy it. Every bearer 401 carries a resource_metadata hint in its WWW-Authenticate challenge, and the insufficient_scope 403 carries the same hint alongside the scope it already names. The hint points at the server's RFC 9728 protected-resource metadata document (/.well-known/oauth-protected-resource, path-suffixed per resource), which names the issuer from the effective validator and, for the ops paths, lists the scope-gated tiers in scopes_supported. craft exec and craft ops follow the hint automatically and fold scope, issuer, and remedy into their refusal message. A disabled tier still answers 404 and discloses nothing.

Wiring the probes

HEALTHCHECK --interval=30s --timeout=3s --start-period=5s \
  CMD bun -e "fetch('http://localhost:9090/health/live').then(r => process.exit(r.ok ? 0 : 1)).catch(() => process.exit(1))"

Docker on /health/live, orchestrator readiness on /health/ready, uptime monitor on /health. Never point a restart trigger at /health or /health/ready, or someone else's outage becomes your crash loop and an ordinary deploy becomes a restart.

Disabling

  • No surface at all: omit ops. The http plugin's own /health and /ready built-ins are unaffected and behave as before.
  • Withhold diagnostics: health.details: 'never'. Statuses are still served.
  • No management API: omit tiers, or set a tier to false. That is the default, so ops: {} already serves health alone and answers 404 on every /ops path.

There is no per-endpoint kill switch for the health paths: a mount that answers 404 on every path it claims is worse than not mounting, so an app that does not want that surface omits the key. The management tiers are the exception, and they are off by default rather than on.

Coexisting with the http built-ins

The http plugin still serves its own /health and /ready built-ins, which are constants: /health always says ok and /ready always answers 200, neither consulting lifecycle or component state. They are unchanged by this plugin.

They coexist on one server. When both are configured, the http /health built-in stands down and this surface answers that path instead, because the report is a strict superset of the constant. The http /ready and /openapi.json are untouched, since this mount claims only /health and /ops.

Point probes at the ops paths. The http built-ins cannot go red, including during boot and drain, which is the failure this surface exists to prevent. Disable them with http: { builtins: { health: { enabled: false }, ready: { enabled: false } } } if you would rather not serve both, and note that any other surface claiming /health still fails at bind rather than silently taking the path.