> ## Documentation Index
> Fetch the complete documentation index at: https://docs.internal.nscale.com/llms.txt
> Use this file to discover all available pages before exploring further.

# NATS JetStream

> The Fleet Manager-owned NATS JetStream plane that carries the Radar projection, inventory events and the gated event postcards.

* **Kind:** Dependency
* **Version:** 3
* **Fleet Manager owner:** Fleet Manager squad, #fleet-manager-chat (individual claimant pending)
* **Counterpart owner:** Fleet Manager squad operates the cluster; consumer teams (CHAD, [Radar](../start-here/glossary#radar-api), Microsoft, Google) are
  listed on their own pages
* **Last reviewed:** 2026-09-18

## What flows

Fleet Manager runs its own [NATS](../start-here/glossary#nats) JetStream plane (ADR-0031, spec 0030, runbook `docs/runbooks/nats-operations.md`):
`nats.fleet-manager.glo1.nscale.com` in production (3 servers, R3 streams) and `nats.fleet-manager-dev.glo1.nscale.com`
in development (1 server, R1). Every Fleet Manager producer and consumer in glo1 production and development is
pointed at it through the `k8s-cluster-fleet-manager` overlay: the [NetBox](../start-here/glossary#netbox) inventory agent, the Matrix connector, the
Radar ingestion agent and the CHAD event publisher as producers, CHAD's consumer, and the Google connector in
development. Radar's `radar-ingest` consumer in the observability estate also binds here. The shared `product-mgmt`
cluster is no longer in the path: the hub chart defaults every client's `nats.tls.secretName` to its
`fleet-<client>-client-tls` Secret and renders no certificate objects of its own, so returning to `product-mgmt`
is a chart revert, not a values change.

Streams in the `fleet` account, all byte-capped with `discard: old`: `fleet_jobs` (CHAD postcards, 90 d, 1 GiB),
`inventory_devices` ([device](../start-here/glossary#node) and cable events, 90 d, 1 GiB), `radar_devices` (7 d, 4 GiB) and `radar_devices_dlq`.
Durable consumers are declared as NACK `Consumer` CRDs for the Matrix and Google connectors and for `radar-ingest`;
CHAD self-creates its durables and its four `CHAD_FM_*_DLQ` streams client-side, and those are CHAD's to operate.
The connectors bind an existing durable and refuse to start if it is missing or its filter differs. Postcard
production stays gated on chart defaults (see [chad](./chad)); the plane carries whatever the overlays enable.

## What breaks if this changes

A plane outage stops the Radar projection, connector deliveries and postcards at once. Running connector pods fall
back to their polling backstops and the outbox retains rows, but Radar goes stale. A connector pod that starts or
restarts during the outage does not fall back: both `src/fleet_manager/api/connectors/matrix/main.py` and
`src/fleet_manager/api/connectors/google/main.py` await the NATS connect before the backstop task is created, so the
pod crash-loops until the plane is reachable again. Retention is `min(maxAge, maxBytes)`: a stream sitting at its
byte cap sheds oldest messages, so a consumer that lags past the effective window loses events silently. Development
has no HA; a pod restart there is a short outage that clients reconnect through on their own. A stream or subject
change needs every consumer page updated.

## Enforcement

* **Principal:** one client certificate per component, identity is the CN (`fleet-<component>-client`, for example
  `fleet-matrix-connector-client`, `fleet-chad-client`); no passwords, NKEYs or JWTs.
* **Mechanism:** Fleet Manager is its own CA. cert-manager in the `nats` namespace issues the server certificate and
  one client `Certificate` per CN; each client cert is pushed to the Fleet Manager [Vault](../start-here/glossary#vault) mount `fleet-manager` at
  `<env>/hub/nats/<client>/client-tls` and materialised back into the `fleet-manager` namespace as
  `fleet-<client>-client-tls` (see [hashicorp-vault](./hashicorp-vault)). The `fleet` account's CN allowlist is the
  whole authorisation boundary and it is flat: any allowed CN can publish, subscribe or purge, so the producer and
  consumer split is a convention. Subject grammar in `src/fleet_manager/core/events/subjects.py`.

## Gaps

* The CN allowlist has no per-role ACL (spec 0030 open question), so a compromised consumer certificate can publish.
* `denyPurgeStreams` stays false in production until post-cutover validation; the legacy `product` streams are
  retained meanwhile.

## Change log

* 2026-09-18 v3: hub chart no longer renders the Product Vault NATS certificate path (FLEET-2692); rollback is a
  chart revert. ADR-0031 accepted.
* 2026-09-16 v2: rewritten for the Fleet Manager-owned plane after the cutover; streams, identities and certificate
  flow from `docs/runbooks/nats-operations.md`.
* 2026-09-09 v1: created; described the shared `product-mgmt` cluster.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.