Skip to main content
  • Kind: Projection
  • Version: 2
  • Fleet Manager owner: Fleet Manager squad, #fleet-manager-chat
  • Counterpart owner: Radar team (SRE), #squad-radar-service
  • Last reviewed: 2026-09-17

​
What flows

When enabled, the hub radar-agent (src/fleet_manager/agents/radar_ingestion_service/) publishes DeviceResourceProjectionV1 (nscale:fleet-manager/device_resource_projection/v1) and the 60-second PulseMessageV1 heartbeat (nscale:fleet-manager/pulse/v1) on the shared legacy subject radar.{region-id}.fleet-manager.{organisation-id}.device.state-updated. Device projections substitute {region-id} with the device site UUID; pulses use the configured current_site_id (the all-zero HUB UUID by default) and the all-zero organisation UUID. radar-ingest-service dispatches by schema version and persists accepted projections to Radar’s Postgres store. Fleet Manager owns and operates radar-agent; Radar owns and operates radar-ingest-service. The chart defaults to disabling radar-agent. When the agent is enabled, the default hierarchy configuration publishes eligible GPU-compute-node projections. radarIngestionAgent.emitResourceHierarchy additionally enables switch and qualifying NVLink-domain rack projections. With hierarchy enabled, resource_class is populated from the resource’s class; parent_resource_class and parent_resource_id are populated only when an NVLink topology group or label supplies the parent relationship. The payload always identifies the resource and carries its health, availability and operational status. Attribution, rack, site, UNI-region and hardware fields are conditional: a missing dependency or hardware mapping is represented as a nullable field, and gpu_type is node-only. A device or rack without a resolvable tenant organisation is not published. The authoritative producer schema and state mapping are src/fleet_manager/agents/radar_ingestion_service/schema.py; the device and rack reconcilers, including rack status aggregation, are in src/fleet_manager/agents/radar_ingestion_service/reconciler.py. The agent polls dynamic resources every 60 seconds, static resources every 30 minutes, and performs a one-hour resync. These are implementation settings, not a documented freshness SLA or delivery guarantee.

​
What breaks if this changes

Changing the subject so it no longer matches the configured radar.*.*.*.device.state-updated consumer filter stops new projections reaching Radar. Changing the schema version, removing or changing the meaning of a consumed field, or changing a consumed enum may cause the ingest service to reject messages after retries and send them to its DLQ. Additive optional fields are not inherently breaking; any breaking change needs a jointly reviewed compatibility and rollout plan.

​
Enforcement

  • Principal: radar-agent is the Fleet Manager publisher. radar-service is Radar’s dormant Fleet Manager OAuth caller; its operations:read|write client remains in the hub roster. The Radar-side NATS consumer identity is fleet-radar-ingest-client.
  • Mechanism: radar-agent uses Fleet Manager’s radar-agent OAuth client to read the Fleet Manager API. NATS publication and consumption use separate mTLS client identities: fleet-radar-ingestion-agent-client publishes and fleet-radar-ingest-client consumes. See docs/runbooks/nats-operations.md.
  • The Fleet Manager OAuth client and scopes are in deployment/charts/hub/values.yaml. The producer schema is in src/fleet_manager/agents/radar_ingestion_service/schema.py; producer coverage is in tests/agents/radar_ingestion_service/ and tests/e2e/test_radar_ingestion.py.
  • The Radar ingest implementation is the counterpart contract consumer. ADR-0040 records the v1-to-v2 compatibility and retirement gates; there is no cross-repository versioned contract test recorded here today.

​
Gaps

  • The NATS subject uses the legacy lowercase grammar. ADR-0040 defines its v1-to-v2 migration, but no implementation schedule or rollout window is recorded here.
  • The retained radar-service Fleet Manager OAuth client and the Radar-only impairment-event API source are unused legacy integration debt, not part of this active projection relationship. Retiring them needs a separately owned review.
  • When an already-published device or rack loses its tenant-organisation mapping, the producer suppresses subsequent projections and emits no tombstone. Radar can retain a stale resource until another mechanism removes or replaces it.
  • Fleet Manager and Radar have not recorded a projection freshness commitment or a concrete deprecation and rollback schedule.

​
Change log

  • 2026-09-17 v2: rewrote the record as the active NATS projection contract; clarified deployment defaults, subject multiplexing, hierarchy fields and reconciler ownership; recorded the v1-to-v2 migration policy and missing-mapping stale-resource gap; and moved dormant impairment integration to explicit cleanup debt.
  • 2026-09-09 v1: relationship register entry created.