> ## Documentation Index
> Fetch the complete documentation index at: https://docs.internal.nscale.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Relationship: NVLink fabric (NMX-M)

> The per-rack NMX-M control plane Fleet Manager moves GPU partitions on and derives the repair-partition latch from.

* **Kind:** Observation (Fleet Manager also writes: the partition moves are its own workflows, so this page is filed under
  the state Fleet Manager derives from the fabric)
* **Version:** 1
* **Fleet Manager owner:** Fleet Manager squad, #fleet-manager-chat (individual claimant pending)
* **Counterpart owner:** unassigned (the NMX-M controllers are site networking infrastructure)
* **Last reviewed:** 2026-09-09

## What flows

`NvLinkIsolateWorkflow` and `NvLinkReturnWorkflow` (`src/fleet_manager/temporal/activities/nvlink.py`) move a returned
break-fix tray's [GPUs](../start-here/glossary#gpu) between the default [NVLink](../start-here/glossary#nvlink) partition and a repair partition on the rack's NMX-M controller. The
worker resolves the controller per Fleet Manager rack name from `nmx_hosts` and talks to `https://<host>/nmx/v1`
through the vendor-isolated client in `src/fleet_manager/core/datacenter/nmx/vendors/nmxm/client.py` (read the
domain, GPUs and partitions, replace or create or delete a partition, poll the operation). Partition membership is
read back from `/nmx/v1/gpus` after every move and recorded on the [device](../start-here/glossary#node) as the durable
`NvLinkGpusInRepairPartition` condition, source `nvlink-fabric` (`src/fleet_manager/core/conditions/registry.py`).
That condition does not move operational status; eligibility reads it and release to the customer pool refuses while
it is asserted (`release_to_customer_pool` in `src/fleet_manager/core/eligibility/policies.py`, failure code
`RELEASE_PARTITION_LATCH_REFUSED`).

Real writes are gated per site: `temporalWorker.nvlinkPartitionAllowed` in the satellite chart defaults false and
`nmx_activity_args_ready` defaults false, so a deployment on chart defaults runs the moves against the simulated
fabric and never contacts a controller. Where a site enables them, `nmx_hosts` must carry the controller's
`domain_uuid` and the move fails closed if the controller reports a different domain.

## What breaks if this changes

An NMX-M API change (the client is pinned to the 4.021.34 shapes) or an unreachable controller fails the move
workflows before mutation; a move that mutates but cannot be verified leaves the latch at `Unknown`, which blocks
release until the same move is re-run. A wrong `nmx_hosts` entry points a rack's moves at another rack's controller,
which the `domain_uuid` check is there to catch. If the latch write to Fleet Manager fails after the fabric moved, the
activity fails and the operator re-runs the move, which no-ops on the fabric and records the latch.

## Enforcement

* **Principal:** the satellite `temporal-worker` (towards Fleet Manager) and the NMX-M service account (towards
  the controller).
* **Mechanism:** towards Fleet Manager, the `temporal-worker` FM OAuth client (private-key JWT) in the satellite
  roster writing `device-condition:create`; towards NMX-M, HTTPS with the service account from the site's
  networking [1Password](../start-here/glossary#1password) vault item named by `temporalWorker.nmxCredentialsItem` (default `svc-nmx-login`,
  `src/fleet_manager/settings/agents/temporal_worker.py`), read only when real writes are enabled.
* Settings: `src/fleet_manager/settings/agents/workflow.py` (`nvlink_partition_allowed`, `nmx_hosts`,
  `nmx_activity_args_ready`).

## Gaps

* No counterpart owner: nobody outside Fleet Manager is named for the controllers, their firmware or the service
  account.
* No contract test against a real NMX-M; the client shapes were captured by hand from one firmware version.

## Change log

* 2026-09-09 v1: created after the repair-partition latch landed on main (#2818); describes present behaviour.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.