Skip to main content
Fleet Manager uses Release Please to automate versioning and releases.

​
Release Flow

  1. Merge your PR into main in nscaledev/fleet-manager.
  2. Merge the Release Please PR. Shortly after your PR merges, Release Please updates its open PR to include your changes. This PR buffers all merged pull requests into a single version bump. Review and merge it when you’re ready to release (see open PRs).
  3. Automatic release to dev. Merging the Release Please PR creates a new GitHub release and bumps the version. The release workflow then opens and auto-merges a PR in nscaledev/k8s-deploy-fleet-manager that bumps fleetManager.chartVersion. The dev environment tracks that repo’s main, so ArgoCD autosyncs the change to dev — no manual action required.
  4. Automatic release to stg, gated on dev e2e. Only a passing dev e2e run promotes the release to stg.
  5. Release to prod. Shortly after the release, a prod-pin PR appears in nscaledev/k8s-cluster-fleet-manager. Merge it to deploy to production — read the dev e2e gate first.
  6. OpenAPI client sync. After the release and the k8s-deploy-fleet-manager chartVersion bump, the release workflow sends a repository_dispatch to nscaledev/nscale-ui, which regenerates the Fleet Manager TypeScript client and opens a PR there. This is triggered by the workflow, not by the dev rollout finishing — so the client PR can appear before dev has fully synced.

​
NetBox Agent Image Pins

Fleet Manager core releases and configured agent image pins are controlled separately. Release Please adds the currently configured agent labels to the Fleet Manager release PR by default. Keep a label when that agent should move to the release version, or remove it before merging when that agent should stay pinned. The final agent tags are visible in deployment/charts/hub/release/agent-versions.yaml on the Release Please PR. The PR-time helper keeps that file aligned with the PR labels:
  • label present: the matching agents.<values_key>.tag is set to the Release Please chart version;
  • label absent: the matching tag is restored to the base branch pin.
When the Release Please PR is merged, the release workflow reads deployment/charts/hub/release/agent-versions.yaml, validates any configured release gates, and opens the automated k8s-deploy-fleet-manager PR to promote fleetManager.chartVersion. The agent pins are bundled with the Fleet Manager hub chart selected by that version. The current mappings are configured in .github/agent-release-policy.json. For example:
  • netbox-agent and netbox-matrix-agent: core, netboxAgent, and netboxMatrixAgent move to the release version;
  • only netbox-agent: core and netboxAgent move, netboxMatrixAgent stays pinned;
  • only netbox-matrix-agent: core and netboxMatrixAgent move, netboxAgent stays pinned;
  • no configured agent labels: core moves, configured agent pins stay unchanged.
Release gates are also config-driven. The current gate requires netboxMatrixAgent to bump when a release changes the matrix schema files or the apply-schema execution path configured in .github/agent-release-policy.json. The schema hook runs from the pinned matrix-agent image and reads the bundled schema.kql, so the release workflow fails before updating k8s-deploy-fleet-manager if netbox-matrix-agent is absent.

​
Release gates are bypassed by default — remove skip-release-gates when your release needs them

Release Please adds skip-release-gates to every release PR by default, which silently disables every required_bumps gate for that release. This is the opt-out model: gates only fire if a reviewer removes the label before merging. If your release touches a gated path, you are responsible for removing skip-release-gates from the Release Please PR before merge. The gates exist to prevent schema/code-vs-image skew (for example, shipping a matrix-schema change while netboxMatrixAgent stays pinned at an older tag — the pre-upgrade migration job would run from the old image and apply stale SQL). They cannot catch this for you when the override is active.
  • src/fleet_manager/agents/netbox_matrix/schema/ (any file)
  • src/fleet_manager/cli/commands/infra/netbox_matrix_agent.py
  • src/fleet_manager/core/clients/azure_kusto/management.py
  • src/fleet_manager/core/clients/azure_kusto/errors.py
These are the exact path-prefixes the matrix-agent gate watches; see required_bumps in .github/agent-release-policy.json for the live list. When you remove the label and a path gate fires, the workflow fails with an explicit message naming the gated path and the agent label you need to keep — fix that, re-merge, and the release proceeds. If a gated path was touched but the change is genuinely safe to release without bumping the agent (a comment-only edit to schema, a refactor that doesn’t change emitted SQL), leave skip-release-gates on and document the rationale in the Release Please PR description. The override is logged in the workflow run and surfaced in the k8s-deploy PR body for after-the-fact audit. The label name is configured under override_label in .github/agent-release-policy.json and can be changed there. To switch back to the opt-in model (gates default-on, label added per release), remove skip-release-gates from extra-label in release-please-config.json. extra-label is the Release Please manifest config key, singular and intentional; do not rename it to extra-labels.

​
Adding a new independently pinned agent

Do these steps in order. The chart pin and policy entry must merge before extra-label is updated, otherwise the next release workflow can generate a Release Please PR for a label that has no chart pin to update.
  1. Wire the agent into the Fleet Manager hub chart. In deployment/charts/hub/:
    • add a <agent>.image: {repository: "", tag: "", pullPolicy: ""} block in values.yaml (empty fields fall back to global.image);
    • add fleet-manager.<agent>Image and fleet-manager.<agent>ImagePullPolicy helpers in templates/_helpers.yaml (copy the netbox-agent pair);
    • in the agent’s deployment template, use fleet-manager.agentPodLabels for the pod template labels and the two new helpers for image / imagePullPolicy. This is what keeps chart-only releases from rolling the pinned agent.
  2. Register the agent in the release policy. Add an entry to agents in .github/agent-release-policy.json with the chart values_key and the Release Please bump_label.
  3. Seed the Fleet Manager release pin. Add agents.<values_key>.tag: "<current-release>" to deployment/charts/hub/release/agent-versions.yaml.
  4. Decide on path gates. If specific source paths must force this agent to bump (e.g. a schema directory or a CLI file), add one required_bumps entry with the agent key and all matching path_prefixes. A trailing / means directory prefix; no trailing / means exact file match.
  5. Add the new label to extra-label in release-please-config.json so future Release Please PRs include it by default. release-please creates the GitHub label automatically the first time it labels a PR.
Labels added after the Release Please PR is merged are too late for that release.

​
Dev E2E gate

A passing dev e2e run promotes the release to stg by moving the staging branch of k8s-deploy-fleet-manager. A red or skipped run leaves stg on its last passing release. To override, send the promote-staging dispatch to that repo by hand with the bump PR’s merge commit as client_payload[sha]. The prod-pin PR in k8s-cluster-fleet-manager carries a dev-e2e commit status for the exact version being promoted, fed by two suites: the API and contract tests, and the device workflows on dev’s emulated nodes. Both must pass. It is not a required check yet — tracked separately — so it informs the merge rather than blocking it. To promote past a red one, skip-dev-e2e-gate on the prod-pin PR forces it green; record why in a comment. Two labels sound alike and do opposite things: skip-dev-e2e-run on the release PR stops the suite running and publishes red, while skip-dev-e2e-gate on the prod-pin PR forces the status green. A run that produces no result publishes red rather than leaving the check pending. A run killed outright publishes nothing, and the missing dev-e2e-passed label on the release PR is the signal instead. Stopping a run is not waiving the gate. The status and the workflow that posts it are defined in k8s-cluster-fleet-manager.

​
ArgoCD Sync

For the dev or prod release to actually roll out after the relevant PR is merged, ArgoCD must sync the change into the cluster.
  • Find the right ArgoCD instance via Teleport — search Teleport for the ArgoCD application link for the target environment.
  • If the change isn’t rolling out, check that autosync is enabled on the application.
  • If autosync is on but the sync is slow, you can force a re-sync from the ArgoCD UI.