> ## Documentation Index > Fetch the complete documentation index at: https://docs.internal.nscale.com/llms.txt > Use this file to discover all available pages before exploring further. # OpenStack Application Credentials > Scoped OpenStack application credentials required for Fleet Manager automation. Use this page when generating scoped OpenStack application credentials for Fleet Manager automation. The access rules below come directly from the repo runbook and are intended to be copied into Horizon or supplied to the OpenStack CLI. ## General Rules * Create the credential inside the project that owns the resources * Keep a record of the application credential ID and secret when they are generated * Apply the path and method rules exactly as written ## UNI Attribution Credentials The OpenStack agent uses three UNI APIs to materialise resource attribution. It authenticates with the UNI mTLS client certificate issued by the UNI deployment. The certificate Secret is delivered to Fleet Manager through [Vault](../../start-here/glossary#vault) and mounted at `/etc/uni-certs`; private key material must not be stored in [Helm](../../start-here/glossary#helm) values, a ConfigMap, or a checked-in file. The UNI API owners must approve the service account and final read-only role before the certificate is issued. The account should be restricted to the deployment's region or organisation wherever the UNI API supports that boundary: | API | Required operation | Scope boundary | | - | - | - | | UNI Region | `GET /api/v2/servers/{server_id}` | The deployment's region | | UNI Kubernetes | `GET /api/v1/organizations/{organization_id}/clusters` | The deployment's organisation | | UNI Identity | `GET /api/v1/organizations/{organization_id}/projects/{project_id}` | The deployment's organisation | The deployment configuration supplies the three UNI base URLs and the name of the Kubernetes Secret containing these keys: | Secret key | Contents | | - | - | | `tls.crt` | UNI client certificate, with the service account CN | | `tls.key` | Matching RSA private key | | `ca.crt` | CA bundle used to verify UNI endpoints | The UNI deployment owns certificate creation, Vault publication, and rotation. Fleet Manager owns consumption and must not copy the values into repository configuration or expose them in logs. Before production rollout, the API owner and Fleet Manager operator must sign off the access review and perform one non-production certificate rotation exercise: 1. Issue a replacement certificate with the same approved read-only role. 2. Publish it to the environment-scoped Vault path and verify the ExternalSecret updates the mounted Secret. 3. Verify the three read paths and attribution lookups in non-production. 4. Revoke the previous certificate and record the result in the deployment notes. The OpenStack agent detects the mounted certificate change and reloads the UNI credentials without requiring a pod restart. The same owner-approved procedure is used for routine rotation and emergency revocation. This contract uses one shared read-only certificate identity for all three APIs; separate Region, Kubernetes, and Identity credentials are not configurable by this ticket. ## Ironic Instance Overview Fleet Manager routes [Ironic](../../start-here/glossary#ironic) API calls to two separate Ironic instances: * **Overcloud** — manages GPU\_COMPUTE and CPU\_BAREMETAL nodes. **Always required.** * **Undercloud** — manages CPU\_HYPERVISOR nodes. Only add if the deployment has FM-managed hypervisor hardware; the OpenStack agent skips the undercloud automatically when no [devices](../../start-here/glossary#node) at that site map to it. ## Create the Credentials Fleet Manager consumes credentials by reading a `clouds.yaml` file from [1Password](../../start-here/glossary#1password) as a document. Each credential uses this shape: ```yaml theme={null} clouds: [CLOUD_NAME]: auth_type: v3applicationcredential auth: auth_url: [TO BE FILLED] application_credential_id: [TO BE FILLED] application_credential_secret: [TO BE FILLED] region_name: [TO BE FILLED] ``` ### Endpoints behind a private CA Add `cacert_pem` — the CA certificate inline — to any cloud whose endpoints present a certificate signed by a private CA. Omit it otherwise; a cloud that names no CA verifies against the public roots. The inline form exists because a `clouds.yaml` delivered as a 1Password document has no file on disk for the standard `cacert` path to point at. Introduced for sys2-lon14, the first dual-stack deployment, whose OpenStack endpoints are served by a one-off self-signed CA. ```yaml theme={null} clouds: overcloud: auth_type: v3applicationcredential auth: auth_url: [TO BE FILLED] application_credential_id: [TO BE FILLED] application_credential_secret: [TO BE FILLED] region_name: [TO BE FILLED] cacert_pem: | -----BEGIN CERTIFICATE----- MIIDxTCCAq2gAwIBAgIU... -----END CERTIFICATE----- ``` ### Overcloud credentials (always required) Source the overcloud OpenStack admin credentials: ```bash theme={null} source [PATH_TO_OVERCLOUD_OPENRC] # [TO BE FILLED] ``` Create the application credential (access rules are in the section below): ```bash theme={null} openstack application credential create \ --description "Fleet Manager [DEPLOYMENT_NAME] overcloud" \ fleet-manager-[DEPLOYMENT_NAME]-overcloud ``` Create a `clouds.yaml` with `overcloud` as the cloud name and upload to 1Password: ```yaml theme={null} clouds: overcloud: auth_type: v3applicationcredential auth: auth_url: [TO BE FILLED] application_credential_id: [TO BE FILLED] application_credential_secret: [TO BE FILLED] region_name: [TO BE FILLED] # Only when the endpoints are behind a private CA. cacert_pem: | -----BEGIN CERTIFICATE----- [TO BE FILLED] -----END CERTIFICATE----- ``` ```bash theme={null} op document create overcloud-clouds.yaml \ --title="openstack-overcloud" \ --vault="Fleet Manager [DEPLOYMENT_NAME] [ENVIRONMENT]" ``` ### Undercloud credentials (only for deployments with CPU\_HYPERVISOR nodes) Source the undercloud OpenStack admin credentials: ```bash theme={null} source [PATH_TO_UNDERCLOUD_OPENRC] # [TO BE FILLED] ``` Create the application credential (use the same access rules as the overcloud): ```bash theme={null} openstack application credential create \ --description "Fleet Manager [DEPLOYMENT_NAME] undercloud" \ fleet-manager-[DEPLOYMENT_NAME]-undercloud ``` Create a `clouds.yaml` with `undercloud` as the cloud name and upload to 1Password: ```yaml theme={null} clouds: undercloud: auth_type: v3applicationcredential auth: auth_url: [TO BE FILLED] application_credential_id: [TO BE FILLED] application_credential_secret: [TO BE FILLED] region_name: [TO BE FILLED] # Only when the endpoints are behind a private CA. cacert_pem: | -----BEGIN CERTIFICATE----- [TO BE FILLED] -----END CERTIFICATE----- ``` ```bash theme={null} op document create undercloud-clouds.yaml \ --title="openstack-undercloud" \ --vault="Fleet Manager [DEPLOYMENT_NAME] [ENVIRONMENT]" ``` ### SLURM credentials Upload the [SLURM](../../start-here/glossary#slurm) credential: ```bash theme={null} op document create clouds-slurm.yaml \ --title="openstack-slurm" \ --vault="Fleet Manager [DEPLOYMENT_NAME] [ENVIRONMENT]" ``` Fleet Manager resolves all documents by title; storing the raw credential fields under any other item name is not read by the runtime. ## Sentinel Operator Credentials Sentinel OpenStack access is operator/debug access. Store these read-only credentials in the operators vault, not the runtime service-credential vault: ```text theme={null} Fleet Manager [DEPLOYMENT_NAME]-operators [ENVIRONMENT] [DEPLOYMENT_NAME]-sentinel-openstack-overcloud ``` Sentinel targets the public overcloud Keystone. The internal-only undercloud and SLURM Keystones (`10.x`) are not reachable from an operator laptop, so Sentinel does not carry credentials for them. Keep runtime service credentials separate: ```text theme={null} Fleet Manager [DEPLOYMENT_NAME] [ENVIRONMENT] openstack-overcloud openstack-undercloud openstack-slurm ``` Preferred creation path: ```bash theme={null} uv run fleetctl deployment-standup openstack sentinel-credential-create [DEPLOYMENT_NAME] [ENVIRONMENT] \ --store ``` The command generates the overcloud `clouds.yaml` and stores it as the `[DEPLOYMENT_NAME]-sentinel-openstack-overcloud` document. It does not print the one-time Keystone secret unless `--print` is explicitly passed. ### Sentinel overcloud read-only profile Use this profile for the `overcloud` Sentinel credential. | Service | Method | Path | Purpose | | - | - | - | - | | baremetal | GET | `/**` | Read all Ironic endpoints (nodes, ports, portgroups, inventory) | | placement | GET | `/**` | Read resource providers, inventories, traits, usages, allocations | | compute | GET | `/**` | Read [Nova](../../start-here/glossary#nova) servers, hypervisors, services, migrations, flavors | | network | GET | `/**` | Read Neutron ports, networks, subnets, routers, floating IPs, security groups | ```yaml theme={null} access_rules: - service: baremetal path: /** method: GET - service: placement path: /** method: GET - service: compute path: /** method: GET - service: network path: /** method: GET ``` Every rule is `method: GET` — that is the read-only safety boundary, and it holds even though the credential carries the `admin` role. Admin is required because the overcloud baremetal and placement APIs are admin-scoped: a project `reader`/`member` cannot see nodes or resource providers. `/**` matches every path in the service, which covers ironicclient's trailing-slash collection requests (`GET /v1/nodes/`, which a single-segment `/v1/*` rule misses — Keystone would reject `openstack baremetal node list` with a 401) and avoids surprise 401s on unlisted read sub-resources during an investigation. Because the `admin` role is cross-project, this credential can read the listed services **across every [tenant](../../start-here/glossary#tenant)** — all of Nova and Neutron metadata, not just one project's. Grant it deliberately, treat the materialized `clouds.yaml` as sensitive, and where the APIs support it prefer trialling a lower-privilege system-scoped `reader` credential via `--role`. Future Sentinel write access should be added by changing the Keystone access rules for the Sentinel application credential. Do not add partial `fleetctl openstack` wrappers for OpenStack mutations; use the real OpenStack CLI so Keystone remains the enforcement point. ## Project-Prep Admin Credential The `deployment-standup openstack application-credential-create` command creates this credential programmatically from the Fleet Manager code. Use this section only when creating the same credential manually in Horizon or with the OpenStack CLI. This credential lets deployment standup create the Fleet Manager project, grant the configured member role, and prepare Nova, Cinder, and Neutron quotas. | Service | Method | Path | Purpose | | - | - | - | - | | identity | GET | `/v3/projects` | Find the target project | | identity | POST | `/v3/projects` | Create the target project | | identity | GET | `/v3/users` | Find the configured project member user | | identity | GET | `/v3/roles` | Find the configured member role | | identity | GET | `/v3/role_assignments` | Check existing project role grants | | identity | PUT | `/v3/projects/*/users/*/roles/*` | Grant the member role on the project | | compute | GET | `/v2.1/os-quota-sets/*` | Read Nova quotas | | compute | PUT | `/v2.1/os-quota-sets/*` | Update Nova quotas | | volumev3 | GET | `/v3/*/os-quota-sets/*` | Read Cinder quotas | | volumev3 | PUT | `/v3/*/os-quota-sets/*` | Update Cinder quotas | | network | GET | `/v2.0/quotas/*` | Read Neutron quotas | | network | PUT | `/v2.0/quotas/*` | Update Neutron quotas | ```yaml theme={null} access_rules: - service: identity path: /v3/projects method: GET - service: identity path: /v3/projects method: POST - service: identity path: /v3/users method: GET - service: identity path: /v3/roles method: GET - service: identity path: /v3/role_assignments method: GET - service: identity path: /v3/projects/*/users/*/roles/* method: PUT - service: compute path: /v2.1/os-quota-sets/* method: GET - service: compute path: /v2.1/os-quota-sets/* method: PUT - service: volumev3 path: /v3/*/os-quota-sets/* method: GET - service: volumev3 path: /v3/*/os-quota-sets/* method: PUT - service: network path: /v2.0/quotas/* method: GET - service: network path: /v2.0/quotas/* method: PUT ``` Save the YAML above to `project_prep_access_rules.yaml` and apply it when creating the project-prep admin credential: ```bash theme={null} openstack application credential create \ --description "Fleet Manager deployment-standup project-prep credential" \ --access-rules project_prep_access_rules.yaml \ openstack-project-admin ``` ## Admin Project: Bare Metal and Placement These permissions are required for Fleet Manager workflows that manage bare-metal inventory through Ironic. Custom traits (e.g. `CUSTOM_CUSTOMER_READY`) are written via the Ironic node-traits API and also written directly to [Placement](../../start-here/glossary#placement) in time-sensitive flows (quarantine, release, enrolment) where waiting for Nova's resource sync interval is not acceptable. | Service | Method | Path | Purpose | | - | - | - | - | | baremetal | GET | `/v1/*` | Read base baremetal metadata | | baremetal | GET | `/v1/nodes/*` | Inspect node status | | baremetal | GET | `/v1/nodes/*/inventory` | Read inspection inventory and LLDP data | | baremetal | GET | `/v1/nodes/*/history` | Read node event history for Node Diagnostics recurrence | | baremetal | POST | `/v1/nodes` | Create new nodes | | baremetal | POST | `/v1/nodes/*` | Trigger node actions | | baremetal | PUT | `/v1/nodes/*` | Replace node records | | baremetal | PUT | `/v1/nodes/*/*/*` | Update nested node resources such as traits | | baremetal | DELETE | `/v1/nodes/*/*/*` | Remove nested node resources such as traits | | baremetal | PATCH | `/v1/nodes/*` | Apply partial node updates | | baremetal | DELETE | `/v1/nodes/*` | Remove nodes | | baremetal | PUT | `/v1/nodes/*/maintenance` | Set maintenance mode | | baremetal | DELETE | `/v1/nodes/*/maintenance` | Clear maintenance mode | | baremetal | GET | `/v1/ports/*` | Inspect ports | | baremetal | POST | `/v1/ports` | Create ports | | baremetal | PATCH | `/v1/ports/*` | Record `local_link_connection` on an existing port before inspection | | baremetal | DELETE | `/v1/ports/*` | Remove ports | | baremetal | GET | `/v1/portgroups/*` | Inspect port groups | | baremetal | POST | `/v1/portgroups` | Create port groups | | baremetal | DELETE | `/v1/portgroups/*` | Remove port groups | | placement | GET | `/resource_providers/*/allocations` | Check an RP for live consumers before unenrolment tears down a node, a dead-tray node swap, or a node-swap recovery rerun | | placement | GET | `/resource_providers/*/inventories` | Read inventories | | placement | GET | `/resource_providers/*/traits` | Read traits | | placement | PUT | `/resource_providers/*/traits` | Write traits directly to Placement | | placement | DELETE | `/resource_providers/*` | Remove resource provider (unenrolment) | | compute | GET | `/v2.1/servers/*` | Check for live Nova instances before unenrolment, before a node-swap recovery teardown, and for orphan detection | | compute | POST | `/v2.1/servers/*/action` | Force-delete the Nova instance attached to an ICE2 node during operator [reclaim](../../start-here/glossary#reclaim) | | network | GET | `/v2.0/ports` | List ports still bound to an Ironic node after it is deleted (orphaned-port detection at teardown) | ```yaml theme={null} access_rules: - service: baremetal path: /v1/* method: GET - service: baremetal path: /v1/nodes/* method: GET - service: baremetal path: /v1/nodes/*/inventory method: GET - service: baremetal path: /v1/nodes/*/history method: GET - service: baremetal path: /v1/nodes/* method: POST - service: baremetal path: /v1/nodes method: POST - service: baremetal path: /v1/nodes/* method: PUT - service: baremetal path: /v1/nodes/*/*/* method: PUT - service: baremetal path: /v1/nodes/*/*/* method: DELETE - service: baremetal path: /v1/nodes/* method: PATCH - service: baremetal path: /v1/nodes/* method: DELETE - service: baremetal path: /v1/ports/* method: GET - service: baremetal path: /v1/ports method: POST - service: baremetal path: /v1/ports/* method: PATCH - service: baremetal path: /v1/ports/* method: DELETE - service: baremetal path: /v1/portgroups/* method: GET - service: baremetal path: /v1/portgroups method: POST - service: baremetal path: /v1/portgroups/* method: DELETE - service: baremetal path: /v1/nodes/*/maintenance method: PUT - service: baremetal path: /v1/nodes/*/maintenance method: DELETE - service: placement path: /resource_providers/*/allocations method: GET - service: placement path: /resource_providers/*/inventories method: GET - service: placement path: /resource_providers/*/traits method: GET - service: placement path: /resource_providers/*/traits method: PUT - service: placement path: /resource_providers/* method: DELETE - service: compute path: /v2.1/servers/* method: GET - service: compute path: /v2.1/servers/*/action method: POST - service: network path: /v2.0/ports method: GET ``` The `baremetal GET /v1/nodes/*/history` rule is newer than most existing site credentials. `/v1/nodes/*` is a single path segment and does **not** match `/v1/nodes//history`, so a credential issued before this rule refuses the call. Node Diagnostics degrades rather than failing — the run still classifies the failure and records `history: unsupported` in its evidence — but it emits no `clean_failed.recurring` finding, so a node that has failed the same way before looks like a first occurrence. Add the rule when you next reissue the site's credential. The endpoint also requires Ironic API microversion 1.78 or newer; older deployments degrade the same way. The `baremetal PATCH /v1/ports/*` rule is newer than existing site credentials. Enrolment uses it only to repair an Ironic port that has no `local_link_connection`, which occurs on hosts with `ironic_settings.skip_cable_verification` enrolled before cabled links were kept. Recreate each site's overcloud credential from the YAML above before relying on that repair; until then it fails with an authorisation error and the node stays at inspection. The `network GET /v2.0/ports` rule is newer than most existing site credentials, and the Neutron port check that uses it is **off by default** for exactly that reason. Turn it on in the same change that reissues a site's credential — set `openstack.neutronPortCheckEnabled: true` in that site's satellite values — otherwise the check stays dormant and orphaned ports go on being invisible. The credential must also be **admin-scoped** (as the overcloud one already is — see the note above). `binding:host_id` is an admin-only Neutron attribute: a project-scoped token gets no 403, it gets every port in the project with the binding fields stripped. Fleet Manager refuses such a response rather than reporting the whole site as orphaned, so the symptom is a "could not check Neutron" warning, not bad advice. Standalone-Ironic sites run no Neutron at all. Leave the rule out and the flag off there; there is nothing to read. Save the YAML above to `admin_access_rules.yaml` and apply it when creating each Ironic credential. For the overcloud (and undercloud if applicable): ```bash theme={null} openstack application credential create \ --description "Fleet Manager [DEPLOYMENT_NAME] overcloud" \ --access-rules admin_access_rules.yaml \ fleet-manager-[DEPLOYMENT_NAME]-overcloud ``` ## Slurm Project: Nova These permissions are required for the SLURM-side compute integration and Image Node workflows. | Service | Method | Path | Purpose | | - | - | - | - | | compute | GET | `/v2.1/servers` | List instances | | compute | GET | `/v2.1/servers/detail` | List instances with full attributes (used to find stale ERROR servers during SLURM adopt) | | compute | GET | `/v2.1/servers/*` | Inspect a single instance | | compute | POST | `/v2.1/servers` | Create a new SLURM or Image Node instance | | compute | POST | `/v2.1/servers/*/metadata` | Set server metadata (topology group) | | compute | DELETE | `/v2.1/servers/*` | Delete a SLURM or Image Node instance | | compute | POST | `/v2.1/servers/*/action` | Rebuild an already-adopted SLURM instance in place onto the current image (force\_rebuild) | ```yaml theme={null} access_rules: - service: compute path: /v2.1/servers method: GET - service: compute path: /v2.1/servers/detail method: GET - service: compute path: /v2.1/servers/* method: GET - service: compute path: /v2.1/servers method: POST - service: compute path: /v2.1/servers/*/metadata method: POST - service: compute path: /v2.1/servers/* method: DELETE - service: compute path: /v2.1/servers/*/action method: POST ``` Create the SLURM-project credential by saving the YAML above to `slurm_access_rules.yaml` and then running: ```bash theme={null} openstack application credential create \ --description "Fleet Manager [DEPLOYMENT_NAME] SLURM compute access" \ --access-rules slurm_access_rules.yaml \ fleet-manager-[DEPLOYMENT_NAME]-slurm ``` ## Where These Credentials Are Used These credentials feed the wider deployment flow: * site rollout and secret bootstrap * Ironic enrolment and node maintenance operations * Ironic node trait updates (CUSTOM\_CUSTOMER\_READY etc.) * Placement inventory and trait reads * Node unenrolment (Ironic teardown, Placement RP deletion, Nova deployed-instance check) * SLURM-side compute lifecycle actions Continue with [Deploy Fleet Manager at a New Site](./deploy-fleet-manager-at-a-new-site) once the credentials and vault storage are in place. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.