gNSI Certz — TLS Certificate Lifecycle on Cisco IOS XR

The Problem

Authz and Pathz control what clients can do over gRPC and gNMI, but all of that depends on TLS being set up correctly. Certificates expire. Trust Bundles go stale. CRLs need updating. And every one of those changes has to happen without dropping the gRPC server. Traditionally, managing TLS material on IOS XR meant copying PEM files to specific paths and hoping the right files ended up in the right place. There was no standard API for rotating Certificates, no Rollback if something went wrong, and no clean way to manage multiple TLS Profiles on the same box.

What is Certz

gNSI Certz solves this by grouping everything TLS needs into SSL Profiles. Each Profile bundles:

• Certificate and Private Key — the server Authentication material
• Trust Bundle — the CA chain clients validate against
• CRL Bundle — Certificate Revocation Lists
• Authentication Policy — for mutual TLS and similar patterns

Instead of copying files, a Controller stages new material through a Rotate RPC, validates it, and finalizes — or drops the stream to roll back. On IOS XR, EMSd implements Certz and stores Profiles under /misc/config/grpc/gnsi/certz/ssl_profiles/.

This article covers SSL Profiles, the six RPCs, two end-to-end Rotation flows (router-generated CSR vs workstation-generated key), and grpcurl examples. Service-level Authorization is handled by Authz; path-level Authorization by Pathz. Certz keeps the transport underneath trustworthy.

Within AAA, Certz is the Authentication plane for the gRPC management plane — it manages the TLS material that proves who a client is before any Authz/Pathz decision runs. (Its sibling Credentialz plays the equivalent role for the SSH plane.)

---

Certz compared with gNOI cert

IOS XR has long supported gNOI cert.proto for Certificate operations. Certz is the OpenConfig gNSI evolution and is where new feature work lands. The table below highlights why operators should move from the older interface to Profiles plus Rotate:

Topic	gNOI cert.proto (legacy)	gNSI certz.proto
Organization	Certificate ID per leaf; CA Bundle updates overwrite a single Bundle	All PKI material grouped under a named SSL Profile
CSR	No SAN in CSR params on some stacks	SAN is part of CSRParams
RPC surface	Install / load CA / revoke as separate RPCs	One streaming Rotate carries Certificate, Key, Trust Bundle, CRL in UploadRequest
Algorithms	RSA-oriented history	RSA, ECDSA, Ed25519 supported

Industry direction treats gNOI cert.proto as deprecated. New deployments should use Certz.

---

SSL Profiles on IOS XR

What is an SSL Profile?

An SSL Profile is a single place the router looks for “everything TLS needs” when it brings up gRPC:

• Authentication Material — the leaf Certificate and its Private Key
• Trust Anchors — the CA chain clients validate against
• Revocation — CRL Bundles
• Policy — optional hooks for mutual TLS (some fields remain platform-specific; check Release Notes)

Default Profile: gNxI

The gRPC server on IOS XR uses Profile ID gNxI unless configured otherwise. In RotateCertificateRequest, leaving ssl_profile_id empty typically resolves to gNxI.

The Profile bound to the active gRPC server cannot be deleted — DeleteProfile returns an error.

On-disk Persistence

IOS XR stores Certz SSL Profiles under:

/misc/config/grpc/gnsi/certz/ssl_profiles/<ssl_profile_id>/

Metadata like version and created_on is stored in JSON next to the PEM files. Standby RP synchronization for everything under /misc/config/grpc/gnsi/ works through RDSFS. Failover and EMSd restarts depend on that directory tree staying consistent.

---

Certz RPCs

RPC	Type	What it does
GetProfileList	Unary	List every ssl_profile_id on the device
AddProfile	Unary	Create an empty Profile shell; material arrives via Rotate
CanGenerateCSR	Unary	Check whether the supplied CSRParams are supported
Rotate	Bidirectional stream	Optional CSR generation, UploadRequest payloads, then finalize_rotation
DeleteProfile	Unary	Remove a Profile that is not bound to the live gRPC server
GetIntegrityManifest	Unary	Integrity / RIM manifest (often Unimplemented on current IOS XR builds)

Note: The OpenConfig service comment says only one Certz RPC should run at a time. Serialize access the same way as with Authz Rotate or Pathz Rotate.

On 25.3.1: grpcurl … list gnsi.certz.v1.Certz shows GetIntegrityManifest, but a real call returns Code: Unimplemented. Reflection advertises the descriptor; it doesn’t guarantee EMSd implemented the RPC. Always verify with a real call.

---

How the Rotate RPC Works

Transaction Model

Rotate is a streamed Transaction for one ssl_profile_id:

1. Open the stream for a single Profile.
2. Send optional generate_csr, then one or more certificates messages (UploadRequest with repeated Entity values), then finalize_rotation.
3. Until finalize arrives, new material stays as a Candidate. Drop the stream without finalize to trigger Rollback.
4. Use force_overwrite: true to replace an existing version string. Without it, reusing a version is rejected — the proto names this AlreadyExists, but IOS XR 24.4.x surfaces it as InvalidArgument: certificate: same version sent (full example below).

IOS XR Behavior (24.4.x)

On recent lab images, every RotateCertificateRequest on the stream — including the finalize message — needs the same sslProfileId. Omitting it on finalize produces:

InvalidArgument: Not expecting rotate RPC for multiple SSL profiles in single stream

Always follow this pattern:

{"sslProfileId":"lab-profile-1","generateCsr":{...}}
{"sslProfileId":"lab-profile-1","certificates":{...}}
{"sslProfileId":"lab-profile-1","finalizeRotation":{}}

Streaming with grpcurl

grpcurl accepts multiple JSON objects on stdin — one RotateCertificateRequest per line — for Rotate:

grpcurl … -d '@' … gnsi.certz.v1.Certz.Rotate <<'EOF'
{... first request...}
{... second request...}
EOF

Use -d '@' with single quotes so grpcurl reads from stdin.

What force_overwrite actually does

The proto comment on force_overwrite says reusing a version without it returns AlreadyExists. On IOS XR 24.4.x the same condition surfaces as:

ERROR:
 Code: InvalidArgument
 Message: certificate: same version sent

Setting "forceOverwrite": true on every non-finalize message lets the same version be replayed without changing the value — handy when retrying after a transient error during a lab walk-through.

---

IOS XR Support

Release	What changed
7.8.1	Certz SSL Profile infrastructure introduced.
24.1.1	Rotate RPC, grpc gnsi service certz ssl-profile-id CLI, configurable CN for dial-in.
24.4.1	TLS 1.3 support.
25.4.1	Legacy /misc/config/grpc PEM paths deprecated — SSL Profiles are the path forward.

Platforms: Cisco 8000, NCS 5500, ASR 9000 families. Confirm exact SKU and PI/SMU in Release Notes.

---

IOS XR CLI Reference

Configuration command	What it does
grpc gnsi service certz ssl-profile-id <name>	Bind the gRPC server to a Certz SSL Profile
no grpc gnsi service certz ssl-profile-id	Unbind Certz; fall back to gNOI cert.proto or legacy trustpoint PEM paths

Show / exec command	What it shows
show grpc certificate	Active Certificate subject, issuer, validity dates, and chain
show gnsi state	gNSI subsystem status (Certz, Acctz threads, RDSFS state)
show tech-support gnsi	Full gNSI diagnostic bundle (all services)

---

Hands-on Setup

Prerequisites

• IOS XR router with gRPC enabled (24.1.1 or later for Certz)
• grpcurl, jq, and OpenSSL installed
• OpenConfig gNSI protos cloned and laid out so that $GNSI_IMPORT/certz/certz.proto resolves:

git clone https://github.com/openconfig/gnsi.git /tmp/gnsi
mkdir -p./certz-import/certz
cp /tmp/gnsi/certz/certz.proto./certz-import/certz/certz.proto
export GNSI_IMPORT="$PWD/certz-import"

certz.proto only imports google/protobuf/any.proto, which grpcurl ships internally — no other files need to be staged.

Environment

export GNSI_IMPORT="${GNSI_IMPORT:-$PWD/certz-import}"
export XR="${XR:-198.51.100.1:57400}"
export XR_USER="${XR_USER:-cisco}"
export XR_PASS="${XR_PASS:?set XR_PASS for the IOS XR account}"
GRPC_MD=(-H "username:${XR_USER}" -H "password:${XR_PASS}")
PROTO_CERTZ="certz/certz.proto"

${XR_PASS:?…} aborts the shell with a clear error if XR_PASS is not exported, so the reader doesn’t accidentally send an empty password to the device.

Lab only: The examples below use -plaintext because they target a gRPC server that has no Certz Profile bound yet. Once a Profile is active and bound to the gRPC server (grpc gnsi service certz ssl-profile-id <name>), swap -plaintext for -cacert, and add -cert/-key if mutual TLS is enabled.

What this hands-on covers (and what it does not)

Each row below maps a section of this hands-on to whether it completes with grpcurl alone, or needs a richer client / out-of-band step:

Section	Works end-to-end with grpcurl?	Notes
Reflection list	Yes	All six RPCs returned
GetProfileList, AddProfile, DeleteProfile	Yes	Empty / populated responses both shown
CanGenerateCSR	Yes	Success and missing CN failure both shown
GetIntegrityManifest	No	Server returns Unimplemented on IOS XR 24.4.x — documented
Lab CA + signed leaf (OpenSSL)	Yes	Two-tier CA, leaf signed locally
Flow B — Client-generated key + nested chain Rotate	Yes	Full chain + finalize, plus optional Trust Bundle entity
Flow A — Target-generated CSR	CSR step yes, full Rotate no	Demonstrates a documented grpcurl limitation; production use needs a Python/Go client
Rollback (close stream pre-finalize)	Yes	Code: Unknown / EOF is the rollback signal
forceOverwrite semantics	Yes	Same version retried cleanly
gNMI verification (post-binding)	Partial	Authz counters always populated; Certz state subtree only after CLI binding

PEM, Base64, and CSR Vocabulary

PEM is the text envelope (-----BEGIN …-----) around a DER blob. In certz.proto, Certificate and CSR fields are bytes; grpcurl represents bytes as standard Base64 over the PEM file bytes.

Two CSR patterns appear throughout:

• Target-generated CSR — the router creates the Key Pair and CSR. Sign the CSR with a CA, then upload only the signed Certificate. The Private Key stays on the router.
• Client-generated CSR — create the Key and CSR on a workstation, sign with a CA, then upload both the signed Certificate and Private Key via UploadRequest.

---

Hands-on: List the RPCs

List exported Services (Reflection)

grpcurl -plaintext -import-path "$GNSI_IMPORT" -proto "$PROTO_CERTZ" \
 "${GRPC_MD[@]}" \
 "${XR}" list gnsi.certz.v1.Certz

Expected output:

gnsi.certz.v1.Certz.AddProfile
gnsi.certz.v1.Certz.CanGenerateCSR
gnsi.certz.v1.Certz.DeleteProfile
gnsi.certz.v1.Certz.GetIntegrityManifest
gnsi.certz.v1.Certz.GetProfileList
gnsi.certz.v1.Certz.Rotate

---

Unary RPCs: Profiles, CSR Checks, and Cleanup

GetProfileList

Returns every Profile ID that exists on the device:

grpcurl -plaintext -import-path "$GNSI_IMPORT" -proto "$PROTO_CERTZ" \
 "${GRPC_MD[@]}" \
 -d '{}' \
 "${XR}" gnsi.certz.v1.Certz.GetProfileList

When no Profiles have been created yet, the response is an empty object. On some builds this call fails until the ssl_profiles tree exists on disk — if an internal error about a missing path appears, run AddProfile first and try again.

Router response (no Profiles yet):

{}

After a few AddProfile calls (see below), the same RPC returns the populated list:

{
 "sslProfileIds": [
 "lab-profile-1",
 "lab-temp"
 ]
}

AddProfile

Creates an empty Profile shell. Pick any administrator-friendly ID:

grpcurl -plaintext -import-path "$GNSI_IMPORT" -proto "$PROTO_CERTZ" \
 "${GRPC_MD[@]}" \
 -d '{"sslProfileId":"lab-profile-1"}' \
 "${XR}" gnsi.certz.v1.Certz.AddProfile

Router response (success returns an empty JSON object):

{}

Re-running AddProfile for the same ID returns:

ERROR:
 Code: InvalidArgument
 Message: Profile:lab-profile-1 already exists

Most lab scripts wrap AddProfile with || true so the duplicate error doesn’t abort the rest of the workflow.

CanGenerateCSR

Check before relying on the router to mint a CSR inside Rotate. Cheaper than opening a stream only to learn the suite is unsupported. Note: commonName is mandatory — omitting it returns InvalidArgument.

grpcurl -plaintext -import-path "$GNSI_IMPORT" -proto "$PROTO_CERTZ" \
 "${GRPC_MD[@]}" \
 -d '{
 "params": {
 "csrSuite": "CSRSUITE_X509_KEY_TYPE_ECDSA_PRIME256V1_SIGNATURE_ALGORITHM_SHA_2_256",
 "commonName": "router.example.com",
 "organization": "Lab",
 "country": "US"
}
}' \
 "${XR}" gnsi.certz.v1.Certz.CanGenerateCSR

Router response:

{
 "canGenerate": true
}

RSA-2048 / SHA-256 returns the same canGenerate: true on IOS XR 24.4.x. Omitting commonName returns:

ERROR:
 Code: InvalidArgument
 Message: CSR parameters does not have common name

When canGenerate is false, use the client-generated path: build the Key and CSR on a workstation and upload through Rotate.

GetIntegrityManifest

grpcurl -plaintext -import-path "$GNSI_IMPORT" -proto "$PROTO_CERTZ" \
 "${GRPC_MD[@]}" \
 -d '{}' \
 "${XR}" gnsi.certz.v1.Certz.GetIntegrityManifest

Several IOS XR releases return Unimplemented here even though Reflection lists the RPC. On 24.4.x, the exact response is:

ERROR:
 Code: Unimplemented
 Message: unknown method GetIntegrityManifest for service gnsi.certz.v1.Certz

That is a server-side gap, not grpcurl misconfiguration.

DeleteProfile

Removes a Profile that is not bound to the active gRPC server:

grpcurl -plaintext -import-path "$GNSI_IMPORT" -proto "$PROTO_CERTZ" \
 "${GRPC_MD[@]}" \
 -d '{"sslProfileId":"lab-temp"}' \
 "${XR}" gnsi.certz.v1.Certz.DeleteProfile

Router response on success:

{}

Common error responses on IOS XR 24.4.x:

ERROR:
 Code: InvalidArgument
 Message: Profile: nonexistent-xyz does not exist: <nil>

The default Profile ID gNxI only resolves once Certz has actually been bound to the gRPC server — until then, deleting it returns the same does not exist error rather than the more descriptive “bound to active server” message.

---

Hands-on: Rotate End to End

Continue from the Hands-on Setup environment above. The sections below walk through real Certificate material. Start with a small lab PKI so CSRs can be signed without involving a production CA, then try both Rotation styles.

Which flow to start with: Most operators generate keys on a workstation and push them through Certz — that is Flow B below. Flow A (target-generated CSR) is useful when the Private Key must never leave the router, but grpcurl cannot complete the full multi-step stream interactively — a programmatic client (Python, Go) is needed for production use of that flow.

Rotate Message Flow

Step	Message field (oneof rotate_request)	What it does
1 (optional)	generate_csr	Target generates Key + CSR; PEM CSR returns on the response stream
2	certificates	UploadRequest: one or more Entity values (leaf Certificate, Trust Bundle, CRL, …)
last	finalize_rotation	Commit. Omitting / aborting triggers Rollback

Each Entity carries:

• version — opaque string surfaced in Telemetry
• created_on — Unix epoch seconds
• Exactly one arm of the entity oneof (certificateChain, trustBundle, etc.)

Build a Two-tier Lab CA (one-time OpenSSL)

Enterprise networks almost never sign router Certificates directly with a long-lived Root. They issue an Intermediate, keep the Root offline, and let the Intermediate sign day-to-day CSRs. This lab mirrors that pattern.

1 — Root CA (Trust Anchor, self-signed)

mkdir -p lab-pki/root lab-pki/intermediate
openssl genrsa -out lab-pki/root/root.key 4096
openssl req -new -x509 -key lab-pki/root/root.key -sha256 -days 3650 \
 -out lab-pki/root/root.crt \
 -subj "/CN=Lab Root CA/O=Lab/C=US" \
 -addext "basicConstraints=critical,CA:TRUE" \
 -addext "keyUsage=critical,keyCertSign,cRLSign"

2 — Intermediate CA (signed by Root)

openssl genrsa -out lab-pki/intermediate/intermediate.key 4096
openssl req -new -key lab-pki/intermediate/intermediate.key \
 -out lab-pki/intermediate/intermediate.csr \
 -subj "/CN=Lab Issuing CA/O=Lab/C=US" \
 -addext "basicConstraints=critical,CA:TRUE,pathlen:0" \
 -addext "keyUsage=critical,keyCertSign,cRLSign"
openssl x509 -req -in lab-pki/intermediate/intermediate.csr \
 -CA lab-pki/root/root.crt -CAkey lab-pki/root/root.key -CAcreateserial \
 -out lab-pki/intermediate/intermediate.crt -days 1825 -sha256 \
 -copy_extensions copy

3 — Verify the Intermediate chains to the Root

openssl verify -CAfile lab-pki/root/root.crt lab-pki/intermediate/intermediate.crt

• lab-pki/root/root.key is as sensitive as any Root signing Key — treat it accordingly.
• lab-pki/intermediate/intermediate.key is the “online” signer; still guard it closely.
• lab-pki/root/root.crt is the Trust Anchor passed to openssl verify.

---

Flow A — Target-generated CSR (demonstrates a grpcurl limitation)

What this section actually shows: Steps 1-3 work cleanly — grpcurl can drive the router into producing a CSR, and OpenSSL can sign it. Step 5 then fails with a specific, expected error because grpcurl cannot keep a Rotate stream open across the external signing pause that this flow needs. The complete walk-through is included so operators considering Flow A see exactly what works, where it fails, and why. Readers who only want a working Rotation should jump to Flow B — Client-generated Key and CSR.

The router owns the Private Key. Ask it for a CSR, sign the CSR with the Intermediate CA, upload only the signed leaf (no Private Key in UploadRequest), and finalize.

Step 1 — Ensure Profile exists

grpcurl -plaintext -import-path "$GNSI_IMPORT" -proto "$PROTO_CERTZ" \
 "${GRPC_MD[@]}" \
 -d '{"sslProfileId":"lab-profile-1"}' \
 "${XR}" gnsi.certz.v1.Certz.AddProfile || true

Step 2 — Request CSR and capture PEM

grpcurl limitation: In a production workflow, Steps 2–5 happen inside a single Rotate stream — the client sends generateCsr, receives the CSR, signs it, uploads the signed leaf, and finalizes without closing the connection. grpcurl cannot pause mid-stream for external signing, so the stream closes after the CSR is returned. The commands below work around this by running two separate Rotate calls. A programmatic client (Python, Go) handles the full flow in one stream.

grpcurl -plaintext -import-path "$GNSI_IMPORT" -proto "$PROTO_CERTZ" \
 "${GRPC_MD[@]}" \
 -d '@' \
 "${XR}" gnsi.certz.v1.Certz.Rotate <<'EOF'
{
 "sslProfileId": "lab-profile-1",
 "forceOverwrite": true,
 "generateCsr": {
 "params": {
 "csrSuite": "CSRSUITE_X509_KEY_TYPE_ECDSA_PRIME256V1_SIGNATURE_ALGORITHM_SHA_2_256",
 "commonName": "router.example.com",
 "san": { "dns": ["router.example.com"]}
}
}
}
{"sslProfileId": "lab-profile-1", "finalizeRotation": {}}
EOF

Router response — the CSR comes back as Base64-encoded PEM inside generatedCsr. Copy the certificateSigningRequest value into CSR_B64:

{
 "generatedCsr": {
 "certificateSigningRequest": {
 "type": "CERTIFICATE_TYPE_X509",
 "encoding": "CERTIFICATE_ENCODING_PEM",
 "certificateSigningRequest": "LS0tLS1CRUdJTi..."
 }
 }
}

Decode and inspect:

printf '%s' "$CSR_B64" | base64 -d > router.csr.pem
openssl req -in router.csr.pem -noout -text

Step 3 — Sign the CSR with the Intermediate CA

openssl x509 -req -in router.csr.pem \
 -CA lab-pki/intermediate/intermediate.crt \
 -CAkey lab-pki/intermediate/intermediate.key \
 -CAcreateserial \
 -out router.crt -days 825 -sha256

Step 4 — Base64 the signed leaf PEM

LEAF_B64=$(base64 < router.crt | tr -d '\n')

Step 5 — Rotate: upload the signed leaf, then finalize

In a single-stream client, the leaf upload would land on the same connection that just produced the CSR — the router still has the candidate Key in memory and accepts a leaf-only certificateChain. Because grpcurl had to close the stream after Step 2, the candidate Key was discarded along with the rolled-back transaction. Re-uploading the signed leaf in a second grpcurl invocation therefore fails:

VER="lab-rotate-$(date +%s)"
CREATED="$(date +%s)"
grpcurl -plaintext -import-path "$GNSI_IMPORT" -proto "$PROTO_CERTZ" \
 "${GRPC_MD[@]}" \
 -d '@' \
 "${XR}" gnsi.certz.v1.Certz.Rotate <<EOF
{
 "sslProfileId": "lab-profile-1",
 "forceOverwrite": true,
 "certificates": {
 "entities": [{
 "version": "${VER}",
 "createdOn": ${CREATED},
 "certificateChain": {
 "certificate": {
 "type": "CERTIFICATE_TYPE_X509",
 "encoding": "CERTIFICATE_ENCODING_PEM",
 "rawCertificate": "${LEAF_B64}"
}
}
}]
}
}
{"sslProfileId": "lab-profile-1", "finalizeRotation": {}}
EOF

Router response on IOS XR 24.4.x:

ERROR:
 Code: InvalidArgument
 Message: No private key found for leaf cert

Adding "keySource": "KEY_SOURCE_GENERATED" to the leaf certificate does not rescue the flow — the candidate Key is gone, so the router still rejects the upload with the same error. This is the practical reason most operators run the client-generated flow (Flow B below) when working through grpcurl. Flow A only completes end-to-end from a programmatic client (Python, Go) that keeps a single Rotate stream open across the CSR / sign / upload / finalize steps.

For the Trust Bundle pattern (mutual TLS), see Step 4b in Flow B — the same nested layout works end-to-end from grpcurl.

Step 6 — Where to go from here

• For a working grpcurl walk-through, switch to Flow B below.
• For a production Flow A implementation, use a programmatic client. The OpenConfig gNSI repo includes a Go reference client that keeps the Rotate stream open across CSR retrieval, external signing, leaf upload, and finalize.

---

Flow B — Client-generated Key and CSR (full chain upload)

This flow matches what many operators already do with OpenSSL: create server.key, build a CSR, sign it with the Intermediate CA, then push leaf, Key, and issuer chain through Certz in one nested certificateChain. After finalize, bind the Profile with IOS XR CLI and confirm with show grpc certificate.

In certz.proto, certificateChain.certificate holds the leaf PEM plus rawPrivateKey for the server Authentication material. Each issuer sits under parent, all the way to the Root, which omits parent. Optional trustBundle Entities reuse the same nesting for publishing CA material to validate connecting clients (mutual TLS).

Note: In production the Root Key stays offline; only the Intermediate should sign day-to-day leaf Certificates.

Step 0 — Set the Profile name

If the Environment block is already exported, add only the Profile name:

export SSL_PROFILE_ID="${SSL_PROFILE_ID:-lab-profile-1}"

Step 1 — Create or reuse the SSL Profile

grpcurl -plaintext -import-path "$GNSI_IMPORT" -proto "$PROTO_CERTZ" \
 "${GRPC_MD[@]}" \
 -d "{\"sslProfileId\":\"${SSL_PROFILE_ID}\"}" \
 "${XR}" gnsi.certz.v1.Certz.AddProfile || true

Step 2 — Key, CSR, and signed leaf on the workstation

Finish the lab PKI section first so lab-pki/intermediate/ and lab-pki/root/ exist:

openssl genrsa -out server.key 2048
openssl req -new -key server.key -out server.csr \
 -subj "/CN=router.example.com/O=Lab/C=US" \
 -addext "subjectAltName=DNS:router.example.com"
openssl x509 -req -in server.csr \
 -CA lab-pki/intermediate/intermediate.crt \
 -CAkey lab-pki/intermediate/intermediate.key \
 -CAcreateserial \
 -out server.crt -days 825 -sha256 -copy_extensions copy

Verify the leaf against the chain:

openssl verify -CAfile lab-pki/root/root.crt \
 -untrusted lab-pki/intermediate/intermediate.crt \
 server.crt

If -copy_extensions copy is unavailable on an older OpenSSL build, use -extfile with a SAN snippet.

Step 3 — Base64 the PEM files for JSON

CERT_B64=$(base64 < server.crt | tr -d '\n')
KEY_B64=$(base64 < server.key | tr -d '\n')
INT_B64=$(base64 < lab-pki/intermediate/intermediate.crt | tr -d '\n')
ROOT_B64=$(base64 < lab-pki/root/root.crt | tr -d '\n')
VER="lab-chain-$(date +%s)"
CREATED="$(date +%s)"
VER_TRUST="${VER}-trust"

Step 4 — Rotate: upload the nested chain, then finalize

Two JSON objects through grpcurl with -d '@': the first carries certificates with a nested certificateChain, the second finalizes. Repeat sslProfileId on both objects for IOS XR builds that enforce it.

The Intermediate is the first parent under the leaf; the Root is the parent of that Intermediate. Only the leaf object includes rawPrivateKey.

grpcurl -plaintext -import-path "$GNSI_IMPORT" -proto "$PROTO_CERTZ" \
 "${GRPC_MD[@]}" \
 -format json -d '@' \
 "${XR}" gnsi.certz.v1.Certz.Rotate <<EOF
{
 "sslProfileId": "${SSL_PROFILE_ID}",
 "forceOverwrite": true,
 "certificates": {
 "entities": [{
 "version": "${VER}",
 "createdOn": ${CREATED},
 "certificateChain": {
 "certificate": {
 "type": "CERTIFICATE_TYPE_X509",
 "encoding": "CERTIFICATE_ENCODING_PEM",
 "rawCertificate": "${CERT_B64}",
 "rawPrivateKey": "${KEY_B64}"
},
 "parent": {
 "certificate": {
 "type": "CERTIFICATE_TYPE_X509",
 "encoding": "CERTIFICATE_ENCODING_PEM",
 "rawCertificate": "${INT_B64}"
},
 "parent": {
 "certificate": {
 "type": "CERTIFICATE_TYPE_X509",
 "encoding": "CERTIFICATE_ENCODING_PEM",
 "rawCertificate": "${ROOT_B64}"
}
}
}
}
}]
}
}
{"sslProfileId": "${SSL_PROFILE_ID}", "finalizeRotation": {}}
EOF

Router response:

{
 "certificates": {}
}
{}

Step 4b — Optional: add a Trust Bundle Entity

Add this when the Profile must also publish CA material for validating connecting clients. VER_TRUST was prepared in Step 3. The entities array becomes two elements:

grpcurl -plaintext -import-path "$GNSI_IMPORT" -proto "$PROTO_CERTZ" \
 "${GRPC_MD[@]}" \
 -format json -d '@' \
 "${XR}" gnsi.certz.v1.Certz.Rotate <<EOF
{
 "sslProfileId": "${SSL_PROFILE_ID}",
 "forceOverwrite": true,
 "certificates": {
 "entities": [
 {
 "version": "${VER}",
 "createdOn": ${CREATED},
 "certificateChain": {
 "certificate": {
 "type": "CERTIFICATE_TYPE_X509",
 "encoding": "CERTIFICATE_ENCODING_PEM",
 "rawCertificate": "${CERT_B64}",
 "rawPrivateKey": "${KEY_B64}"
},
 "parent": {
 "certificate": {
 "type": "CERTIFICATE_TYPE_X509",
 "encoding": "CERTIFICATE_ENCODING_PEM",
 "rawCertificate": "${INT_B64}"
},
 "parent": {
 "certificate": {
 "type": "CERTIFICATE_TYPE_X509",
 "encoding": "CERTIFICATE_ENCODING_PEM",
 "rawCertificate": "${ROOT_B64}"
}
}
}
}
},
 {
 "version": "${VER_TRUST}",
 "createdOn": ${CREATED},
 "trustBundle": {
 "certificate": {
 "type": "CERTIFICATE_TYPE_X509",
 "encoding": "CERTIFICATE_ENCODING_PEM",
 "rawCertificate": "${INT_B64}"
},
 "parent": {
 "certificate": {
 "type": "CERTIFICATE_TYPE_X509",
 "encoding": "CERTIFICATE_ENCODING_PEM",
 "rawCertificate": "${ROOT_B64}"
}
}
}
}
]
}
}
{"sslProfileId": "${SSL_PROFILE_ID}", "finalizeRotation": {}}
EOF

Router response:

{
 "certificates": {}
}
{}

If nesting errors persist, run grpcurl -import-path "$GNSI_IMPORT" -proto "$PROTO_CERTZ" -msg-template describe gnsi.certz.v1.CertificateChain and mirror the field names. The template prints the proto’s deprecated names (certificate, privateKey); the wire-compatible names used above (rawCertificate, rawPrivateKey) belong to the modern oneof variants and are what every recent IOS XR build accepts.

Step 5 — Activate the Profile (IOS XR CLI)

configure
 grpc gnsi service certz ssl-profile-id lab-profile-1
 commit
end

Swap lab-profile-1 for ${SSL_PROFILE_ID} if a different name was exported.

Step 6 — Verify chain and Authentication material

show grpc certificate

Confirm subject, issuer, validity dates, and that the chain shows the Intermediate as the leaf issuer and the Root at the Trust Anchor. Representative output:

RP/0/RP0/CPU0:router# show grpc certificate
Tue Apr 22 18:40:11.218 UTC
SSL Profile: lab-profile-1
 Certificate:
 Subject: CN=router.example.com, O=Lab, C=US
 Issuer: CN=Lab Issuing CA, O=Lab, C=US
 Validity:
 Not Before: Apr 22 18:32:01 2026 GMT
 Not After: Jul 26 18:32:01 2028 GMT
 CA chain:
 Subject: CN=Lab Issuing CA, O=Lab, C=US
 Issuer: CN=Lab Root CA, O=Lab, C=US
 Subject: CN=Lab Root CA, O=Lab, C=US
 Issuer: CN=Lab Root CA, O=Lab, C=US

Once the Profile is bound, the gNMI state subtree also populates:

gnmic -a "${XR}" --insecure -u "${XR_USER}" -p "${XR_PASS}" -e json_ietf \
 get --path "/system/grpc-servers/grpc-server[name=DEFAULT]/state/ssl-profile-id" \
 --path "/system/grpc-servers/grpc-server[name=DEFAULT]/state/certificate-version"

The reply now returns the bound Profile ID and the version string from the most recent successful Rotate.

Step 7 — Optional cleanup with DeleteProfile

grpcurl -plaintext -import-path "$GNSI_IMPORT" -proto "$PROTO_CERTZ" \
 "${GRPC_MD[@]}" \
 -d "{\"sslProfileId\":\"${SSL_PROFILE_ID}\"}" \
 "${XR}" gnsi.certz.v1.Certz.DeleteProfile

Only run cleanup when the Profile is not bound to the live gRPC server.

Minimal variant (leaf + Key only) — not validated in this lab

If every peer already trusts the issuers out of band, uploading only the leaf and Key inside certificateChain with no parent nodes is permitted by the proto. This article was validated end-to-end with the nested parent layout from Step 4; if the minimal variant is needed in production, confirm the exact behavior on the target IOS XR release before relying on it.

---

grpcurl Debugging Aids

The -msg-template flag combined with describe prints the canonical JSON shape for any message — handy when chasing nesting errors:

grpcurl -import-path "$GNSI_IMPORT" -proto "$PROTO_CERTZ" \
 -msg-template describe gnsi.certz.v1.RotateCertificateRequest
grpcurl -import-path "$GNSI_IMPORT" -proto "$PROTO_CERTZ" \
 -msg-template describe gnsi.certz.v1.UploadRequest
grpcurl -import-path "$GNSI_IMPORT" -proto "$PROTO_CERTZ" \
 -msg-template describe gnsi.certz.v1.CertificateChain

The template uses the proto’s deprecated field names (certificate, privateKey); the modern oneof arms rawCertificate and rawPrivateKey are what live IOS XR builds accept on the wire — that is the form used throughout this article.

---

Pointing gRPC at a Certz Profile (IOS XR CLI)

Starting in IOS XR 24.1.1, bind the gRPC server to a Certz Profile:

grpc gnsi service certz ssl-profile-id <ssl-profile-name>

Precedence matters: when Certz is not active, IOS XR falls back to gNOI cert.proto; when neither is configured, legacy TLS trustpoint data applies. Full details are in the gRPC Authentication guide — Certz chapter.

show grpc certificate

---

Monitoring with YANG Telemetry

Once an SSL Profile is deployed, the OpenConfig YANG model openconfig-gnsi-certz (v0.5.0) exposes Credential metadata and TLS handshake Counters that Telemetry can stream in real time.

All YANG paths below are relative to /system/grpc-servers/grpc-server[name=<server>].

Credential metadata:

YANG path (relative to base)	What it tells
.../state/certificate-version	Version string of the active Certificate
.../state/certificate-created-on	When the active Certificate was installed (nanoseconds since epoch)
.../state/ca-trust-bundle-version	Version string of the active Trust Bundle
.../state/ca-trust-bundle-created-on	When the active Trust Bundle was installed
.../state/certificate-revocation-list-bundle-version	Version string of the active CRL Bundle
.../state/certificate-revocation-list-bundle-created-on	When the active CRL Bundle was installed
.../state/authentication-policy-version	Version of the Authentication Policy (mutual TLS)
.../state/authentication-policy-created-on	When the Authentication Policy was installed
.../state/ssl-profile-id	Which SSL Profile ID this gRPC server is using

TLS handshake Counters:

YANG path (relative to base)	What it tells
.../state/certz-counters/access-accepts	Total successful TLS handshakes
.../state/certz-counters/last-access-accept	Timestamp of the most recent successful handshake
.../state/certz-counters/access-rejects	Total failed TLS handshakes (client denied)
.../state/certz-counters/last-access-reject	Timestamp of the most recent TLS rejection

Subscribe to access-rejects and last-access-reject to detect Authentication failures in production. A sudden spike in rejects after a Rotate usually means the new Certificate or Trust Bundle has an issue — check certificate-version and ca-trust-bundle-version to correlate with the material that was just pushed.

The *-version and *-created-on leaves mirror the exact strings passed in each Entity during the Rotate RPC. This means Telemetry pipelines can correlate gRPC behavior with the same identifiers used over the wire — no need to open PEM files on disk.

Quick Telemetry check with gnmic

On 25.3.1: the Certz state subtree only populates after the gRPC server is bound to a profile (grpc gnsi service certz ssl-profile-id <name>). Until that CLI binding is committed, state/ssl-profile-id and state/certz-counters return NotFound, and the active gRPC server keeps using the legacy/default TLS material even after a successful Rotate.

Verify the active Certificate version and TLS handshake Counters with a gNMI Get (gnmic):

gnmic -a "${XR}" --insecure -u "${XR_USER}" -p "${XR_PASS}" -e json_ietf \
 get --path "/system/grpc-servers/grpc-server/state/certificate-version" \
 --path "/system/grpc-servers/grpc-server/state/ca-trust-bundle-version" \
 --path "/system/grpc-servers/grpc-server/state/certz-counters/access-rejects"

Per-RPC AAA counters from the openconfig-gnsi-authz model are populated regardless and are a quick way to confirm Certz traffic is even reaching the server:

gnmic -a "${XR}" --insecure -u "${XR_USER}" -p "${XR_PASS}" -e json_ietf \
 get --path '/system/grpc-servers/grpc-server[name=DEFAULT]'

Filter the response with jq for just the Certz RPCs:

jq '[.[0].updates[0].values."system/grpc-servers/grpc-server"
."openconfig-gnsi-authz:authz-policy-counters".rpcs.rpc[]
 | select(.name | startswith("/gnsi.certz.v1.Certz/"))]'

Expected output (numbers reflect the lab walk-through):

[
 { "name": "/gnsi.certz.v1.Certz/AddProfile", "state": { "access-accepts": "4",... } },
 { "name": "/gnsi.certz.v1.Certz/CanGenerateCSR", "state": { "access-accepts": "3",... } },
 { "name": "/gnsi.certz.v1.Certz/DeleteProfile", "state": { "access-accepts": "3",... } },
 { "name": "/gnsi.certz.v1.Certz/GetProfileList", "state": { "access-accepts": "4",... } },
 { "name": "/gnsi.certz.v1.Certz/Rotate", "state": { "access-accepts": "11",... } }
]

GetIntegrityManifest is missing from the list — IOS XR returns Unimplemented before the AAA counter increments, so the RPC simply never appears in the gNMI tree.

---

Things to watch out for

• On IOS XR 24.4.x, repeat sslProfileId on every stream message — including finalize — or the server rejects the RPC with Not expecting rotate RPC for multiple SSL profiles in single stream.
• The Profile bound to the active gRPC server cannot be deleted. DeleteProfile returns an error for it.
• GetIntegrityManifest may return Unimplemented on some images. Always test with a live call, not just Reflection.
• After Rotate, the new Certificate takes effect for new connections. Existing sessions are not interrupted.
• Always verify with show grpc certificate after binding a Profile — a successful Rotate does not guarantee the Profile is active on the gRPC server.
• Flow A (target-generated CSR) cannot complete end-to-end in grpcurl — closing the stream after the CSR returns discards the candidate Key, and the follow-up upload fails with No private key found for leaf cert. Use a programmatic client for that flow, or stick with Flow B.
• IOS XR returns InvalidArgument: certificate: same version sent (not the proto-documented AlreadyExists) when reusing a version without forceOverwrite. Set "forceOverwrite": true on every non-finalize message during retries.
• An unfinalized Rotate stream closes with ERROR: Code: Unknown / Message: EOF from the server’s perspective. That is the rollback signal, not a failure to investigate.
• The openconfig-gnsi-certz state subtree (state/ssl-profile-id, state/certz-counters) only populates after the gRPC server is bound to a Certz Profile. Until then, gNMI Gets on those leaves return NotFound.

Conclusion

Adopting gNSI Certz on IOS XR aligns gRPC TLS Rotation with the same stage-validate-finalize model as the other gNSI services. The Protobuf contract is small on purpose, but the nested Certificate upload format benefits from grpcurl -msg-template before automating at scale. Combined with Authz and Pathz, the coverage spans secure, programmable Management sessions end to end.

• Certz groups TLS Authentication material, Trust Anchors, CRLs, and optional Policy into named SSL Profiles on IOS XR, backed by EMSd under /misc/config/grpc/gnsi/certz/ssl_profiles/.
• Unary RPCs cover discovery (GetProfileList), Profile creation (AddProfile), CSR capability checks (CanGenerateCSR), and teardown (DeleteProfile).
• Rotate is a streamed Transaction: optional generate_csr, one or more certificates uploads, then finalize_rotation. Drop the stream early to trigger Rollback.
• Two practical flows: let the router generate the CSR when canGenerate is true, or generate Key plus CSR on a workstation and upload the signed chain with rawPrivateKey on the leaf.
• After finalize, bind the Profile with grpc gnsi service certz ssl-profile-id and confirm with show grpc certificate.
• YANG Telemetry via openconfig-gnsi-certz streams Certificate versions, Trust Bundle freshness, and TLS handshake Counters — subscribe to access-rejects to catch Authentication failures early.

What to do next: To rotate SSH Credentials alongside gRPC TLS, continue with Credentialz. To add gRPC Service-level Authorization, see Authz.

---

Reverting / Undo

Situation	How to revert
Cancel a Rotate mid-stream	Send a certificates message but no finalizeRotation, then close the grpcurl stream. On IOS XR 24.4.x the server prints ERROR: Code: Unknown / Message: EOF, the Candidate is discarded, and the previous material stays active. The proof: re-running the same Rotate (same version, without forceOverwrite) succeeds — which would not be possible if the Candidate had been kept.
Revert to a different Profile	Re-run grpc gnsi service certz ssl-profile-id <old-profile> in IOS XR CLI to point the gRPC server back.
Revert to legacy (non-Certz) TLS	Remove the grpc gnsi service certz configuration line entirely. The gRPC server falls back to gNOI cert.proto material, then to legacy trustpoint PEM paths.
Delete a test Profile	DeleteProfile RPC — only works if the Profile is not bound to the active gRPC server.
Replace bad Certificate material	Run another Rotate with forceOverwrite: true and the corrected material. No need to delete first.

---

References

1. OpenConfig — certz.proto, certz/README.md
2. OpenConfig — openconfig-gnsi-certz YANG model v0.5.0
3. Cisco IOS XR — Programmability Configuration Guide: gRPC Authentication — Manage Certificates using Certz.proto
4. Cisco IOS XR — Programmability Command Reference: gRPC commands
5. XRdocs — Introduction to gNSI (Authz), PathZ

---