---
name: snowflake-rotate-keypair
description: Rotate an existing RSA key pair for a Snowflake user with zero downtime using dual key slots. Use when performing key rotation via RSA_PUBLIC_KEY_2, replacing compromised keys, or scheduling regular key rotation for service users.
disable-model-invocation: true
---

# Rotate Snowflake Key Pair

Rotate an existing RSA key pair for a Snowflake user with zero downtime. Use this skill when a key pair needs to be replaced — either on a regular rotation schedule or because a key may have been compromised.

## Context

Snowflake supports two active public keys per user via the `RSA_PUBLIC_KEY` and `RSA_PUBLIC_KEY_2` properties. This allows zero-downtime key rotation: you assign the new public key to the unused slot, verify it works, then remove the old one. At no point is the user unable to authenticate.

In `permifrost.yml`, these map to the `rsa_public_key` and `rsa_public_key_2` meta fields on a user.

Reference: [Snowflake key-pair rotation docs](https://docs.snowflake.com/en/user-guide/key-pair-auth#configuring-key-pair-rotation)

## Prerequisites

- `openssl` installed on your system
- `uv` installed ([installation guide](https://docs.astral.sh/uv/getting-started/installation/))
- `op` CLI installed and signed in (see [1password skill](../../../../../security/1password/))
- The user already has a working key pair (i.e. `rsa_public_key` is set in `permifrost.yml` and deployed)
- Access to the client's 1Password vault

## Steps

**Checklist:**

- [ ] 1. Generate a new key pair
- [ ] 2. Add the new public key to `rsa_public_key_2` in `permifrost.yml`
- [ ] 3. Deploy via permifrost (PR + merge + production run)
- [ ] 4. Test authentication with the new key pair
- [ ] 5. Update 1Password with the new key pair
- [ ] 6. Swap keys: move new key to `rsa_public_key`, remove `rsa_public_key_2`
- [ ] 7. Deploy again via permifrost
- [ ] 8. Update any services using the old key
- [ ] 9. Delete local key files

### 1. Generate a new key pair

Use the [snowflake-generate-keypair](../snowflake-generate-keypair/) skill to create a new key pair:

```bash
/path/to/snowflake-generate-keypair/scripts/generate_keypair.sh \
    --client <client_name> \
    --user <username> \
    --op-vault "<client_vault>" \
    --op-title "<Client Name> - Snowflake: <username>"
```

### 2. Add the new public key as `rsa_public_key_2`

In `permifrost.yml`, add the new public key as `rsa_public_key_2` **alongside** the existing `rsa_public_key`. Do not remove the old key yet — both must be active during the transition.

```yaml
  - <username>:
      can_login: yes
      member_of:
        - <role>
      meta:
        default_role: <role>
        type: service
        rsa_public_key: |
          -----BEGIN PUBLIC KEY-----
          <OLD public key — keep this during transition>
          -----END PUBLIC KEY-----
        rsa_public_key_2: |
          -----BEGIN PUBLIC KEY-----
          <NEW public key — add this>
          -----END PUBLIC KEY-----
```

**Note:** If tundri/permifrost does not support `rsa_public_key_2` as a meta field, you can set it directly in Snowflake using SQL (requires `ACCOUNTADMIN` or `SECURITYADMIN`):

```sql
ALTER USER <USERNAME> SET RSA_PUBLIC_KEY_2='<new_public_key_content_without_header_and_footer>';
```

Strip the `-----BEGIN PUBLIC KEY-----` and `-----END PUBLIC KEY-----` delimiters and any newlines before passing to the SQL command.

### 3. Deploy via permifrost

Create a PR with the `rsa_public_key_2` addition, get it reviewed, merge, and wait for the permifrost production run to complete.

At this point, both the old and new public keys are active in Snowflake. The user can authenticate with either key pair.

### 4. Verify the new key pair against `RSA_PUBLIC_KEY_2`

Use the [test-keypair-auth](../test-keypair-auth/) skill with `SNOWFLAKE_VERIFY_KEY_SLOT=2` to confirm the new key was deployed to the correct slot. A basic auth test alone is not sufficient during rotation — it succeeds if *either* key slot matches, so it can't confirm `rsa_public_key_2` was actually deployed.

```bash
SNOWFLAKE_ACCOUNT="<account_identifier>" \
SNOWFLAKE_USER="<USERNAME>" \
SNOWFLAKE_KEY_PATH="/tmp/<client>_<username>_snowflake.p8" \
SNOWFLAKE_KEY_PASSPHRASE="<new_passphrase>" \
SNOWFLAKE_VERIFY_KEY_SLOT="2" \
uv run /path/to/gemma-skills/.../test-keypair-auth/scripts/test_keypair_auth.py
```

The script will:
1. Connect to Snowflake using the new private key (verifies auth works)
2. Compute the local key's SHA-256 fingerprint
3. Query `DESC USER` to get `RSA_PUBLIC_KEY_FP` and `RSA_PUBLIC_KEY_2_FP`
4. Confirm the fingerprint matches slot 2 specifically

**Expected output:**

```
Connection successful!
  ...

Fingerprint Verification
------------------------
  Local key fingerprint:  Azk1Pq...
  RSA_PUBLIC_KEY_FP: Xb2mNq... (no match)
  RSA_PUBLIC_KEY_2_FP: Azk1Pq... (MATCH)

  Verified: key matches RSA_PUBLIC_KEY_2
```

Only proceed to the next step if the verification passes (exit code 0).

**Note:** Fingerprint verification requires the connecting role to have permission to run `DESC USER` (typically `SECURITYADMIN` or `ACCOUNTADMIN`). If the user doesn't have this permission, connect with a privileged user separately to verify.

### 5. Update 1Password with the new key pair

Update the user's 1Password entry with the new key files and passphrase. Use the `<client>_<username>_snowflake_private_key` / `_public_key` naming convention for file fields:

```bash
op item edit "<item_id>" \
  --vault "<vault>" \
  "passphrase[password]=<new_passphrase>" \
  "<client>_<username>_snowflake_private_key[file]=./<client>_<username>_snowflake.p8" \
  "<client>_<username>_snowflake_public_key[file]=./<client>_<username>_snowflake.pub"
```

Find the item ID with:

```bash
op item list --vault "<vault>" | grep -i "<search_term>"
```

### 6. Swap keys: promote new key, remove old key

In `permifrost.yml`, replace the old `rsa_public_key` with the new public key and remove `rsa_public_key_2`:

```yaml
  - <username>:
      can_login: yes
      member_of:
        - <role>
      meta:
        default_role: <role>
        type: service
        rsa_public_key: |
          -----BEGIN PUBLIC KEY-----
          <NEW public key — moved here from rsa_public_key_2>
          -----END PUBLIC KEY-----
        # rsa_public_key_2 removed
```

If you set `RSA_PUBLIC_KEY_2` via direct SQL in step 2, unset it now:

```sql
ALTER USER <USERNAME> UNSET RSA_PUBLIC_KEY_2;
```

### 7. Deploy again via permifrost

Create a PR with the key swap, merge, and wait for the permifrost production run. After this, only the new key pair is active.

### 8. Update any services using the old key

If the rotated user is a service account (e.g. `permifrost`, `dbt`, `airflow`, `airbyte`), update the service configuration to use the new private key and passphrase:

| Service | What to update |
|---|---|
| **CI/CD (GitHub Actions)** | Update the `PERMISSION_BOT_PRIVATE_KEY_ENCODED_BASE64` and `PERMISSION_BOT_KEY_PASSPHRASE` repository secrets (see [setup-tundri-cicd](tundri-permifrost/setup-tundri-cicd/)) |
| **Local dev `.env`** | Update `PERMISSION_BOT_KEY_PATH` and `PERMISSION_BOT_KEY_PASSPHRASE` (see [setup-tundri-local-dev](tundri-permifrost/setup-tundri-local-dev/)) |
| **dbt Cloud** | Update the connection's private key and passphrase |
| **Airflow / Airbyte** | Update the connection credentials in the respective tool |

### 9. Delete local key files

```bash
rm <client>_<username>_snowflake.p8 <client>_<username>_snowflake.pub
```

## Example: Rotating the permifrost key for client Acme

```bash
# 1. Generate new key pair
/path/to/snowflake-generate-keypair/scripts/generate_keypair.sh \
    --client acme --user permifrost \
    --op-vault "Acme Corp" --op-title "Acme - Snowflake: permifrost"

# 2. Add rsa_public_key_2 to permifrost.yml (edit manually or with agent)
# 3. PR + merge + permifrost run

# 4. Verify new key matches RSA_PUBLIC_KEY_2
SNOWFLAKE_ACCOUNT="abc12345.eu-central-1" \
SNOWFLAKE_USER="PERMIFROST" \
SNOWFLAKE_KEY_PATH="./acme_permifrost_snowflake.p8" \
SNOWFLAKE_KEY_PASSPHRASE="<new_passphrase>" \
SNOWFLAKE_VERIFY_KEY_SLOT="2" \
uv run test_keypair_auth.py

# 5. Update 1Password
op item edit "<item_id>" --vault "Acme" \
  "passphrase[password]=<new_passphrase>" \
  "acme_permifrost_snowflake_private_key[file]=./acme_permifrost_snowflake.p8" \
  "acme_permifrost_snowflake_public_key[file]=./acme_permifrost_snowflake.pub"

# 6. Swap rsa_public_key and remove rsa_public_key_2 in permifrost.yml
# 7. PR + merge + permifrost run

# 8. Update CI/CD secrets (base64-encode new private key for GitHub Actions)
base64 -w 0 acme_permifrost_snowflake.p8
# Update PERMISSION_BOT_PRIVATE_KEY_ENCODED_BASE64 and PERMISSION_BOT_KEY_PASSPHRASE in repo secrets

# 9. Clean up
rm acme_permifrost_snowflake.p8 acme_permifrost_snowflake.pub
```

## Guardrails

- **Never remove the old key before verifying the new key works.** Always test authentication with the new key pair (step 4) before proceeding.
- **Both keys are active during the transition.** This is by design — it prevents authentication failures during rotation.
- **Update all services before removing the old key.** If a service still uses the old private key when `rsa_public_key` is swapped, it will fail to authenticate.
- **Do not skip the second permifrost deployment.** Leaving `rsa_public_key_2` permanently set means the old key remains valid, which defeats the purpose of rotation.

## Validation

- [ ] New key pair generated and tested successfully against Snowflake
- [ ] New key pair stored in 1Password (replacing old files and passphrase)
- [ ] `rsa_public_key` in `permifrost.yml` contains the new public key
- [ ] `rsa_public_key_2` is removed from `permifrost.yml`
- [ ] All services using this key pair have been updated
- [ ] Local key files are deleted