Secure SSH Keys: How to Generate, Rotate, and Revoke Safely

SSH keys are the silent gatekeepers of your digital infrastructure—often created once, forgotten, and left unchecked for years. When a new engineer joins your team, they might generate a key, copy it across multiple servers, and then move on after months or years. Meanwhile, their access lingers, creating hidden vulnerabilities that attackers can exploit. The solution isn’t just about adding another security layer; it’s about adopting a disciplined approach to key lifecycle management.

The consequences of neglecting SSH key management extend far beyond forgotten passwords. Unlike passwords, SSH keys lack built-in expiration, centralized dashboards, or automatic revocation mechanisms. This leads to a phenomenon known as key sprawl, where organizations end up with hundreds or even thousands of keys scattered across servers—many belonging to former employees or obsolete systems. Security teams face three critical risks: orphaned access, unknown exposure, and failed audits. Addressing these issues requires more than a new tool; it demands a consistent, organization-wide discipline.

Why SSH Key Management Fails in Most Organizations

SSH keys operate under the assumption that once they’re set up, they’ll function indefinitely. This assumption is dangerously outdated. Unlike passwords that can be reset, or OAuth tokens that expire, SSH keys often remain active long after they’re needed. Security surveys reveal that large organizations frequently have ten times more SSH keys than employees, with no clear ownership or expiration dates.

The risks associated with poor SSH key management are both immediate and long-term:

• Orphaned access: Keys tied to former employees, contractors, or decommissioned systems remain active, granting unauthorized entry.
• Unknown exposure: Without an inventory, teams cannot track which keys have access to which servers, leaving blind spots in security monitoring.
• Audit failures: Compliance reports become unreliable when there’s no verifiable record of who had access to what, and when.

The solution begins with recognizing that SSH key management isn’t a technical challenge—it’s a process challenge. Teams must shift from reactive fixes to proactive governance, ensuring that every key has a clear purpose, owner, and expiration date.

Best Practices for Generating Secure SSH Keys

The first step in securing SSH access is generating keys using modern algorithms and configurations. Legacy algorithms like DSA and RSA (with insufficient key lengths) no longer meet today’s security standards. Instead, prioritize ed25519 for new deployments, as it offers superior security with minimal performance overhead.

To generate an ed25519 key with strong encryption, use this command:

ssh-keygen -t ed25519 -a 100 -C "engineer@company.com" -f ~/.ssh/id_ed25519_work

Here’s what each flag does:

• -t ed25519: Specifies the key type (ed25519 is the gold standard).
• -a 100: Sets the number of key derivation function rounds, slowing down brute-force attempts.
• -C "engineer@company.com": Adds a comment for identification (use email or a clear label).
• -f ~/.ssh/id_ed25519_work: Defines the output file path for the key pair.

For environments constrained by legacy systems, RSA with a minimum key length of 4096 bits remains viable, though ed25519 is preferred:

ssh-keygen -t rsa -b 4096 -a 100 -C "engineer@company.com" -f ~/.ssh/id_rsa_legacy

The Case for Passphrases and ssh-agent

A common misconception is that passphrases make SSH keys impractical. In reality, passphrases encrypt the private key on disk, preventing unauthorized access if the key file is compromised. Without a passphrase, anyone who copies your private key gains immediate access to all systems it unlocks.

The workaround for the inconvenience of typing passphrases repeatedly is `ssh-agent`, a background process that temporarily stores decrypted keys in memory. To initialize and add a key to the agent:

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519_work

On macOS, the system Keychain integrates with ssh-agent, allowing you to store the passphrase securely and avoid re-entering it after each login or reboot.

One Key Per Context: Avoiding Single Points of Failure

Using a single SSH key to access every server introduces unnecessary risk. If that key is compromised, an attacker gains access to your entire infrastructure. Instead, adopt a context-based key strategy, where each key is scoped to a specific use case:

• Personal projects: id_ed25519_personal
• Work infrastructure: id_ed25519_work
• Client-specific access: id_ed25519_client_acme
• CI/CD deployments: id_ed25519_deploy (no passphrase, strict permissions)
• Production servers: id_ed25519_prod (strong passphrase, extra security)

To automate key selection, configure ~/.ssh/config to route each key to its intended destination:

Host *.internal
  IdentityFile ~/.ssh/id_ed25519_work
  IdentitiesOnly yes

Host bastion.prod.example.com
  IdentityFile ~/.ssh/id_ed25519_prod
  IdentitiesOnly yes

The IdentitiesOnly yes directive prevents SSH from attempting to use other keys in your agent, reducing the risk of unintended access attempts.

Organizing and Managing Keys Across Teams

For small to medium-sized teams, a Git-managed key registry provides a straightforward way to track and audit SSH keys. Store only public keys in a repository, ensuring private keys never leave developer workstations. Structure the registry like this:

ssh-keys/
├── users/
│   ├── alice.pub
│   ├── bob.pub
│   └── carol.pub
├── servers/
│   ├── web-prod.txt
│   ├── db-prod.txt
│   └── bastion.txt
└── deploy-keys/
    ├── github-actions.pub
    └── jenkins.pub

Each public key file should include metadata such as the owner’s email and the creation date in a comment line:

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAlhXmQp... alice@company.com 2024-05-15

Key management rules should include:

• Requiring pull requests for all key additions or removals to maintain an audit trail.
• Using a simple script to sync authorized_keys files on servers from the registry.
• Periodically reviewing the registry to identify and revoke unused keys.

Restricting Key Permissions with authorized_keys Options

Even after a key is authorized, its capabilities can be limited using authorized_keys restrictions. These restrictions are enforced server-side, independent of client actions. Examples include:

# Full access
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5... alice@company.com

# Read-only deploy key (restricted to a single command)
command="/usr/local/bin/deploy.sh",no-pty,no-agent-forwarding,no-x11-forwarding ssh-ed25519 AAAAC3NzaC1lZDI1NTE5... deploy-key

# Tunnel-only key (restricted to port forwarding)
restrict,port-forwarding,permitopen="db.internal:5432" ssh-ed25519 AAAAC3NzaC1lZDI1NTE5... tunnel-key

# IP-restricted key (limits access to a specific subnet)
from="203.0.113.0/24" ssh-ed25519 AAAAC3NzaC1lZDI1NTE5... office-access-key

These restrictions ensure that even if a key is compromised, its misuse is contained to the intended use case.

Scaling Up: Tools for Larger Teams

When managing keys across dozens of servers and hundreds of employees, manual processes no longer suffice. Solutions like HashiCorp Vault’s SSH Secrets Engine can automate key lifecycle management by acting as a certificate authority. Instead of static keys, Vault issues short-lived certificates that automatically expire, reducing the risk of orphaned access.

Another scalable approach is Teleport, an open-source tool designed for managing SSH access at scale. Teleport centralizes authentication, auditing, and revocation, providing a unified dashboard for teams to monitor and control SSH access across their infrastructure.

The transition from ad-hoc key management to a structured, automated system isn’t just about security—it’s about operational efficiency. Teams that invest in disciplined key management reduce the risk of breaches while gaining clarity and control over their infrastructure’s access landscape.

As organizations continue to expand their digital footprints, the importance of robust SSH key management will only grow. The tools and strategies outlined here provide a foundation for building a secure, scalable, and auditable access control system. By treating SSH keys with the same rigor as passwords, certificates, or tokens, teams can eliminate silent vulnerabilities and ensure their infrastructure remains protected.