SSH: “WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!” and “Host key verification failed”

You run ssh to a machine you have connected to before, and instead of a login prompt you get a wall of warning:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ED25519 key sent by the remote host is
SHA256:oBQGFSNvmhWP4k11cssoXC8MSYwvgiiwWSdnzeeizcc.
Please contact your system administrator.
Add correct host key in /home/you/.ssh/known_hosts to get rid of this message.
Offending ED25519 key in /home/you/.ssh/known_hosts:1
Host key for [127.0.0.1]:2222 has changed and you have requested strict checking.
Host key verification failed.

ssh has stopped before sending your password or key. That is the intended behaviour, not a malfunction. This article explains what it is checking, how to decide whether the change is legitimate, and how to remove the old entry without throwing away the rest of your known servers.

Applies to: the OpenSSH client on Linux, macOS and Windows (including the ssh.exe that ships with Windows 10 and 11). The wording of the last lines varies a little between versions. The commands are the same.

What is going on

Every SSH server has a host key, a key pair generated when OpenSSH was installed on it. The first time you connect, ssh shows you the key’s fingerprint, asks whether to trust it, and saves the public key in ~/.ssh/known_hosts against the name or address you typed. From then on, on every connection, ssh compares the key the server presents with the saved one. If they differ, it refuses, because the one thing that should never change on a server you trust is the key proving which server it is.

A changed key has two possible explanations, and the warning is honest about both:

• The server really changed. This is by far the most common. It was reinstalled or rebuilt, restored from an image, replaced by new hardware (a Raspberry Pi with a fresh SD card is the classic), a cloud VM was recreated and got the old IP address, a cloned VM regenerated its keys, or you have reached a different machine behind a load balancer or a reused DHCP address.
• Something is impersonating the server. A machine in the middle is presenting its own key (a man-in-the-middle attack). Much rarer, but it is the reason the check exists.

ssh cannot tell those apart. You have to. That is the whole job of the diagnosis below.

Diagnosis

1. Was there a reason for the key to change?

Before you touch anything, ask whether you or someone you work with did anything to that server since you last connected: reinstall, restore from backup or snapshot, rebuild from an image, hardware swap, moved the IP, rebuilt the VM in the cloud console. If yes, the explanation is probably innocent, so go to step 2 to prove it. If nothing happened, treat the warning as real and find out before you continue. Check with whoever runs the server, and connect from a different network if you can.

2. Read the warning for the file and line

Two lines in the warning tell you where the old key lives and which key the server presented:

The fingerprint for the ED25519 key sent by the remote host is
SHA256:oBQGFSNvmhWP4k11cssoXC8MSYwvgiiwWSdnzeeizcc.
...
Offending ED25519 key in /home/you/.ssh/known_hosts:1

The path and number after “Offending” are the file and the line to remove (line 1 here). The first fingerprint is what the server is presenting right now. Write it down. If the file is /etc/ssh/ssh_known_hosts and not your own, someone set that up system-wide, see Notes.

To see what you have saved for that name, search for it, and print the fingerprint of the saved key so you can compare it with the one above:

ssh-keygen -F server1.example.com
ssh-keygen -F "[192.168.1.50]:2222"
ssh-keygen -lf ~/.ssh/known_hosts

In the test, ssh-keygen -F printed # Host [127.0.0.1]:2222 found: line 1 and the saved key, and ssh-keygen -lf printed a different SHA256: fingerprint from the one in the warning. That mismatch is the problem in one line.

3. Check the new key against the server itself

Now confirm the fingerprint the server presented belongs to the server. Do this through a route that does not rely on the connection you are suspicious of: the server’s console, the cloud provider’s web console or serial console, or a colleague who is logged in there. On the server:

ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub
ssh-keygen -lf /etc/ssh/ssh_host_ecdsa_key.pub
ssh-keygen -lf /etc/ssh/ssh_host_rsa_key.pub

Pick the line whose type matches the warning (ED25519 in the example). In the test the server-side fingerprint was SHA256:oBQGFSNvmhWP4k11cssoXC8MSYwvgiiwWSdnzeeizcc, identical to the one in the warning. If they match, the server you reached is the server you meant, and the old saved key is simply out of date.

ssh-keyscan is not proof. ssh-keyscan -t ed25519 -p 2222 host from your own machine prints the key the network hands you right now. In the test it showed the same fingerprint as the warning, but that is the same answer an impostor would give. It is useful for reading a key out so you can compare it with the console, not for deciding whether to trust it.

Root cause

The saved host key for that name (or [name]:port) in known_hosts no longer matches the key the server presents, usually because the server was reinstalled or replaced and generated a new key pair. The old entry is stale. Replacing it, after you have verified the new key, fixes the problem. Nothing is wrong with your own SSH keys or your account.

The fix

Verify first, then delete. Only remove the entry once you have done diagnosis step 3, or you know for certain the server was rebuilt. Removing the entry and accepting whatever key appears is exactly what an attacker is hoping you will do.

Remove just the one entry

ssh-keygen -R deletes every saved key for a given name from known_hosts, leaving the other lines alone. It keeps a backup of the file as known_hosts.old.

# name or address, exactly as you type it after "ssh"
ssh-keygen -R server1.example.com
ssh-keygen -R 192.168.1.50

# a server on a non-standard port (note the brackets and quotes)
ssh-keygen -R "[192.168.1.50]:2222"

The output looks like this:

# Host [127.0.0.1]:2222 found: line 1
known_hosts updated.
Original contents retained as known_hosts.old

Use the same form ssh used. If you connect on a non-standard port, ssh saves the entry as [host]:port. ssh-keygen -R host without the port finds nothing. In the test it printed Host 127.0.0.1 not found in known_hosts and still exited with status 0, so a script (or you, scanning quickly) can think it worked. Look for the “found: line N” and “known_hosts updated” lines.

It works on hashed files too. If your known_hosts shows lines starting with |1| instead of names (the HashKnownHosts option), you cannot read the host names, but ssh-keygen -R and -F still match by the plain name you give them. In the test, -R "[127.0.0.1]:2222" removed the hashed entry. The man page says this is what -R is for. Hashing is also why you cannot just open the file and delete the right line by hand.

If the warning pointed at a different file with -f, give it too: ssh-keygen -R host -f /path/to/known_hosts.

Reconnect and verify the fingerprint

ssh user@server1.example.com

Because the old entry is gone, ssh treats this as a first connection and shows the prompt:

The authenticity of host '[127.0.0.1]:2222 ([127.0.0.1]:2222)' can't be established.
ED25519 key fingerprint is: SHA256:oBQGFSNvmhWP4k11cssoXC8MSYwvgiiwWSdnzeeizcc
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])?

Compare that fingerprint with the one you verified on the server in step 3. If they match, type yes. If they do not, type no and stop. (You can also paste the expected fingerprint at the prompt instead of yes, which only accepts a matching key.)

Verification

# the new key is saved, with the fingerprint you checked
ssh-keygen -F server1.example.com
ssh-keygen -lf ~/.ssh/known_hosts

# and the connection works with no warning
ssh user@server1.example.com

The login should go straight through with no banner. In the test, after -R and a reconnect, known_hosts held a single new line for the server and the old key was gone.

Notes

• Do not delete the whole known_hosts file. It removes the saved fingerprint of every server you have verified, and you will accept the next ones blind.
• Do not set StrictHostKeyChecking no. The ssh_config man page says it lets ssh connect to hosts with changed keys, which switches off this check for everything. The related accept-new setting is safer for new servers (it adds unknown hosts automatically) but it still refuses a changed key, so it will not make this error go away.
• Two names, two entries. The entry is saved under the name you typed. If you sometimes connect by hostname and sometimes by IP address, each has its own entry, and you may need to run ssh-keygen -R for both.
• The backup file keeps readable names. After hashing or removing entries, known_hosts.old still holds the original lines. If you hash on purpose for privacy, delete the .old file when you are done.
• System-wide file. If the warning says “Offending … key in /etc/ssh/ssh_known_hosts”, the entry belongs to the whole machine. Removing it needs root: sudo ssh-keygen -R host -f /etc/ssh/ssh_known_hosts. Do that only if you manage that machine.
• Rebuilding servers often? Restore the same host keys from backup when you rebuild a server (copy /etc/ssh/ssh_host_* across), or use SSH certificates, so the key does not change and you never see this warning for a rebuild.

Can’t explain why the key changed? Post the warning (hide anything private) and what you know about the server in The Patch Panel and we will work through it. The best answer earns points on Top of the Stack.