Down and out: SSH 'Connection refused' and 'Permission denied', step-by-step guide

Anyone managing a server discovers it always at the worst possible moment: your terminal responds with Connection refused or Permission denied (publickey) and the site or control panel becomes unreachable. First rule: don't touch sshd_config until you understand which of the two errors you're facing, because the causes and remedies are completely different.

What the two errors mean

Connection refused means the packet reached the machine (or a firewall in front of it) but no process answered on that port: either the port is wrong, or the firewall is blocking it, or sshd isn't running. Permission denied (publickey) means the opposite: the server answered, the TCP connection succeeded, but it rejected the key or password you offered.

Connection refused: causes in order

1. Port changed. If someone moved SSH from port 22 to another port, the command to run (from your local system) is ssh -p 2222 user@host, replacing 2222 with the correct port. To see which port sshd is listening on, from the server: sudo ss -tlnp | grep sshd.

2. Firewall or security group. On Ubuntu with UFW: sudo ufw status verbose should show the SSH port in ALLOW. If your server is a VPS or in the cloud, also check the upstream firewall (security group, cloud firewall) in your provider's dashboard: UFW can pass and traffic can still be blocked before reaching your machine. Often the problem lies with the provider, not you: if UFW is fine and the port remains closed from outside, it's time to open a support ticket.

3. sshd service stopped. Only if you already have alternative access (provider's web console, another active user): sudo systemctl status ssh on Ubuntu/Debian, sudo systemctl status sshd on CentOS/RHEL/AlmaLinux. If it's stopped: sudo systemctl start ssh (Ubuntu) or sudo systemctl start sshd (RHEL-like), then sudo systemctl enable ssh so it starts automatically on the next reboot.

Permission denied: causes in order

1. Fail2ban has blocked you. This happens after too many failed attempts, even yours (wrong key repeated, a deployment script trying with the wrong user). The default values from the fail2ban project, read from the official configuration file, are: a 10-minute ban after 5 failed attempts within a 10-minute window. To check from the emergency console: sudo fail2ban-client status sshd. To unblock your IP immediately: sudo fail2ban-client set sshd unbanip YOUR.IP.HERE.

2. Wrong key or .ssh permissions too open. From your client, to see which key is being offered and where it stops: ssh -vvv user@host 2>&1 | grep -i offering. On the server, permissions must be 700 on the ~/.ssh folder and 600 on the ~/.ssh/authorized_keys file, owned by the user logging in and not by root. To check them: stat -c "%a %U %G %n" ~ ~/.ssh ~/.ssh/authorized_keys. To fix them: chmod 700 ~/.ssh && chmod 600 ~/.ssh/authorized_keys && chown -R $(id -un):$(id -gn) ~/.ssh.

3. Disk full. A disk at 100% prevents sshd from writing temporary session files and login fails even with the correct key. From the emergency console: df -h. If root is full, free up space first (old logs, apt cache with sudo apt clean), then try normal login again.

Where to look at logs to figure out which case is yours

On Ubuntu 24.04 with rsyslog installed the file is /var/log/auth.log, otherwise use the journal: sudo journalctl -u ssh -n 50. On Debian 12 rsyslog is not installed by default, so auth.log may not exist at all: in that case you need journalctl. On CentOS/RHEL/Rocky the equivalent file is /var/log/secure and the service in the journal is called sshd: sudo journalctl -u sshd -n 50. The line that matters, in case of a fail2ban block, is the one containing "Ban" followed by the IP; in case of a rejected key it's the one with "Failed publickey" followed by the key fingerprint.

How to access via emergency console

Every VPS or cloud provider offers a web console (VNC or serial) that completely bypasses SSH and uses local login. From there you can work as if you were sitting in front of the machine: you can restart sshd, fix permissions, free up disk space or unblock fail2ban without needing the network. The name of the feature varies by provider (console, KVM, VNC console, serial console): you'll find it in the management dashboard of the individual instance.

Two precautions before touching sshd_config

First rule: never close your only active SSH session whilst modifying /etc/ssh/sshd_config. Keep that session open, open a second terminal and test the connection from there: if something goes wrong, the first session remains your safety net to revert.

Second rule: before modifying any parameter (port, PubkeyAuthentication, PermitRootLogin), verify that your provider's emergency console works and you know how to reach it. If the change breaks sshd and you don't have a safety net, the only way forward is to open a support ticket with your provider, attaching the exact time of your last successful connection, the IP you were connecting from, and the last log line you read, if you collected it before losing access.

How to verify it's fixed

After every change, test the connection from a new terminal, not from your safety session: ssh -v user@host. If it responds with a shell prompt, it's fixed. If you changed the port or firewall rules, also verify from a different network (for example from your phone on 4G) to rule out that the problem was only with your local IP.