Linux Server Hardening: A Production Checklist That Actually Holds
Most Linux servers are breached through boring, preventable mistakes — not zero-days. A weak SSH password, an open port nobody remembers, a package that was never patched, a service running as root. I have seen a production box owned within hours of going live because it shipped with password authentication on port 22 and a username of admin.
Hardening is not one tool. It is a layered system: reduce what is exposed, control who can get in, patch what remains, and watch for the rest. This is the checklist I run on every production server, and the one I would defend in a DevOps interview.
Start With the Attack Surface
Before you harden anything, find out what is actually listening:
ss -tulpn nmap -sV -p- your-server-ip Every open port is a promise you have to keep. If you do not know why a port is open, close it. A database that only the app server needs should never be reachable from the internet. A management panel should never sit on a public port without IP allowlisting or a VPN in front of it.
My rule: a production server exposes SSH (key-only, non-standard port optional), 80 and 443, and nothing else by default. Everything else earns its port with a written reason.
SSH: The Front Door
SSH is the first thing attackers test, so it gets the most attention.
Keys only, no passwords
# /etc/ssh/sshd_config PasswordAuthentication no PermitRootLogin no PubkeyAuthentication yes AllowUsers deploy MaxAuthTries 3 Then reload SSH — and test in a second session before you close the first one. Locking yourself out of your own server is the classic hardening self-own.
Key hygiene
- One key per person, per device. Shared keys mean you can never revoke one person.
- Ed25519 keys:
ssh-keygen -t ed25519 -C "name@device". - Protect private keys with a passphrase and store them in a password manager or hardware key, never in a repo or a chat.
- Rotate keys when someone leaves. Immediately, not "next sprint".
Fail2ban: stop the noise
sudo apt install fail2ban sudo systemctl enable --now fail2ban Fail2ban watches your logs and bans IPs after repeated failures. It does not stop a determined attacker, but it kills the automated scanners that hammer port 22 all day. Check it is actually banning: sudo fail2ban-client status sshd.
Firewall: Default Deny
A firewall is not optional, even on a cloud VM with security groups. Defence in depth means the host protects itself too.
sudo ufw default deny incoming sudo ufw default allow outgoing sudo ufw allow 22/tcp sudo ufw allow 80/tcp sudo ufw allow 443/tcp sudo ufw enable On a server with nftables or firewalld, the principle is identical: default deny, allow only what the server's job requires, log the drops. Test from outside after every change — a firewall rule you never tested is a hope, not a control.
Patching: The Unsexy Thing That Saves You
Most breaches exploit known vulnerabilities with patches already available. The fix is boring and relentless:
sudo apt install unattended-upgrades sudo dpkg-reconfigure -plow unattended-upgrades Unattended upgrades apply security patches automatically. For kernel updates that need a reboot, schedule a maintenance window and use needrestart or livepatch where it fits — but never let "we can't reboot" become "we haven't patched in a year".
My monthly routine: check apt list --upgradable, review what is pending, apply, reboot if the kernel changed, and verify the service came back up. Patching without verification is half a job.
Users, Sudo, and Least Privilege
- No shared accounts. Every human gets their own user, their own key, and their own sudo entry.
- Sudo should require a password (
Defaults timestamp_timeout=15), and sudoers changes should go through version control, not a quick edit on the box. - Service accounts run services, not shells:
usermod -s /usr/sbin/nologin appuser. - Never run an application as root. If a web app needs port 80, use a reverse proxy or capabilities (
setcap), not root. - Audit who has sudo:
getent group sudo— and remove anyone who should not be there.
File Permissions and the Filesystem
chmod 700 ~/.ssh chmod 600 ~/.ssh/authorized_keys chmod 600 /etc/ssh/ssh_host_*_key Private keys with loose permissions are a breach waiting to happen. Beyond that: mount /tmp with noexec,nosuid,nodev, keep /var on its own partition so logs cannot fill the root disk, and set umask 027 for service accounts so new files are not world-readable by default.
Kernel and System Hardening
A few sysctl settings close whole classes of attack:
# /etc/sysctl.d/99-hardening.conf net.ipv4.ip_forward = 0 net.ipv4.conf.all.accept_redirects = 0 net.ipv4.conf.all.send_redirects = 0 net.ipv4.conf.all.rp_filter = 1 net.ipv4.tcp_syncookies = 1 kernel.dmesg_restrict = 1 kernel.kptr_restrict = 2 fs.suid_dumpable = 0 Apply with sysctl --system. These are not magic, but they remove easy wins for attackers: IP spoofing, redirect abuse, and information leaks from kernel messages.
Disable what you do not use: unused kernel modules, unused services (systemctl list-unit-files --state=enabled), and legacy protocols. Every running service is code an attacker can reach.
Logging and Auditing: Prove It Happened
If you cannot see it, you cannot respond to it.
- Ship logs off the box: rsyslog or journald forwarding to a central log server. An attacker who owns the box will delete local logs first.
- Enable auditd for the events that matter: sudo use, authentication failures, changes to
/etc/passwdand SSH keys. - Set up alerting on the signals that matter: failed SSH logins spiking, a new user created, sudo used at 3 AM, a service stopping unexpectedly.
- Check your logs regularly — or better, have a tool check them and page you. A log nobody reads is a diary, not a control.
Tools that help: auditd, aide or tripwire for file integrity, lynis for a periodic hardening audit (lynis audit system), and your existing Prometheus/Grafana stack for the metrics that show something changed.
Backups: Hardening's Last Line
Hardening reduces the chance of a breach. Backups decide whether a breach — or ransomware — is a bad day or a business-ending event.
- Backups must be off the server, versioned, and tested. A backup you have never restored is a rumour.
- Test restores quarterly, at minimum. Restore to a clean VM, verify the service starts, and time it.
- Keep at least one copy the server itself cannot delete — immutable storage or a separate account with write-only access.
Production Pitfalls Checklist
The failures I actually see, in order of frequency:
- Password auth still on. The single most common breach vector. Keys only.
- Root login allowed.
PermitRootLogin no— always. - No firewall on the host. "The cloud security group handles it" is not a host control.
- Unpatched for months. Known CVEs, public exploits, no patch window.
- One shared SSH key for the whole team. You cannot revoke one person without breaking everyone.
- Services running as root. A web app bug becomes a full system compromise.
- No fail2ban, no rate limiting. Automated scanners get unlimited tries.
- Logs only on the box. After a breach, the evidence is gone with the attacker.
- Backups never tested. The restore fails exactly when you need it most.
- Hardening done once, never revisited. A new service, a new user, a new port — and the checklist is out of date.
What Hiring Managers Actually Ask
If you are targeting remote DevOps or sysadmin roles, this topic comes up because it reveals how you think about operations:
- "A server was breached — walk me through your first hour." (Answer: isolate it, preserve logs and disk image, check what changed and how they got in, rotate credentials, then rebuild — never "clean it and put it back".)
- "How do you harden 50 servers consistently?" (Answer: configuration management — Ansible, Terraform, or an image pipeline — never 50 manual checklists you cannot audit.)
- "How do you know a server was tampered with?" (Answer: file integrity monitoring, centralised logs, and alerts on the events that should never happen.)
The answer they want is never "I ran a hardening script once." It is the system around it: automation, least privilege, patching, monitoring, and a tested recovery path.
The Monthly Hardening Routine
My monthly routine takes twenty minutes:
-
ss -tulpn— anything listening that should not be? -
getent group sudoandlastlog— who has access, who actually logged in? -
apt list --upgradable— patch, reboot if needed, verify services. -
fail2ban-client statusand firewall status — are the controls actually running? -
lynis audit system— what drifted since last month? - Test a backup restore — or at least verify the latest backup completed and is readable.
Do that, and hardening stops being a one-time project and becomes how the server stays safe. Security is infrastructure: boring, automated, monitored, and never a 3 AM surprise.
Your turn: What is the first thing you harden on a fresh Linux server — SSH, firewall, or patching? What bit you the first time? Tell me in the comments.


