Setup SSH on Linux Server: Step-by-Step Secure Connection Guide

SSH (Secure Shell) is a protocol that allows secure remote access to Linux servers. Setting up SSH on a Linux server enables system administrators and developers to manage servers remotely, execute commands, and transfer files securely. Learning to setup SSH on a Linux server is essential for anyone managing Linux systems.

In this article, we will guide you through the process of installing and configuring SSH, testing the connection, troubleshooting common issues, and following best practices for secure and efficient server access.

Prerequisites

Before setting up SSH, ensure your Linux server meets these requirements:

• Supported Linux distributions: Ubuntu, Debian, CentOS, Fedora
• User permissions: User with sudo privileges
• System updates: Keep your system updated using apt update && apt upgrade or yum update
• Network considerations: Ensure port 22 (default SSH port) is open

Having these prerequisites ensures a smooth SSH setup without connectivity or permission issues.

Setup SSH on Linux Server

SSH allows secure command-line access to Linux servers. In this section, we will cover the installation of the SSH server, starting the service, enabling it on boot, and verifying the setup.

• Installing SSH Server

For Ubuntu/Debian systems:

sudo apt update
sudo apt install openssh-server

For CentOS/Fedora systems:

sudo yum install openssh-server
sudo systemctl enable sshd
sudo systemctl start sshd

• Starting and Enabling SSH

Enable SSH service to start at boot:

sudo systemctl enable ssh    # Ubuntu/Debian
sudo systemctl enable sshd   # CentOS/Fedora
sudo systemctl start ssh

• Verifying SSH Installation

Check SSH status:

sudo systemctl status ssh    # Ubuntu/Debian
sudo systemctl status sshd   # CentOS/Fedora

Test connection locally:

ssh localhost

Configuring SSH

Configuring SSH properly improves security and ensures smooth remote access. This section explains key configuration steps to secure your Linux server.

• Editing SSH Configuration

Open the SSH configuration file:

sudo nano /etc/ssh/sshd_config

Key settings to consider:

• Change default port (Port 22) to reduce unauthorized access attempts
• Disable root login (PermitRootLogin no)
• Allow only specific users (AllowUsers youruser)
• Enable key-based authentication (PasswordAuthentication no)

• Setting Up SSH Key-Based Authentication

Generate SSH key pair on your local machine:

ssh-keygen -t rsa -b 4096

Copy the public key to the server:

ssh-copy-id user@your-server-ip

Test login:

ssh user@your-server-ip

• Restarting SSH Service

Apply configuration changes:

sudo systemctl restart ssh    # Ubuntu/Debian
sudo systemctl restart sshd   # CentOS/Fedora

Configuring Firewall

Allow SSH traffic:

sudo ufw allow 22/tcp    # Ubuntu/Debian
sudo firewall-cmd --permanent --add-service=ssh   # CentOS/Fedora
sudo firewall-cmd --reload

Troubleshooting Common Issues

Even with proper setup, SSH connections can fail. Learning to fix SSH issues in Linux ensures secure and reliable access to your server.

Common Issues and Fixes:

• Cannot Connect to Server:

Check if the SSH service is running:

sudo systemctl status ssh   # Ubuntu/Debian
sudo systemctl status sshd  # CentOS/Fedora

Verify firewall rules and port settings.

• Permission Denied (Key-based Authentication):

Check key permissions:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

• Root Login Denied:

Ensure PermitRootLogin is correctly set in /etc/ssh/sshd_config

• Connection Timeout:

Verify network connectivity and ensure the correct IP address and port are used, particularly if you have changed from the default port of 22.

By following these troubleshooting steps, most SSH connectivity issues can be resolved quickly.

Best Practices for Managing SSH on Linux

Following best practices for SSH ensures secure and efficient remote access to Linux servers. Proper management protects your server from unauthorized access and reduces security risks.

Security Measures

• Use key-based authentication instead of passwords
• Change the default SSH port to a custom one
• Limit user access with AllowUsers
• Enable two-factor authentication for added security

Monitoring and Logging

• Monitor SSH login attempts via /var/log/auth.log
• Set up fail2ban to block repeated failed login attempts

Regular Updates and Maintenance

• Keep the SSH server and system packages updated
• Periodically review user access and remove unnecessary accounts
• Backup configuration files and key pairs

By following these practices, SSH access remains secure, reliable, and efficient.

Conclusion

Learning to setup SSH on a Linux server is essential for remotely managing your systems securely and efficiently. By following this guide, you now know how to install and configure SSH, set up key-based authentication, troubleshoot common issues, and apply best practices for security and performance. SSH provides a secure and robust way to access Linux servers, making it a fundamental tool for administrators and developers. For more, visit the Official OpenSSH Documentation.