CentOS 8.x Securing a Linux machine

<yambe:breadcrumb self="Securing a Linux machine">Security tips|Security tips</yambe:breadcrumb> <yambe:breadcrumb self="Securing a Linux machine">CentOS 8.x System Administration|System Administration</yambe:breadcrumb>

CentOS 8.x Securing a Linux machine

This article is written assuming CentOS 8.x OS. However the general principles of hardening will apply to any other system. Only the commands need to be changed based on the flavor / distribution of Linux being hardened.

This article is work in progress

Securing a Linux system is a must for any system that is accessible publically or is in potentially hostile environment. General tips for securing a Linux machine have two parts:

• Initial security hardening of the system
• Regular tasks to monitor and ensure system is safe

Initial security hardening of system

Update system

It makes sense to keep system fully updated with latest patches and security updates. This is easier for enterprise distributions such as CentOS which are purpose built for servers and hence typically have older version of software compared to Desktop editions (eg Fedora) which might come with latest versions. Using slightly older version of software that has been tested for more than a year is safer for servers.

To update the system fully use:

dnf update -y

Setup firewalld

It is important for system to have host based firewall setup. To setup firewalld Refer:

Basic setup

CentOS 7.x Basic firewalld configuration

Advanced details about firewalld

CentOS 8.x firewalld

Setup fail2ban

Any system which is exposed to public Internet starts getting attacks immediately. If we leave SSH port open then we can see thousands of bad login attempts on any system per day. To ensure that such attackers get only limited no. of chances (Bad password attempts) to attack system, we can setup fail2ban. Fail2ban will ban IP for some duration (default 900 seconds) if it makes more than a fixed no. of bad login attempts. The no. of attempts allowed for root user are typically lesser than no. of attempts allowed for other users. Limiting only a few (9-10) attempts every 900 seconds (15 minutes), is more than enough to ensure that system cannot be exploited using dictionary based attacks / bruteforce attacks.

To setup fail2ban refer:

• CentOS 7.x fail2ban

Note that fail2ban supports many other applications such as dovecot, postfix apart from sshd. Hence we should try to secure as many applications via fail2ban as possible.

Earlier versions of OS used to use Denyhosts, which is now deprecated and we should use fail2ban instead.

Setup logwatch and outgoing alert emails

Any Linux system generates many useful logs that go to various different log files. It is not possible for human administrators to go through many log files of many systems and look for issues and anomalies. Hence, it makes sense to configure Logwatch to go through various log files and send one email per day related to events / logs that were seen on that system in past 24 hours.

Since logwatch can send email enable outgoing email from system using CentOS 8.x postfix send email through relay or smarthost with smtp authentication if required

To install logwatch use:

dnf -y install logwatch

After default setup consider:

Increasing Logwatch detail

Increasing detail of logwatch output

Configure From and To address for logwatch

CentOS 7.x Zimbra command line for sending logwatch email

Disable too much logs from kernel when log detail is high

Disable too much logs from kernel when log Detail is high

Create customer logwatch service or scripts

Creating new logwatch service or scripts

Disable IPv6 connectivity if not required

If you are confident that IPv6 is not being used and not required, then it can be disabled to avoid IPv6 related attacks / entry points using:

• CentOS 8.x Disabling IPv6

Configure backups

It is critical to have backups for all important data from all systems. For automated backups Refer CentOS 7.x rsnapshot or Rsnapshot

Configure longer history retention with date/time

By default only last 1000 commands are stored in history without any timestamps. It makes sense to have system store longer history and along with timestamps.

Refer:

• Storing date / time along with commands in history

SSL certificate

If server has public FQDN, it makes sense to have recognized SSL Certificates purchased from a provider such as Installing lets-encrypt SSL certificate

Use of recognized SSL certificate in place of self-signed certificates, makes it easy for users to know that they are connecting to the right machine

Containers or virtualization for isolation (lxc, kvm)

If the machine is having multiple functionalities such as DNS, Web server, database etc. then it makes sense to separate them among various containers or virtual machines. This ensures that compromise of one of these systems does not affects others.

To learn about containers refer CentOS 8.x lxc

For virtual machines refer CentOS 8.x KVM

If the base machine is only used for virtualization (and not as Desktop with GUI) and all functionality is served by containers / VMs then it makes sense to have specialized distributions such as Proxmox virtual environment on base machine.

Avoid Using FTP, Telnet, And Rlogin / Rsh Services

(SSH, SFTP, Owncloud, etc.)

Secure services

Securing SSH service

Change SSH port from default 22

For this use following steps:

1. Edit /etc/ssh/sshd_config and use

   Port 22   #Dont remove this yet
   Port 5000

   Replace 5000 with desired port no.

2. Restart sshd

   systemctl restart sshd

3. Try to connect to SSH over new port (eg 5000 in above example)
4. If connection is not working check firewall. By default firewall rules allow only connection to port 22. Enable connections to port 5000 from your IP. Refer CentOS 8.x firewalld
5. Once connection is wroking edit /etc/ssh/sshd_config and comment port 22:

   #Port 22
   Port 5000

6. Restart sshd for new settings to take effect

   systemctl restart sshd

7. Validate that you are able to connect on different port and that connection to port 22 is not working.

Protect SSH port access via firewall

Based on machines which need SSH access to server, protect access to SSH port via firewall.

If SSH port is changed from 22 to some other port, by default firewall will not have exception for that port. In such cases instead of adding exception for port, allow access to all / desired port only from specific IPs / subnets of admin stations. Example

#Allow all ports
firewall-cmd --add-rich-rule 'rule family="ipv4" source address="1.1.1.1/32" accept' --permanent    

#Allow access to port 5000 
firewall-cmd --add-rich-rule 'rule family="ipv4" source address="1.1.1.1/32" port port=5000 accept' --permanent    

#Reload and validate
firewall-cmd --reload
firewall-cmd --zone=public --list-all 

Refer: CentOS 8.x firewalld

Allow SSH only for required users

You can limit users who have SSH access to system using options such as:

Allowusers root

This will allow only root user to login into the system. If you have another user with sudo access and want to disable root access then enable only that user in Allowusers.

First test the setting by connecting in another terminal without closing current root ssh connection. If you are again able to get remote root access directly or via another users, then only close the current connection

Refer:

• Restricting SSH access to given users
• https://ostechnix.com/allow-deny-ssh-access-particular-user-group-linux/

Restrict users who need file transfer to their home folder using SFTP chroot

If users need abilitty to transfer files via scp/sftp (but not rsync) then they can be restricted to the home folders using sftp chroot. Refer Chrooting sftp users to home directory with openSSH

Use key based authentication for SSH. At least for root user disable password based SSH

Ideally we should disable password based SSH for all users using:

1. Edit /etc/ssh/sshd_config and set

   PasswordAuthentication no

2. Restart sshd service

   systemctl restart sshd

However, if above is not practical then at least disable password based SSH for root user using following in '/etc/ssh/sshd_config':

PermitRootLogin without-password

and reload sshd service

To understand key based authentication Refer Configuring authorized_keys file for public key based access

Secure SSH keys with password

If key based SSH is used, it makes sense to secure SSH keys via password. This way if someone has access to your system temporarily and they copy your ssh private keys, they cannot use them without knowing the password. Most Linux systems allow automatic unlock of ssh identity when doing a GUI login onto system, by saving ssh key password in keyring. Thus, this does not leads to any inconvinience.

Refer Passphrase for ssh-keys

Secure other services eg (Web, proxy, DNS, MySQL etc.) setup on that server.

• TODO
• OWASP for web applications

Physical security

It is important to focus on physical security of the system along with software security hardening. Physical security is important because it can prevent accidental /intentional shutdown / reboot / network disconnection. Note following points that explain either potential security issue if someone gets physical access to system or option to secure system from physical access:

BIOS boot passwords

It is possible to configure password in BIOS to be asked during boot. This can be circumvented by doing BIOS reset using jumpers but this requires opening the system and know-how of hardware. This would also take longer compared to booting the system without password by adversary.

Grub single mode root access

It is possible on most systems to edit grub and go to single mode. Then in single mode unrestricted root access is available to copy files / reset root password / change security settings, etc.

Boot from live CD/DVD/USB

A person can boot system from live CD/DVD/USB and mount file-systems. Then settings can be changed, files can be copied, passwords can be reset etc. without leaving any logs on original OS installed on physical hard-drive.

Take hard-disk and put in another system

This has same risks as booting with live CD/DVD/USB. Once the hard-disk is inserted into another system and booted using another hard-drive, the partitions and files of current system can be accessed on remote system without any problem.

BIOS hard-disk encryption

It should be possible to set password on hard-disk on BIOS. This makes hard-disk unusable unless same password is supplied during boot. Even if this hard-disk is taken to another system, the data would remain secured unless the attacker knows the configured hard-disk password.

File and folder encryption

: If encryption / protecting entire disk is not possible / practical then there are encrypted filesystems that can be used to encryt data when system is running. If someone steals hard-disk or boots system using live CD/DVD they cannot get the data unless they know the encryption keys / passwords to decrypt the files. Refer Ecryptfs or EncFS

One time TODO

• Setup anti-virus scan via clamav
• OSSEC
• Configure audit daemon
• Configure two factor authentication for applications including SSH (Google authenticator)
• Remove unwanted packages
  • Many people suggest removing X11 or graphical packages, if they are not required
  • Removing cc,gcc, etc. should make it difficult for attacker to compile programs
• Monitor User Activities (psacct, acct)?
• Record system CPU, Memory, Disk usage statistics?
• Zabbix montioring?
• DRL hardening script?
• CIS Linux benchmark??
  • https://blog.rsisecurity.com/basics-of-the-cis-hardening-guidelines/
  • https://docs.google.com/document/d/1P65M9Qmhhrn0sKK3_qKs23WnXKXf_BExgThTH9Uy9Kc/edit#
• Honeypots?

Regular tasks to monitor and ensure system is safe

TODO

• Do VA/PT of your system using tools
• Go through logwatch report (Daily)
• Additional updates (dnf -y update) -- Automatic updates??
• Installed packages list (dnf history)
• List of open ports (ss -alnp). Avoid non-encrpted (non-SSL/non-TLS) services
• Validate list of users (getent passwd)
  • Validate the UID of Non-Root Users
  • Validate shell of non-human users
• Validate sudo access
• Validate SSH authorized keys
• Validate firewall rules including people who have SSH access
• Validate cron files and crontab entries
• Validate atd entries or disable at daemon
• Validate kernel modules (Honeypots, keyloggers, etc.)
• Validate backups are happening properly (Including application / DB backups)
  • If possible restore backup (Note steps)
• Validate fail2ban status and no. of machines blocked
• Disable unwanted services
• Disable SUID and SGID Permission
• Maintain Word-Writable Files
• Look for files modified using chattr (lsattr)
• Look at relevant application logs (/var/log/httpd, /var/log/maillog, etc.) and OS logs (/var/log/messages)
• Look at audit logs
• Try to crack existing passwords
• Look for rootkits using chkrootkit and rkhunter (

Reformat compromised system

Refer:

• https://www.ubuntupit.com/best-linux-hardening-security-tips-a-comprehensive-checklist/
• https://www.tecmint.com/linux-server-hardening-security-tips/
• https://www.cyberciti.biz/tips/linux-security.html
• https://www.cyberciti.biz/faq/howto-check-linux-rootkist-with-detectors-software/
• https://www.rootusers.com/23-hardening-tips-to-secure-your-linux-server/
• https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/security_hardening/overview-of-security-hardening-security-hardening

<yambe:breadcrumb self="Securing a Linux machine">Security tips|Security tips</yambe:breadcrumb> <yambe:breadcrumb self="Securing a Linux machine">CentOS 8.x System Administration|System Administration</yambe:breadcrumb>