Being paranoid is a good thing when it comes to computers. Every system can be compromised, regardless of how hardened the system is.
Here are some essential tips for checking whether your system may be compromised.
The first place to look at are log files across the system. Check for any unusual activity that you might suspect as malicious behaviour, such as login attempts you don’t recognize. Using tools such as “last” to check the times/dates of recent logins to the system. If your logs are written to the same system being inspected, they could have easily been edited to cover an intruders tracks, so whilst it’s not always useful to check log files, they are still worth having a look. The best scenario is where your system writes logs to a different machine. This way an intruder would not be able to edit out their activities as easily.
Check to see what is actively connected to the system. In the event you find an irc connection, and are not running an ircd client, this is a big red flag. You may also want to inspect and process spawned by perl, python, sh, or otherwise. It’s common for scripts to be run that give remote access to the machine.
# netstat -tupn
Many hackers will add programs to the cron scripts. One technique is to “dump” a script repeatedly, which gives remote access, in the event they are deleted by the sysadmin.
Ensure that none of the scripts in /etc/cron* or other cron directories are not world-writable, otherwise an unprivileged user could easily execute a script as root by using cron.
Check the crontab aswell:
# crontab -e
A common way to obtain access with root rights on a system, is to create a setuid/setgid file. Look for these, as they can potentially give an unprivileged user root access at a later time, given the chance an intruder left something behind with the idea to come back at a later date.
# find / -user root -perm -4000 -print
# find / -group kmem -perm -2000 -print
You can also append “-xdev” to these commands to ignore device nodes.
There will of course be plenty of setuid/setgid files present on the system, it’s a matter of noticing something that has changed, or something that shouldn’t be there.
It’s always good policy to know who you are letting into your system. Refer to your iptables (or other firewall) manual. A common red flag can be that you as the sysadmin have physical access to the machine, so you only connect over the LAN. If you were to find an exception for a remote host/address which has been added to the system, it’s most likely the system has been compromised.
If you find files you are unfamiliar with, in your webroot, system folders or otherwise, this is a clear sign of a breach of some kind. Commonly tools such as “nmap” may be installed on the system by an intruder, or they may leave behind source files used for building exploit software on the machine.
Check for unauthorized user accounts on the system
# more /etc/passwd
A red flag is finding multiple users with UID 0, or accounts with no password.
A hacker may run tcpdump or other form of sniffer on the system to harvest more information such as user credentials from the network. Finding a rogue sniffer on a system would suggest it has been compromised.
Many hackers may modify something such as “/bin/sh” to point somewhere else. Common system files may be altered to include command line options which give instant root access. One way to keep track of system binaries is to use a tool like “rkhunter”. You can generate a hash for each binary, and then recheck for any mismatches at a later date, making it easier to find out what has changed on the system in the event of intrusion. However, system updates will change any binaries that are given an update, so that’s something to keep in mind when checking.
Look for any services that are running which shouldn’t be. Many times a hacker will run a service disguised as a common program, such as inetd or httpd. It’s also wise to inspect the init scripts for any backdoored features. You may see inetd running on your system, but it could be a trojan horse version of a secure shell.
A common trick to hide mallicious files on a system is to name them something like “.. ” or “…” or even “..^G”. When running “ls”, nothing may appear out of the ordinary, and these can be easy to overlook.
# find / -name ".*" -print