I don't think the timing could be better on the editorial calendar than to have the deadline for the Hacking issue coincide with the Black Hat and DEF CON conferences. Although I have long been interested in security, usually it's from the sysadmin defensive posture or from a post-hack forensics angle. This year, I was fortunate enough to look over the shoulder of a talented group of hackers from the neg9 team as they competed in an open CTF tournament. Essentially, the tournament pitted multiple teams against each other as they all attempted to solve various puzzles and security challenges against locked-down servers. I thought for this column, I would list some tips for the defensive sysadmin that came to mind as I viewed computer security from an offensive perspective, especially in a restricted environment like a hacking competition.

First, before I get too many angry letters, I'm well aware of the long controversy over the word “hacker” and its multiple connotations. I refer to both clever tricks and security breaches as hacks and their perpetrators as hackers in this article for a few reasons:

Now that that's out of the way, the first thing I found interesting about security from the offensive position was just how important it is to learn basic, classic command-line tools like vi, nc and friends. In this competition and in a proper locked-down system, you not only run minimal services, you also try to restrict the set of programs you install on the machine so they won't be available to a would-be attacker. If you want to attack a system but only know how to edit text files in Emacs, port scan with Nmap or write Ruby scripts, what do you do when those tools aren't installed? Even in a restricted environment, you can count on certain tools to be installed, such as vi, nc, dd, sh and all the other great two-letter commands. From the offensive perspective, if you know how to use those commands, you won't be hamstrung if you do happen on a minimal system. From the defensive perspective, if you do limit the programs installed on your system only to those that are necessary, you will slow down hackers who expect newer, fancier tools to be on the system.

I think if you were going to master only one of these two-letter commands for hacking purposes besides vi, nc is the best candidate. If you are unfamiliar with nc (or netcat), it is an incredibly versatile tool that allows you to open or listen for TCP and UDP connections. It's the original network Swiss Army knife, and it's a valuable tool to have in your arsenal whether you're a sysadmin or a hacker. In the case of both hacking and troubleshooting, it's useful because you can use it like telnet to connect to a remote server and port and start an interactive session:

$ nc mail.example.org 25
220 mail.example.net ESMTP Postfix
. . .
QUIT

You also could open one nc session on a port in listen mode and start a second nc session on a remote host to connect to that port and send text back and forth like a basic chat program. On the listening host, run:

$ nc -l 31337

On the remote host, type:

$ nc hostname 31337

You also can substitute the IPs for hostnames in both examples. Once the connection is made, anything typed on one end is displayed on the other, and you can press Ctrl-D in either session to close the connection.

A number of sysadmins have long used this functionality as a quick-and-dirty file-transfer protocol. Start the first nc session in listen mode, and redirect its output to a file:

$ nc -l 31337 > output_file

On the remote machine from which you want to send the file, you would type:

$ nc hostname 31337 < input_file

Once the file has finished transferring, the connection will close automatically.

Another incredibly useful function of nc is as a port scanner when something more sophisticated isn't around. Just use the -z option to have nc test only whether a port is open instead of connecting to it, add -v for verbose output, and provide a port range as arguments. So to scan a host for open ports between 20 and 25 (good for testing for open FTP, telnet, SSH and SMTP services), you would type:

$ nc -zv host.example.org 20-25
nc: connect to host.example.org port 20 (tcp) failed: 
 ↪Connection refused
Connection to host.example.org 21 port [tcp/ftp] succeeded!
Connection to host.example.org 22 port [tcp/ssh] succeeded!
nc: connect to host.example.org port 23 (tcp) failed: 
 ↪Connection refused
nc: connect to host.example.org port 24 (tcp) failed: 
 ↪Connection refused
Connection to host.example.org 25 port [tcp/smtp] succeeded!

Another interesting point at the competition involved a system in which you could log in, however, a number of directories, including the home directory, were mounted over NFS read-only. This presented an interesting set of limitations. For instance, because the machine was being attacked by multiple teams who each could log in as the same user, everyone was kicking each other off the machine. This meant every time you wanted to log in, you had to type in the password manually, yet because the home directory was read-only, you couldn't automate the process with SSH keys. It also meant you needed to be creative with where you stored your scripts, as you couldn't write to the main directory to which your user normally had access.

For the defender, you can see why mounting secure directories as read-only over NFS presents a problem for attackers—attackers can't write to the directory as users, and also even if they become root, they also might have to exploit the NFS server to remount the filesystem in read-write mode. As an attacker, that just meant you needed to be a bit more creative with where you store your files.

Most command-line users are aware of the existence of the /tmp directory on a Linux system. This is a directory to which all users can write, and special permissions are set on the directory so that although anyone can write to files there, once a file is created, other users can't access it. When hacking this particular restricted system, you could very well store your scripts in /tmp, but that's the first place other teams would notice your files and would likely delete them (bad) or modify them to do something you didn't expect (worse). Luckily, there are at least two other less-well-known directories on a Linux system that are writable by everyone: /var/tmp and /dev/shm.

The /var/tmp directory serves much the same function as /tmp does in that all users and programs can write to it; however, unlike the /tmp directory, many users are unaware that it exists or forget to look there. Also unlike the /tmp directory, which gets erased when the system reboots, any files you put in /var/tmp persist between reboots.

The /dev/shm directory is also a hacker favorite and for good reason—it's even less well-known among administrators that it's world-writable. An even better reason than that for an attacker to use this directory is that its contents actually are stored only in RAM, not on a disk, so once the system reboots, the files are gone and not even forensics techniques can recover them.

At a particular point in the competition, some smart hackers on the neg9 team discovered a fault in a custom service running as root on one of the competition systems. They were able to turn this flaw into a root exploit that allowed them to write 33 bytes as root anywhere on the filesystem they wanted. The tricky part was that if they chose an existing file, it would be truncated with those 33 bytes.

This presents an interesting problem. If your only root access was the ability to write 33 bytes to a file, how would you convert that into full root access? You might, for instance, write a new file into /etc/cron.d/ that executes an exploit script every minute. What would you do though if that doesn't work? When you put yourself in an offensive mindset in a restricted environment like that, you end up discovering creative ways to approach a system.

There were a few ways you could exploit this option to get full root access. You could, for instance, look in /etc/inittab to see what getty process was set to auto-respawn and then replace the existing getty (for instance, /sbin/agetty) with your custom 33-byte shell code. Then, when getty respawns (which you potentially could force), it would respawn as root, and you'd have your custom code running with root permissions.

The way the team exploited the flaw was to write a new short init script that made vim SUID root. Then, they wrote to a file in /proc that forced the system to reboot, so that upon the reboot, their init script would run. With vim set to SUID root, they could edit root-owned files and do things like change root's password and enable root logins with SSH and finally undo the SUID permissions on vim. From a defender's point of view, this kind of attack is quite tricky to protect against and essentially comes down to basic security practices, such as keeping packages up to date and putting services in sandboxes when possible.

All in all, I have to say, I rather enjoyed this different perspective on security. Whether you are a sysadmin who stays on the defensive side or a hacker who thinks in offensive terms, I think both sides can benefit from putting on the other's hat every now and then.