Since the middle of 2001, I have been working on NSA Security Enhanced Linux [see page 20 in this issue] both on packaging it for Debian and in general development. When describing the project to Linux users, I find much confusion exists about what SE Linux does; it is difficult to gain a full understanding of SE Linux from reading the documentation or attending a presentation. Also, many people who have prior experience in security want to gain some practical experience but don't have the time to install SE Linux to experiment. I decided that a good way to teach people about SE Linux would be to set up a machine with public access for anyone to use.

Demonstrating SE Linux in a regular configuration is not particularly exciting, as the only noticeable operations it restricts for non-root users are ps ax and dmesg. In a default configuration, ps ax shows an unpriviledged user only the other processes in the same user domain, and dmesg is blocked. This is similar to the restrictions imposed by OpenWall and is nothing new in itself. I decided to grant root access to the world using only SE Linux for security, so users can see exactly what it is capable of doing.

On June 6–9, 2002, at LinuxTag in Karlsruhe, Germany, I ran an SE Linux demo machine at the Debian stand. This was the first SE Linux play machine. At the time, the default policy was less restrictive than it currently is. It allowed setuid and DAC_OVERRIDE capabilities for regular users (user_t domain). For a regular SE Linux configuration, this is fine. SE Linux does not use uids when deciding whether to grant access, and DAC_OVERRIDE allows overriding the UNIX access controls, but not any SE controls. The reason these capabilities were granted was to allow running setuid programs from the user_t domain without needing SE Linux domains for such programs. So although those capabilities were satisfactory for the typical user, they were not suitable for the unusual situation of having a root user who should be banned from accessing other uids in the same domain. I removed these capabilities from user_t, restricted the root account to the user_r role and it was ready to go.

In recent releases, the default policy has changed to not grant setuid or DAC_OVERRIDE capabilities to user_t. So, the most significant security policy difference between my play machine and a real server is that on the play machine unprivileged users are permitted to read the kernel message log (dmesg) and the security policy source as an aid to understanding SE Linux.

My SE Linux challenge is based on a machine deliberately configured to be less secure than real servers, by granting log file access, granting read access to the security policy and allowing unprivileged users root access. In spite of these factors, little success was had in breaking the security.

On the first day of LinuxTag, a potential issue with /boot files was reported. A user believed he could determine the LILO password from the LILO map file. I immediately changed the policy to restrict the access to /boot to prevent such problems. Of course, if you have physical access to a machine you usually can break the security somehow, but we want to make it as difficult as possible.

During the event, I started work on support for multiple user roles. The initial reason for this was one of my colleagues used the play machine for more serious purposes. He lost all his files, because they were created by the root:user_r:user_t security context as uid root, the same as users who were testing the security. The standard test that everyone ran as root was rm -rf /, which deleted all his files. The system itself was unharmed, as files in /bin, /etc and other system directories cannot be unlinked or written to by user_t. After I gave my friend an account with the domain user1_t, his files could not be accessed by a root user in domain user_t.

On June 17, 2002, I set up an SE play machine on a Cobalt Qube that is available on the Internet for everyone to use. The first machine was on-line intermittently until July 11. The uptime for the play machines has not been great, because they need to be monitored continually. Such a machine would have the potential to become a risk to everyone, including me, my ISP and people who use it, if it was cracked and I didn't respond fast enough. So whenever I go on holidays or am busy with work, I have to take it off-line.

The machine had its own iptables setup to prevent undesired network access from leaving the local machine. It also was placed behind a firewall, which applied similar restrictions on data transfers. This setup prevented any user from even probing my firewall from the inside unless they first cracked the security of the play machine. I initially allowed most outbound network connections other than SMTP, but I soon changed this to allow only outbound connections to a Web proxy. SSH tunnels could be used for other Net access. Also, I denied X forwarding so that if a user mistakenly enables it on his client, his machine can't be attacked by other users on the play machine.

When the play machine had been on-line for less than a day, a user reported that /etc/shadow could be read. This directory was declared to be outside the scope of the LinuxTag demonstration, but I should have fixed it before putting the machine on-line. I changed the shadow file to have the type shadow_t, which required changes to the spasswd wrapper program and the SE policy for it.

Adding full support for shadow_t was difficult because, in many instances, the same program changes /etc/passwd and /etc/shadow by re-creating them, thus giving them the default context of etc_t. I could have modified those programs to use the open_secure(2) system call to specify the security context at file creation time. I decided not to, however, because it would involve a lot of work on security critical applications, creating the risk that an error might weaken security. Instead, I wrote wrapper code to run those programs and set /etc/passwd back to etc_t after the program exits. I also made shadow_t the default type for those programs when creating files in /etc. Still, even when /etc/shadow had the type etc_t, it prevented unauthorized root users from writing to it. It was read-only to root users in the user_t domain.

The next day, someone discovered that /dev/nvram was not adequately protected. It was writable by everyone, therefore any user could make the machine unbootable by scrambling the BIOS setting. Potentially, they could have made the Qube BIOS pass different parameters to the kernel to weaken security on the next boot. The Cobalt BIOS performs the functions that a bootloader such as LILO would perform on other machines. I changed the policy to fix that glitch immediately. It is important to note that different platforms, either different CPU architecture or a different hardware, may require similar minor changes to the security policy to match different device nodes in /dev. With the current policy there is little risk of this causing insecurity, as the default is to deny most operations related to device nodes.

Some people were concerned that I had not appropriately granted authorization and wanted reassurance that they were not doing anything illegal, so I changed the /etc/motd to state that the machine was put on-line for the purpose of security testing. I explained that it would be acceptable to break the security in any way, including methods that may render the machine unusable, as long as I was informed of the method used. I also stated that it was not to be used for launching attacks on other machines, although I tried to make that impossible with firewall rules. Finally, I requested that no one try denial-of-service (DoS) attacks, as they are boring and that is not the aim of the exercise.

From June 20 on, the operation of the play machine was fairly uneventful. In February 2003, I put a play machine on-line at the Debian stand at OSDEM and announced it as a capture the flag contest. This received a surprising amount of interest; at times there were up to 30 people watching one person trying to crack the security. A user managed to get the easy flag, accessing a file in a specified non-root account after logging in as root. He did this by setting the EDITOR environment variable and running crontab -e. The crontab program ran the editor with more SE privileges than a regular program and allowed greater access. Even though the exploit would not work in a typical server configuration, because you don't give untrusted users root access even if you are running SE Linux, I changed the policy for the crontab program to prevent doing so. Even with this, the crontab attack still was confined to a single user role. Therefore, any accounts that were in different domains, such as the one I used for running the play machine, could not be touched.

One ongoing problem I experienced was that of resource usage. Many users thought they had achieved something by filling the filesystem or running the machine out of other resources. The message about DoS attacks didn't seem to receive much notice.

Another interesting problem I had was trying to convince users they actually were root. I had GCC installed, and many users compiled their own versions of ps and other utilities in the belief that they weren't really root and that it was all a trick with modified utilities. One user even had assembly code to call the getuid() system call to determine whether I had modified libc6. Although that user really was root, it would be a fun exercise to modify libc6 to pretend that someone had logged in as root when they really had not. I encourage readers to try this out for themselves.

Of course, not all users were so difficult to convince. I gave the password to a “black hat” person who was seeking machines for the installation of a rootkit. He tried installing his rootkit but found that all the relevant directories (/bin, /sbin and /etc) and the files they contained were not writable. He asked for assistance in installing, but I was unable to help him.

If you want to run your own security test machine, the first thing to do is find a suitable place to house the server. This is easier said than done, as such a machine attracts a lot of network scans and penetration attempts on the network. The terms of service of most ISPs prohibit such things, and you risk being disconnected.

Once you have arranged the hosting, you have to devise a good method for taking the machine off-line in a hurry in case something goes wrong. Direct physical access to the power switch is convenient for this purpose. A method of controlling the power or hardware reset over the Internet also is a good option. Failing that, you should install the test machine on its own switch port, for a managed switch, or, as a cheaper option, on a crossover cable to a Linux machine running Netfilter. This way, you can disable network access to the entire machine quickly.

The next thing to do is choose suitable hardware. For example, an iPAQ is not ideal for this type of machine, as it is possible to render it unusable through software. Commodity desktop PC hardware is a good option, though. The worst-case scenario would be replacing the motherboard, which is cheap and easy. Another good option is to obtain free hardware, so you won't have lost any money if the system dies. Some nice hardware seems to end up in the rubbish nowadays.

Once you have the machine basically configured, you have to set up suitable packet filters to prevent it from being used for attacks on other machines. How strict these filters are depends on the agreement you have with your ISP. If you have no specific agreement regarding such access—if you are using a home broadband connection—then the filters should be very strict. If your contract specifically permits running servers, you can allow greater access, even the ability to host Web pages. Granting more network access allows more interesting tests to be performed. A frequent complaint from users was the test machine didn't have enough access granted to allow a wide enough range of testing. For the next play machine, I plan to provide full network access, so users can receive mail on the machine, host Web pages and do most other things that they request.

The firewall should be set up both on the test machine and on any other machines on the same physical network. The test machine can be configured reasonably with Netfilter to discard or reject the packets silently, without logging them, although you may want to log them for interest. The router should be configured to log all such packets when it drops them, so you know if someone gets past the filter on the test machine or cracks its security in any other way. If your ISP knows of your plans for a security test server, then a minimal firewall should work. This will prevent SMTP connections, spoofed source IP addresses on packets being sent and connections to Web-mail services such as Hotmail, which includes blocking access to Web proxies and configuring any local Web proxies to not allow the test machine access to Web mail.

Having any machines other than the test machine and the router on the same LAN may be a bad idea, as it may allow the test machines to be used to attack the other machines. Having several security test machines on the same physical network may be fun, though, as it would allow them to be used to attack each other. If you have only one test machine, connecting it to the router by a crossover Ethernet cable or a null-modem cable running PPP probably is a good option.

Once the machine is connected and all firewalls are arranged, the difficult work begins. You have to determine how to limit the access that is granted and audit it as appropriate. For SE Linux, all that needs to be done is to change the root entry in the users' file to user root roles { user_r };. Another option is to remove the root entry from the database entirely, as the default identity of user_u is permitted only the role user_r and gives the extra restriction of preventing password changes. To change the password of a nonprivileged account, the identity must match the user name.

The policy database then has to be recompiled and loaded into the kernel to apply the change. After that, the root user has no significant access to the system, so make sure you grant some other account administrative privileges first.

The next time I set up a test machine, I plan to get someone with legal experience to review the usage conditions to make sure they state what is permitted in a clear and legally binding language. I will place the password on a Web page that has the usage conditions and change it regularly, so people can't get in without reading the conditions. Too many people were obviously not reading the conditions, particularly regarding local DoS attacks through fork bombs and using all available disk space.

If you run an SE Linux play machine, please let me know so I can publicize it on my Web page.

I have been using the IRC channel #selinux on irc.debian.org for supporting the play machine and for answering general SE Linux questions. I encourage anyone else who is running such a security test machine, whether SE Linux or some other system, to join that channel to discuss it.

The Cobalt division of Sun generously supported my work through the gift of a RaQ server. All SE Linux play machines after LinuxTag were run on Cobalt hardware.