Stop Disabling SELinux!

I see a lot of people coming by #centos and similar channels asking for help when they’re experiencing a problem with their Linux system. It amazes me how many people describe their problem, and then say something along the lines of, “and I disabled SELinux...”. Most of the time SELinux has nothing to do with the problem, and if SELinux is the cause of the problem, why would you throw out the extra security by disabling it completely rather than configuring it to work with your application? This may have made sense in the Fedora 3 days when selinux settings and tools weren’t quite as fleshed out, but the tools and the default SELinux policy have come a long way since then, and it’s very worthwhile to spend a little time to understand how to configure SELinux instead of reflexively disabling it. In this post, I’m going to describe some useful tools for SELinux and walk through how to configure SELinux to work when setting up a Drupal web site using a local memcached server and a remote MySQL database server -- a pretty common setup for sites which receive a fair amount of traffic.

This is by no means a comprehensive guide to SELinux; there are many of those already!
http://wiki.centos.org/HowTos/SELinux
http://fedoraproject.org/wiki/SELinux/Understanding
http://fedoraproject.org/wiki/SELinux/Troubleshooting

Too Long; Didn’t Read Version

If you’re in a hurry to figure out how to configure SELinux for this particular type of setup, on CentOS 6, you should be able to use the following two commands to get things working with SELinux:
# setsebool -P httpd_can_network_connect_db 1
# setsebool -P httpd_can_network_memcache 1

Note that if you have files existing somewhere on your server and you move them to the webroot rather than untar them there directly, you may end up with SELinux file contexts set incorrectly on them which will likely deny access to apache to read those files. If you are having a related problem, you’ll see something like this in your /var/log/audit/audit.log:
type=AVC msg=audit(1324359816.779:66): avc: denied { getattr } for pid=3872 comm="httpd" path="/var/www/html/index.php" dev=dm-0 ino=549169 scontext=root:system_r:httpd_t:s0 tcontext=root:object_r:user_home_t:s0 tclass=file

You can solve this by resetting the webroot to its default file context using the restorecon command:
# restorecon -rv /var/www/html

Server Overview

I’m going to start with a CentOS 6 system configured with SELinux in targeted mode, which is the default configuration. I’m going to be using httpd, memcached, and PHP from the CentOS base repos, though the configuration wouldn’t change if you were to use the IUS PHP packages. MySQL will be running on a remote server which gives improved performance, but means a bit of additional SELinux configuration to allow httpd to talk to a remote MySQL server. I’ll be using Drupal 7 in this example, though this should apply to Drupal 6 as well without any changes.

Initial Setup

Here we will setup some prerequisites for the website. If you already have a website setup you can skip this section.

We will be using tools such as audit2allow which is part of the policycoreutils-python package. I believe this is typically installed by default, but if you did a minimal install you may not have it.
# yum install policycoreutils-python

Install the needed apache httpd, php, and memcached packages:
# yum install php php-pecl-apc php-mbstring php-mysql php-pecl-memcache php-gd php-xml httpd memcached

Startup memcached. The CentOS 6 default configuration for memcached only listens on 127.0.0.1, this is great for our testing purposes. The default of 64M of RAM may not be enough for a production server, but for this test it will be plenty. We’ll just start up the service without changing any configuration values:
# service memcached start

Startup httpd. You may have already configured apache for your needs, if not, the default config should be enough for the site we’ll be testing.
# service httpd start

If you are using a firewall, then you need to allow at least port 80 through so that you can access the website -- I won’t get into that configuration here.

Install Drupal. I’ll be using the latest Drupal 7 version (7.9 as of this writing). Direct link: http://ftp.drupal.org/files/projects/drupal-7.9.tar.gz
Download the tarball, and expand it to the apache web root. I also use the --strip-components=1 argument to strip off the top level directory, otherwise it would expand into /var/www/html/drupal-7.9/
# tar zxf drupal-7.9.tar.gz -C /var/www/html --strip-components=1

Also, we need to get the Drupal site ready for install by creating a settings.php file writable by apache, and also create a default files directory which apache can write to.
# cd /var/www/html/sites/default/
# cp default.settings.php settings.php
# chgrp apache settings.php && chmod 660 settings.php
# install -d -m 775 -g apache files

Setup a database and database user on your MySQL server for Drupal. This would be something like this:
mysql> CREATE DATABASE drupal;
mysql> GRANT ALL ON drupal.* TO drupal_rw@web-server-ip-here IDENTIFIED BY 'somepassword';

Test this out by using the mysql command line tool on the web host.
# mysql -u drupal_rw -p -h drupal

That should connect you to the remote MySQL server. Be sure that is working before you proceed.

Now for the Fun Stuff

If you visit your new Drupal site at http://your-hostname-here, you’ll be presented with the Drupal installation page. Click ahead a few times, setup your DB info on the Database Configuration page -- you need to expand “Advanced Options” to get to the hostname field since it assumes localhost. When you click the button to proceed, you’ll probably get an unexpected error that it can’t connect to your database -- this is SELinux doing its best to protect you!

Allowing httpd to Connect to a Remote Database

So what just happened? We know the database was setup properly to allow access from the remote web host, but Drupal is complaining that it can’t connect. First, you can look in /var/log/audit/audit.log which is where SELinux will log access denials. If you grep for ‘httpd’ in the log, you’ll see something like the following:
# grep httpd /var/log/audit/audit.log
type=AVC msg=audit(1322708342.967:16804): avc: denied { name_connect } for pid=2724 comm="httpd" dest=3306 scontext=unconfined_u:system_r:httpd_t:s0 tcontext=system_u:object_r:mysqld_port_t:s0 tclass=tcp_socket

That is telling you, in SELinux giberish language, that the httpd process was denied access to connect to a remote MySQL port. For a better explanation of the denial and some potential fixes, we can use the ‘audit2why’ utility:
# grep httpd /var/log/audit/audit.log | audit2why
type=AVC msg=audit(1322708342.967:16804): avc: denied { name_connect } for pid=2724 comm="httpd" dest=3306 scontext=unconfined_u:system_r:httpd_t:s0 tcontext=system_u:object_r:mysqld_port_t:s0 tclass=tcp_socket

Was caused by:
One of the following booleans was set incorrectly.
Description:
Allow HTTPD scripts and modules to connect to the network using TCP.

Allow access by executing:
# setsebool -P httpd_can_network_connect 1
Description:
Allow HTTPD scripts and modules to connect to databases over the network.

Allow access by executing:
# setsebool -P httpd_can_network_connect_db 1

audit2why will analyze the denial message you give it and potentially explain ways to correct it if it is something you would like to allow. In this case, there are two built in SELinux boolean settings that could be enabled for this to work. One of them, httpd_can_network_connect, will allow httpd to connect to anything on the network. This might be useful in some cases, but is not very specific. The better option in this case is to enable httpd_can_network_connect_db which limits httpd generated network connections to only database traffic. Run the following command to enable that setting:
# setsebool -P httpd_can_network_connect_db 1

It will take a few seconds and not output anything. Once that completes, go back to the Drupal install page, verify the database connection info, and click on the button to continue. Now it should connect to the database successfully and proceed through the installation. Once it finishes, you can disable apache write access to the settings.php file:
# chmod 640 /var/www/html/sites/default/settings.php

Then fill out the rest of the information to complete the installation.

Allowing httpd to connect to a memcached server

Now we want to setup Drupal to use memcached instead of storing cache information in MySQL. You’ll need to download and install the Drupal memcache module available here: http://drupal.org/project/memcache
Install that into your Drupal installation, and add the appropriate entries into settings.php. For this site, I did that with the following:
# mkdir /var/www/html/sites/default/modules
# tar zxf memcache-7.x-1.0-rc2.tar.gz -C /var/www/html/sites/default/modules

Then edit settings.php and add the following two lines:
$conf['cache_backends'][] = 'sites/default/modules/memcache/memcache.inc';
$conf['cache_default_class'] = 'MemCacheDrupal';

Now if you reload your site in your web browser, you’ll likely see a bunch of memcache errors -- just what you wanted! I bet it’s SELinux at it again! Check out /var/log/audit/audit.log again and you’ll see something like:
type=AVC msg=audit(1322710172.987:16882): avc: denied { name_connect } for pid=2721 comm="httpd" dest=11211 scontext=unconfined_u:system_r:httpd_t:s0 tcontext=system_u:object_r:memcache_port_t:s0 tclass=tcp_socket

That’s very similar to the last message, but this one is for a memcache port. What does audit2why have to say?
# grep -m 1 memcache /var/log/audit/audit.log | audit2why
type=AVC msg=audit(1322710172.796:16830): avc: denied { name_connect } for pid=2721 comm="httpd" dest=11211 scontext=unconfined_u:system_r:httpd_t:s0 tcontext=system_u:object_r:memcache_port_t:s0 tclass=tcp_socket

Was caused by:
One of the following booleans was set incorrectly.
Description:
Allow httpd to act as a relay

Allow access by executing:
# setsebool -P httpd_can_network_relay 1
Description:
Allow httpd to connect to memcache server

Allow access by executing:
# setsebool -P httpd_can_network_memcache 1
Description:
Allow HTTPD scripts and modules to connect to the network using TCP.

Allow access by executing:
# setsebool -P httpd_can_network_connect 1

Again, audit2why gives us a number of options to fix this. The best bet is to go with the smallest and most presice change for our needs. In this case there’s another perfect fit: httpd_can_network_memcache. Enable that boolean with the following command:
# setsebool -P httpd_can_network_memcache 1

Success! Now httpd can talk to memcache. Reload your site a couple of times and you should no longer see any memcache errors. You can be sure that Drupal is caching in memcache by connecting to the memcache CLI (telnet localhost 11211) and typing ‘stats’. You should see some number greater than 0 for ‘get_hits’ and for ‘bytes’.

What are all these booleans anyway?

Now we’ve used a couple SELinux booleans to allow httpd to connect to memcached and MySQL. You can see a full list of booleans which you can control by using the command ‘getsebool -a’. They are basically a preset way for you to allow/deny certain pre-defined access controls.

Restoring default file contexts

As I mentioned briefly in the ‘TL;DR’ section, another common problem people experience is with file contexts. If you follow my instructions exactly, you won’t have this problem because we untar the Drupal files directly into the webroot, so they will inherit the default file context for /var/www/html. If, however, you were to untar the files in your home directory, and then use ‘mv’ or ‘cp’ to place them in /var/www/html, they will maintain the user_home_t context which apache won’t be able to read by default. If this is happening to you, you will see the file denials logged in /var/log/audit/audit.log -- something like this:
type=AVC msg=audit(1324359816.779:66): avc: denied { getattr } for pid=3872 comm="httpd" path="/var/www/html/index.php" dev=dm-0 ino=549169 scontext=root:system_r:httpd_t:s0 tcontext=root:object_r:user_home_t:s0 tclass=file

The solution in this case is to use restorecon to reset the file contexts back to normal:
# restorecon -rv /var/www/html

Update: It was noted that I should also mention another tool for debugging audit messages, 'sealert'. This is provided in the setroubleshoot-server package and will also read in the audit log, similar to what I described with audit2why.
# sealert -a /var/log/audit/audit.log