HOWTO build an Apache Security Proxy
From RADION OpenLab
Contents |
1 Release history
| Draft | 26.08.2005 | Kamil Wencel | Official Release for Apache 1.3.33 | KamilWencel 19:55, 29 October 2005 (UTC) |
| Release 1.0 | 05.09.2005 | Kamil Wencel | Official Release for Apache 1.3.33 | KamilWencel 19:55, 29 October 2005 (UTC) |
| Release 1.1 | 27.10.2005 | Kamil Wencel | Minor Cosmetics | 62.245.223.133 15:15, 28 Oct 2005 (UTC) |
| Release 1.2 | 29.10.2005 | Kamil Wencel | Update for Apache 1.3.34 | KamilWencel 19:55, 29 October 2005 (UTC) |
2 Introduction
It's a shame but it is true : The good old times, when you could build applications without security in mind are gone. As a matter of fact you should consider spending more time on security technology & philosophy than on the actual application itself. You cannot use an application which technically performs its tasks but cannot be deployed in an insecure environment. With that in mind this HOWTO evolved. No matter how much time you invest into making your application secure, there will always be a chance you might forgott something or didn't know better. Therefore you should have some fallback option like an application layer filter in front of your application server ( which might be an apache with php as well ) - that will add a great deal of security, if handled mindfully.
This HOWTO will help you to deploy a jailrooted Apache with ModSecurity, ModSSL and PHP enabled. It will NOT cover all the specifics of PHP configure options or SSL configuration for webservers. If you would like to contribute to these topics write your own HOWTO at RADION Openlabs.
To follow this HOWTO you need at least good basic knowledge of Apache and the way it works. Don't forget that mod_security basically is an Intrusion Detection System ( IDS ) and as such needs a lot of attention regarding the rules. You cannot expect a "fire-and-forget" solution, these rules live and have to have regular maintenance. You will have to tweak a lot to avoid getting caught up in false positives. Sometimes you'll have to disable it at all, because the specific purpose of the site you want to protect always gets trapped within the IDS.
3 What ModSecurity can do for you
In order to add security benefit to your infrastructure you have to know how each and every piece in your security chain works otherwise you will not be able to maintain security. At first glance ModSecurity is just another module for the Apache web server. In the simplest terms ModSecurity offers the following features :
- ModSecurity checks all in and outgoing HTTP traffic ( GET,POST,HEAD,etc. )
- Attack patterns to check can be included into a single config file
- Easy to understand config
- chroot Apache without the hassle of providing a fully separated environment manually
This HOWTO describes how to setup both, intrusion detection and a chrooted environment.
When chroot mode is enabled you have to understand the startup procedure to not to stumble over configuration issues. When you start Apache the parent process opens all logfiles - which should be located outside of the jailroot from my point of view - and provides internal pointers to these logfiles for its children.
4 Definitions
We used a pretty much default gentoo stage1 system for this HOWTO. None of the packages we want to use were emerged from portage.
We want Apache to run in a JailRoot to lessen the risk of the whole machine becoming compromised.
4.1 General conventions
- All source files will reside in /usr/local/src/
- All documents, HTML, PHP, images etc. will go into the JailRoot in /www
- All logfiles will be in /var/log/apache
- The apache start / stop script will reside in /etc/init.d/apache
- Apache will not be installed to /usr but to /usr/local/apache/1.3.39
- Apache Virtual containers will reside in /www/container/
- Apache Virtual container logfiles will go respectively into /var/log/apache/container/$containername
4.2 Recommended partitioning and mountpoints
Basically the harddisk is partitioned as follows :
Disk /dev/hda: 160.0 GB, 160041885696 bytes 255 heads, 63 sectors/track, 19457 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/hda1 * 1 4 32098+ 83 Linux /dev/hda2 5 129 1004062+ 82 Linux swap / Solaris /dev/hda3 130 1126 8008402+ 83 Linux /dev/hda4 1127 19457 147243757+ 83 Linux
and these partitions are mounted as :
| /dev/hda1 | /boot | Boot Partition |
| /dev/hda2 | swap | Swap Partition |
| /dev/hda3 | / | Root Filesystem |
| /dev/hda4 | /www | Apache JailRoot Filesystem |
4.3 Considerations
Configuration needs a bit of a split personality. For example while you and your OS will see the path to the container as /www/container/www.mydomain.org is you stick to the recommended configuration of this HOWTO in your hhtpd.conf you'll have to write /container/www.mydomain.org because after loading als its libs and additional modules apache chroots and simply doesn't see anything above /www or /www itself anymore.
5 Recommended hardware
It is hard to forecast which services you are going to offer and how many visitors you may get using them. So the real hardware requirements of your services depend highly on your own estimations. Basically two things you are going to need :
- As much CPU power as you can get
- As much RAM as you can get
We also recommend to use SMP ( Multiprocessor ) systems, as especially Apache will benefit from having more CPU's to fork onto. Regarding main memory ( RAM ) you should have at least 1024MB of it and an equally sized swap partition.
Remember, CPU and RAM usage also highly depend on the size of your ModSecurity ruleset and of course : SSL can really drain a machine down because of CPU expensive key calculations. If you plan to use a lot of SSL connections consider buying a SSL hardware accelerator card and dedicate a server for SSL connections only.
6 Required software
The following links will transport you as close to the download as possible. If you want further links regarding the software used, please have a look at the appendix.
6.1 Open Source
| Apache 1.3.39 | http://httpd.apache.org/download.cgi |
| PHP 5.2.4 | http://www.php.net/get/php-5.2.4.tar.bz2/from/a/mirror |
| ModSSL | http://www.modssl.org/source/ |
| ModSecurity | http://www.modsecurity.org/download/index.html |
| ModGZip | http://sourceforge.net/projects/mod-gzip/ |
6.2 Closed Source
| Zend Optimizer | http://www.zend.com/store/products/zend-optimizer.php |
Put all files you download into /usr/local/src.
7 Unpacking packages
# tar xvfz apache_1.3.39.tar.gz # tar xvfz mod_ssl-2.8.30-1.3.39.tar.gz # tar xvfz modsecurity-1.9.5.tar.gz # tar xvfz mod_gzip-1.3.26.1a.tgz # tar xvfz ZendOptimizer-2.5.10a-linux-glibc21-i386.tar.gz # tar xvjpf php-5.2.4.tar.bz2
8 Building required components
8.1 Building Apache
Apache is going to be build to support SSL, PHP, ModSecurity and ModGZip. To achieve that manually without emerging we have to perform several steps in a certain order. We are going to start with ModSSL :
8.1.1 Configure ModSSL in conjunction with Apache
# cd /usr/local/src/mod_ssl-2.8.30-1.3.39
If you are using a gentoo system you'll most likely want to use the same CFLAGS your gentoo system uses to build its packages and copy the CFLAGS out of /etc/make.conf. The CFLAGS used in the example below are working perfectly for us on a x86 Pentium3 ( Dual-Tualatin ) machine.
# CFLAGS="-march=pentium3 -mtune=pentium3 -O3 -pipe -fomit-frame-pointer" \ ./configure \ --with-apache=../apache_1.3.39 \ --with-ssl=/usr \ --prefix=/usr/local/apache/1.3.39 \ --datadir=/www \ --enable-module=most \ --enable-shared=max \ --enable-module=ssl \
This should create the following output :
Configuring mod_ssl/2.8.30 for Apache/1.3.39
+ Apache location: ../apache_1.3.39 (Version 1.3.39)
+ OpenSSL location: /usr
+ Auxiliary patch tool: ./etc/patch/patch (local)
+ Applying packages to Apache source tree:
o Extended API (EAPI)
o Distribution Documents
o SSL Module Source
o SSL Support
o SSL Configuration Additions
o SSL Module Documentation
o Addons
Done: source extension and patches successfully applied.
Configuring for Apache, Version 1.3.39
+ using installation path layout: Apache (config.layout)
Creating Makefile
Creating Configuration.apaci in src
+ enabling mod_so for DSO support
+ configured for Linux platform
+ setting C compiler to gcc
+ setting C pre-processor to gcc -E
+ using "tr [a-z] [A-Z]" to uppercase
+ checking for system header files
+ adding selected modules
o rewrite_module uses ConfigStart/End
disabling DBM support for mod_rewrite
(perhaps you need to add -ldbm, -lndbm or -lgdbm to EXTRA_LIBS)
o ssl_module uses ConfigStart/End
+ SSL interface: mod_ssl/2.8.30
+ SSL interface build type: DSO
+ SSL interface compatibility: enabled
+ SSL interface experimental code: disabled
+ SSL interface conservative code: disabled
+ SSL interface vendor extensions: disabled
+ SSL interface plugin: Built-in SDBM
+ SSL library path: /usr
+ SSL library version: OpenSSL 0.9.7e 25 Oct 2004
+ SSL library type: installed package (stand-alone)
+ enabling Extended API (EAPI)
+ using system Expat
+ using -ldl for vendor DSO support
+ checking sizeof various data types
+ doing sanity check on compiler and options
Creating Makefile in src/support
Creating Makefile in src/regex
Creating Makefile in src/os/unix
Creating Makefile in src/ap
Creating Makefile in src/main
Creating Makefile in src/modules/standard
Creating Makefile in src/modules/extra
Creating Makefile in src/modules/proxy
Creating Makefile in src/modules/ssl
Now proceed with the following commands:
$ cd ../apache_1.3.39
$ make
$ make certificate
$ make install
Let's proceed as suggested by the build script to change into Apache's source directory and start compile it.
8.1.2 Compile Apache
If you try to build Apache now it will fail building the mod_auth_dbm module due to a missing ndbm.h header file. However, the file is on the system as part of the gdbm package, but unfortunately it is not located in /usr/inculde but in /usr/include/gdbm/. You can easily tell Apache where to look for it in the standard modules Makefile:
# vi /usr/local/src/apache_1.3.39/src/modules/standard/Makefile
Change the INCLUDES1= into INCLUDES1=-I/usr/include/gdbm/
INCLUDES1=-I/usr/include/gdbm/
Now everything should be alright (at least it was here)
# cd /usr/local/src/apache_1.3.39 # make -j3
the last output should look as follows :
| % make certificate TYPE=existing (existing cert) | | CRT=/path/to/your.crt [KEY=/path/to/your.key] | | | | Use TYPE=dummy when you're a vendor package maintainer, | | the TYPE=test when you're an admin but want to do tests only, | | the TYPE=custom when you're an admin willing to run a real server | | and TYPE=existing when you're an admin who upgrades a server. | | (The default is TYPE=test) | | | | Additionally add ALGO=RSA (default) or ALGO=DSA to select | | the signature algorithm used for the generated certificate. | | | | Use 'make certificate VIEW=1' to display the generated data. | | | | Thanks for using Apache & mod_ssl. Ralf S. Engelschall | | rse@engelschall.com | | www.engelschall.com | +---------------------------------------------------------------------+ make[1]: Leaving directory `/usr/local/src/apache_1.3.34' <=== src
As you can see Apache & mod_ssl are already built right now. All we have to do now is to install the files to /usr/local/apache/1.3.39/ by typing make install.
# make install
make[1]: Leaving directory `/usr/local/src/apache_1.3.34' +--------------------------------------------------------+ | You now have successfully built and installed the | | Apache 1.3 HTTP server. To verify that Apache actually | | works correctly you now should first check the | | (initially created or preserved) configuration files | | | | /usr/local/apache/1.3.39/conf/httpd.conf | | | and then you should be able to immediately fire up | | Apache the first time by running: | | | | /usr/local/apache/1.3.39/bin/apachectl start | | | Or when you want to run it with SSL enabled use: | | | | /usr/local/apache/1.3.39/bin/apachectl startssl | | | Thanks for using Apache. The Apache Group | | http://www.apache.org/ | +--------------------------------------------------------+
8.1.3 Rebuild Apache Benchmark to support SSL
Even if you build Apache the way we did before ( prepatching apache's source before the build ) the Apache Benchmark binary "ab" does not support SSL. If you don't need to benchmark your Apache to figure out how much connections your machine can handle you can skip this part and move on to building PHP.
We are going to remedy that disfunction properly by starting to delete the current ab binary :
# cd /usr/local/src/apache-1.3.39/src/support # rm -f ab
Now we'll have to rebuild ab manually with the following make string to get it to support SSL enablend benchmarks :
# make ab CFLAGS="${CFLAGS} -DUSE_SSL -lcrypto -lssl -I/usr/include/openssl -L/usr/lib" || die
gcc -DUSE_SSL -lcrypto -lssl -I/usr/include/openssl -L/usr/lib -o ab -L../os/unix -L../ap ab.o -lm -lap -los -lm -lcrypt -lndbm -lexpat -ldl
After the binary is build it has to be copied manually into the appropriate apache binary folder :
# cp ab /usr/local/apache/1.3.39/bin/
8.2 Building the PHP Module
# cd /usr/local/src/php-5.2.4
The next step is to build the PHP SAPI Module for Apache and the PHP CLI. The following build string fits our needs - you probably will have different topics to cover so consider this only as an example. You can have a thorough look for yourself on http://www.php.net/manual/en/configure.php where you'll find a list of the most common configure options.
I recommend to build the PHP configure string in a text editor and copy the complete screen into your shell. This way you can always tweak things if your configure screws up and if it doesn't you can instantly copy it into your own documentation. Trust me, after 3 month you'll have forgotten most of the options and you'll be grateful to have some documentation of your own.
If you are using a gentoo system you'll most likely want to use the same CFLAGS your gentoo system uses to build its packages and copy the CFLAGS out of /etc/make.conf. The CFLAGS used in the example below are working perfectly for us on a x86 Pentium3 ( Dual-Tualatin ) machine.
CFLAGS="-march=pentium3 -mtune=pentium3 -O3 -pipe -fomit-frame-pointer" \ ./configure --with-apxs=/usr/local/apache/1.3.39/bin/apxs --with-gettext=/usr \ --with-mysql --with-zlib-dir=/usr --enable-calendar --with-freetype-dir=/usr \ --with-zlib=/usr --with-gd --enable-gd-native-ttf --with-config-file-path=/etc \ --with-openssl=/usr \ --enable-sysvshm=yes --enable-sysvsem=yes --enable-sysvmsg --enable-bcmath \ --without-mm --enable-inline-optimization \ --enable-pcntl --enable-sigchild --with-curl --enable-mbstring \ --with-freetype-dir=/usr --enable-exif --with-mcrypt=/usr --with-mhash=/usr
This should output this :
Generating files updating cache ./config.cache creating ./config.status creating php4.spec creating main/build-defs.h creating scripts/phpize creating scripts/man1/phpize.1 creating scripts/php-config creating scripts/man1/php-config.1 creating sapi/cli/php.1 creating main/php_config.h creating main/internal_functions.c creating main/internal_functions_cli.c +--------------------------------------------------------------------+ | License: | | This software is subject to the PHP License, available in this | | distribution in the file LICENSE. By continuing this installation | | process, you are bound by the terms of this license agreement. | | If you do not agree with the terms of this license, you must abort | | the installation process at this point. | +--------------------------------------------------------------------+ | *** NOTE *** | | The default for register_globals is now OFF! | | | | If your application relies on register_globals being ON, you | | should explicitly set it to on in your php.ini file. | | Note that you are strongly encouraged to read | | http://www.php.net/manual/en/security.globals.php | | about the implications of having register_globals set to on, and | | avoid using it if possible. | +--------------------------------------------------------------------+ Thank you for using PHP.
After configuring PHP we have to build it, of course.
# make -j3
main/internal_functions_cli.lo -lcrypt -lcrypt -lpq -lmysqlclient -lmhash -lmcrypt -lltdl -lfreetype -lpng -lz -lz -lssl -lcrypto -lresolv -lm -ldl -lnsl -lcrypt -lcrypt -o sapi/cli/php /usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.5-20050130/../../../../i686-pc-linux- gnu/bin/ld: warning: creating a DT_TEXTREL in object. Build complete. (It is safe to ignore warnings about tempnam and tmpnam).
As the build says, it really is safe to ignore warnings about tempnam and tmpnam. The next step obviously would be to install the PHP package but we'd like to run some tests first :
# make test
===================================================================== TIME END 2005-10-28 14:20:27 ===================================================================== TEST RESULT SUMMARY --------------------------------------------------------------------- Exts skipped : 65 Exts tested : 23 --------------------------------------------------------------------- Number of tests : 630 Tests skipped : 242 (38.4%) Tests warned : 0 (0.0%) Tests failed : 0 (0.0%) Tests passed : 388 (60.8%) --------------------------------------------------------------------- Time taken : 30 seconds ===================================================================== Please allow this report to be sent to the PHP QA team. This will give us a better understanding in how PHP's test cases are doing. Note that the report will include detailed configuration data about your system so if you are worried about exposing sensitive data, save this to a file first and remove any sensitive data and then send this file to php-qa@lists.php.net. (choose "s" to just save the results to a file)? [Yns]: n
Now we can install the build and tested package. The modules will be installed into /usr/local/apache/1.3.34/libexec and the CLI binary will be installed into /usr/local/bin This step causes no version conflicts when you update an existing configuration.
# make install
Installing PHP SAPI module: apache [activating module `php4' in /usr/local/apache/1.3.34/conf/httpd.conf] cp libs/libphp4.so /usr/local/apache/1.3.34/libexec/libphp4.so chmod 755 /usr/local/apache/1.3.34/libexec/libphp4.so cp /usr/local/apache/1.3.34/conf/httpd.conf /usr/local/apache/1.3.34/conf/httpd.conf.bak cp /usr/local/apache/1.3.34/conf/httpd.conf.new /usr/local/apache/1.3.34/conf/httpd.conf rm /usr/local/apache/1.3.34/conf/httpd.conf.new Installing PHP CLI binary: /usr/local/bin/ Installing PHP CLI man page: /usr/local/man/man1/ Installing PEAR environment: /usr/local/lib/php/ [PEAR] Archive_Tar - already installed: 1.1 [PEAR] Console_Getopt - already installed: 1.2 [PEAR] PEAR - already installed: 1.3.5 Wrote PEAR system config file at: /usr/local/etc/pear.conf You may want to add: /usr/local/lib/php to your php.ini include_path [PEAR] HTML_Template_IT- already installed: 1.1 [PEAR] Net_UserAgent_Detect- already installed: 2.0.1 [PEAR] XML_RPC - already installed: 1.3.1 Installing build environment: /usr/local/lib/php/build/ Installing header files: /usr/local/include/php/ Installing helper programs: /usr/local/bin/ program: phpize program: php-config Installing man pages: /usr/local/man/man1/ page: phpize.1 page: php-config.1
The next steps only apply to you if you install a new server. If you update Apache, PHP or some modules you don't want to screw up your existing configuration.
# cp php.ini-recomended /etc/php.ini
8.3 Install Zend Optimizer
# cd /usr/local/src/ZendOptimizer-2.5.10a-linux-glibc21-i386 # ./install.sh
Follow the on screen instructions
8.4 Building ModSecurity via APXS
To build ModSecurity we first change into its source directory ...
# cd /usr/local/src/modsecurity-1.9.5
Then we are going to build the module via apxs of our freshly built Apache 1.3.39 ...
# /usr/local/apache/1.3.39/bin/apxs -cia apache1/mod_security.c
This should create the following output :
gcc -DLINUX=22 -DHAVE_SET_DUMPABLE -I/usr/include/db1 -DMOD_SSL=208125 -DUSE_HSREGEX -DEAPI -march=pentium3 -mcpu=pentium3 -O3 -pipe -fomit-frame-pointer -fpic -DSHARED_MODULE -I/usr/local/apache/1.3.39/include -c apache1/mod_security.c gcc -shared -o apache1/mod_security.so mod_security.o [activating module `security' in /usr/local/apache/1.3.39/conf/httpd.conf] cp apache1/mod_security.so /usr/local/apache/1.3.39/libexec/mod_security.so chmod 755 /usr/local/apache/1.3.39/libexec/mod_security.so cp /usr/local/apache/1.3.39/conf/httpd.conf /usr/local/apache/1.3.39/conf/httpd.conf.bak cp /usr/local/apache/1.3.39/conf/httpd.conf.new /usr/local/apache/1.3.39/conf/httpd.conf rm /usr/local/apache/1.3.39/conf/httpd.conf.new
8.5 Building ModGZip via APXS
ModGZip is an Apache module which is built via Apache's APXS. ModGZip comes with a Makefile
which is best left untouched. To build it we change into its source directory ...
# cd /usr/local/src/mod_gzip-1.3.26.1a
and then type :
# make APXS=/usr/local/apache/1.3.39/bin/apxs
When you see something like this your build procedure should be completed :
/usr/local/apache/1.3.39/bin/apxs -Wc,-Wall,-O3,-fomit-frame-pointer,-pipe -c mod_gzip.c mod_gzip_debug.c mod_gzip_compress.c -o mod_gzip.so gcc -DLINUX=22 -DHAVE_SET_DUMPABLE -I/usr/include/db1 -DMOD_SSL=208125 -DUSE_HSREGEX -DEAPI -march=pentium3 -mcpu=pentium3 -O3 -pipe -fomit-frame-pointer -fpic -DSHARED_MODULE -I/usr/local/apache/1.3.39/include -Wall,-O3,-fomit-frame-pointer,-pipe -c mod_gzip.c gcc -DLINUX=22 -DHAVE_SET_DUMPABLE -I/usr/include/db1 -DMOD_SSL=208125 -DUSE_HSREGEX -DEAPI -march=pentium3 -mcpu=pentium3 -O3 -pipe -fomit-frame-pointer -fpic -DSHARED_MODULE -I/usr/local/apache/1.3.39/include -Wall,-O3,-fomit-frame-pointer,-pipe -c mod_gzip_debug.c gcc -DLINUX=22 -DHAVE_SET_DUMPABLE -I/usr/include/db1 -DMOD_SSL=208125 -DUSE_HSREGEX -DEAPI -march=pentium3 -mcpu=pentium3 -O3 -pipe -fomit-frame-pointer -fpic -DSHARED_MODULE -I/usr/local/apache/1.3.39/include -Wall,-O3,-fomit-frame-pointer,-pipe -c mod_gzip_compress.c gcc -shared -o mod_gzip.so mod_gzip_compress.o mod_gzip_debug.o mod_gzip.o -o mod_gzip.so -Wc,-Wall,-O3,-fomit-frame-pointer,-pipe
# make install APXS=/usr/local/apache/1.3.39/bin/apxs
/usr/local/apache/1.3.39/bin/apxs -A -i mod_gzip.so [preparing module `gzip' in /usr/local/apache/1.3.39/conf/httpd.conf] cp mod_gzip.so /usr/local/apache/1.3.39/libexec/mod_gzip.so chmod 755 /usr/local/apache/1.3.39/libexec/mod_gzip.so cp /usr/local/apache/1.3.39/conf/httpd.conf /usr/local/apache/1.3.39/conf/httpd.conf.bak cp /usr/local/apache/1.3.39/conf/httpd.conf.new /usr/local/apache/1.3.39/conf/httpd.conf rm /usr/local/apache/1.3.39/conf/httpd.conf.new
9 Configuration
9.1 httpd.conf
Example global configuration :
ServerType standalone ServerRoot "/usr/local/apache/current" PidFile /var/run/httpd.pid ScoreBoardFile /var/log/apache/httpd.scoreboard Timeout 300 KeepAlive On MaxKeepAliveRequests 100 KeepAliveTimeout 15 MinSpareServers 5 MaxSpareServers 10 StartServers 5 MaxClients 150 MaxRequestsPerChild 0 ################################################################ ### load mod_security first otherwise it won't work properly ### LoadModule security_module libexec/mod_security.so LoadModule vhost_alias_module libexec/mod_vhost_alias.so LoadModule env_module libexec/mod_env.so LoadModule define_module libexec/mod_define.so LoadModule config_log_module libexec/mod_log_config.so LoadModule mime_magic_module libexec/mod_mime_magic.so LoadModule mime_module libexec/mod_mime.so LoadModule includes_module libexec/mod_include.so LoadModule dir_module libexec/mod_dir.so LoadModule cgi_module libexec/mod_cgi.so LoadModule action_module libexec/mod_actions.so LoadModule alias_module libexec/mod_alias.so LoadModule rewrite_module libexec/mod_rewrite.so LoadModule access_module libexec/mod_access.so LoadModule auth_module libexec/mod_auth.so LoadModule anon_auth_module libexec/mod_auth_anon.so LoadModule digest_module libexec/mod_digest.so LoadModule proxy_module libexec/libproxy.so LoadModule cern_meta_module libexec/mod_cern_meta.so LoadModule expires_module libexec/mod_expires.so LoadModule headers_module libexec/mod_headers.so LoadModule usertrack_module libexec/mod_usertrack.so LoadModule log_forensic_module libexec/mod_log_forensic.so LoadModule unique_id_module libexec/mod_unique_id.so LoadModule setenvif_module libexec/mod_setenvif.so LoadModule php4_module libexec/libphp4.so LoadModule gzip_module libexec/mod_gzip.so <IfDefine SSL> LoadModule ssl_module libexec/libssl.so </IfDefine> ClearModuleList AddModule mod_security.c AddModule mod_vhost_alias.c AddModule mod_env.c AddModule mod_define.c AddModule mod_log_config.c AddModule mod_mime_magic.c AddModule mod_mime.c AddModule mod_include.c AddModule mod_dir.c AddModule mod_cgi.c AddModule mod_actions.c AddModule mod_alias.c AddModule mod_rewrite.c AddModule mod_access.c AddModule mod_auth.c AddModule mod_auth_anon.c AddModule mod_digest.c AddModule mod_proxy.c AddModule mod_cern_meta.c AddModule mod_expires.c AddModule mod_headers.c AddModule mod_usertrack.c AddModule mod_log_forensic.c AddModule mod_unique_id.c AddModule mod_so.c AddModule mod_setenvif.c AddModule mod_php4.c AddModule mod_gzip.c <IfDefine SSL> AddModule mod_ssl.c </IfDefine>
9.1.1 global ModSecurity configuration
################################################################ ### global mod_security configuration ########################## <IfModule mod_security.c> # Turn the filtering engine On or Off SecFilterEngine On Include /etc/mod_sec.conf # set chroot path SecChrootDir /www # Make sure that URL encoding is valid SecFilterCheckURLEncoding On # Unicode encoding check SecFilterCheckUnicodeEncoding On # Only allow bytes from this range SecFilterForceByteRange 1 255 # Only log suspicious requests SecAuditEngine RelevantOnly # The name of the audit log file SecAuditLog /var/log/apache/modsec_audit_log # Debug level set to a minimum SecFilterDebugLog /var/log/apache/modsec_debug_log SecFilterDebugLevel 0 # Should mod_security inspect POST payloads SecFilterScanPOST On # By default log and deny suspicious requests with HTTP status 406 SecFilterDefaultAction "deny,log,status:406" </IfModule>
9.1.2 Global ModGZip configuration
################################################################ ### global mod_gzip configuration ############################## <IfModule mod_gzip.c> mod_gzip_on yes mod_gzip_dechunk yes mod_gzip_keep_workfiles No mod_gzip_temp_dir /tmp mod_gzip_minimum_file_size 500 mod_gzip_maximum_file_size 5000000 mod_gzip_maximum_inmem_size 1000000 mod_gzip_add_header_count Yes mod_gzip_min_http 1001 mod_gzip_item_include file \.htm$ mod_gzip_item_include file \.html$ mod_gzip_item_include mime ^text/html mod_gzip_item_include mime ^text/plain mod_gzip_item_include file \.php$ mod_gzip_item_include handler x-httpd-php mod_gzip_item_include mime ^application/x-httpd-php$ mod_gzip_item_include mime httpd/unix-directory mod_gzip_item_exclude file "\.css$" mod_gzip_item_exclude file "\.js$" # %h %l %u %t \"%V %r\" %<s %b mod_gzip: LogFormat "%h \"%V %r\" - %{mod_gzip_result}n In:%{mod_gzip_input_size}n - Out:%{mod_gzip_output_size}n = %{mod_gzip_compression_ratio}n Prozent." mod_gzip_info2 CustomLog /var/log/apache/mod_gzip.log mod_gzip_info2 </IfModule>
Example Reverse Proxy Container :
### test.radion.org ####################################################################### <VirtualHost *:80> ServerAdmin youremail@yourdomain.com ServerName yourhost.yourdomain.org ServerAlias yourhost.anotherdomain.org www.yourdomain.com LogLevel warn ErrorLog /var/log/apache/container/host.yourdomain.org/error_log CustomLog /var/log/apache/container/host.yourdomain.org/access_log combined env=!dontlog CustomLog /var/log/apache/container/host.yourdomain.org/clickstream "%h %t %{cookie}n %r %t" ProxyPass / http://your.internal.appserver.com/ ProxyPassReverse / http://your.internal.appserver.com/ </VirtualHost>
9.2 tweaking the system to use SSL in JailRoot
The JailRoot where all Apache children are confined brings a major dilemma when it comes to SSL. Obviously the parent process and the children involved in SSL communication have to share the same SSL mutex. If your httpd.conf is properly configured you find the following section :
Check that the variables SSLSessionCache and SSLMutex are pointing to /var/cache/apache/ssl/ as described in the httpd.conf section above.
<IfModule mod_ssl.c> SSLPassPhraseDialog builtin SSLSessionCache dbm:/var/cache/apache/ssl/ssl_scache SSLSessionCacheTimeout 300 SSLMutex file:/var/cache/apache/ssl/ssl_mutex SSLRandomSeed startup file:/dev/urandom 512 SSLRandomSeed connect file:/dev/urandom 512 SSLLog /var/log/apache/ssl_engine_log SSLLogLevel warn </IfModule>
Now you should create the directory /www/var/apache/ssl
# mkdir /www/var/ # mkdir /www/var/apache # mkdir /www/var/apache/ssl
We have to link back this directory to the real /var/ so that the parent process see's exactly the same path with exactly the same files :
# ln -s /www/var/cache/apache/ /var/cache
Your /var/cache directory should look like similiar to this :
# ls -al /var/cache total 16 drwxr-xr-x 4 root root 4096 Sep 10 08:34 . drwxr-xr-x 13 root root 4096 Oct 5 18:57 .. lrwxrwxrwx 1 wwwrun wwwrun 22 Oct 9 17:30 apache -> /www/var/cache/apache/ drwxrwxr-x 3 root portage 4096 Oct 10 15:02 edb drwxrwxr-x 19 root man 4096 Sep 6 00:09 man
Both /var/cache/apache/ssl/ and /www/var/cache/apache/ssl/ should now contain the same content :
# ls -al /www/var/cache/apache/ssl
total 16 drwxr-xr-x 2 wwwrun wwwrun 4096 Oct 29 12:44 . drwxr-xr-x 3 wwwrun wwwrun 4096 Sep 10 08:33 .. -rw------- 1 wwwrun root 0 Oct 29 12:44 ssl_mutex.11801 -rw------- 1 wwwrun root 0 Sep 10 23:13 ssl_mutex.19929 -rw------- 1 wwwrun root 0 Oct 7 21:56 ssl_mutex.26688 -rw------- 1 wwwrun root 0 Oct 29 12:44 ssl_scache.dir -rw------- 1 wwwrun root 8192 Oct 29 23:20 ssl_scache.pag
# ls -al /var/cache/apache/ssl/
total 16 drwxr-xr-x 2 wwwrun wwwrun 4096 Oct 29 12:44 . drwxr-xr-x 3 wwwrun wwwrun 4096 Sep 10 08:33 .. -rw------- 1 wwwrun root 0 Oct 29 12:44 ssl_mutex.11801 -rw------- 1 wwwrun root 0 Sep 10 23:13 ssl_mutex.19929 -rw------- 1 wwwrun root 0 Oct 7 21:56 ssl_mutex.26688 -rw------- 1 wwwrun root 0 Oct 29 12:44 ssl_scache.dir -rw------- 1 wwwrun root 8192 Oct 29 23:24 ssl_scache.pag
9.3 /etc/init.d/apache script
Here is a modified gentoo apache start/stop script to be placed into /etc/init.d/
#!/sbin/runscript # Copyright 1999-2004 Gentoo Foundation # Distributed under the terms of the GNU General Public License v2 # $Header: /var/cvsroot/gentoo-x86/net-www/apache/files/apache.rc6,v 1.20 2004/07/15 00:24:42 agriffis Exp $ opts="${opts} reload configcheck" depend() { need net use dns logger netmount before sshd } start() { ebegin "Starting Apache HTTP Server" env -i PATH=$PATH PERL5LIB=PERL5LIB /sbin/start-stop-daemon -o --quiet \ --start --startas /usr/local/apache/current/bin/httpd \ --pidfile /var/run/httpd.pid -- ${APACHE_OPTS} eend $? } stop() { ebegin "Pre-Checking httpd.conf" /usr/local/apache/current/bin/httpd -T ${APACHE_OPTS} &> /dev/null if [ "$?" = "0" ] then einfo "Shutting down Apache HTTP Server" #/usr/local/apache/current/bin/apachectl stop >/dev/null start-stop-daemon -o --quiet --stop --pidfile /var/run/httpd.pid eend $? else einfo "Errors in httpd.conf - SHUTDOWN aborted" eend 128 fi } configcheck() { ebegin "Re-Check httpd.conf" /usr/local/apache/current/bin/httpd -T ${APACHE_OPTS} eend $? } reload() { ebegin "Gracefully restarting apache" /usr/local/apache/current/bin/httpd -t ${APACHE_OPTS} &>/dev/null if [ "$?" = "0" ] then if [ -f /var/run/httpd.pid ] then kill -USR1 $(</var/run/httpd.pid) eend $? else svc_start eend $? fi else if [ -f /var/run/httpd.pid ] then svc_stop fi #show the error(s) /usr/local/apache/current/bin/httpd -t ${APACHE_OPTS} eend 1 fi }
Until now I did not find any way to restart Apache gracefully. If you send a kill -USR1 to httpd manually Apache will simply die but not start again. If someone else has this problem or even has found a solution for that particular problem, don't hesitate to change it directly here in the wiki or send us an email.
Also, if you can contribute working init scripts for other UNIXes / Linuxes we would gladly appreciate them.
10 Activate apache
# ln -sf /usr/local/apache/1.3.34 /usr/local/apache/current
# ls -al /usr/local/apache
total 16 drwxr-xr-x 4 root root 4096 Sep 12 10:17 . drwxr-xr-x 13 root root 4096 Sep 10 18:25 .. drwxr-xr-x 9 root root 4096 Sep 12 10:17 1.3.34 lrwxrwxrwx 1 root root 7 Sep 10 18:08 current -> 1.3.34/
11 Testing
11.1 General function
top - 23:31:29 up 20 days, 4:38, 4 users, load average: 0.01, 0.02, 0.00 Tasks: 50 total, 1 running, 49 sleeping, 0 stopped, 0 zombie Cpu0 : 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0% si Cpu1 : 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0% si Mem: 1556012k total, 884264k used, 671748k free, 211920k buffers Swap: 1004052k total, 0k used, 1004052k free, 336840k cached PID PPID TIME+ P %CPU %MEM PR NI S VIRT SWAP RES SHR UID COMMAND 13832 11440 0:00.18 1 0.3 0.1 16 0 R 1860 916 944 748 0 top 12019 11802 0:35.60 0 0.0 1.2 16 0 S 21612 3656 17m 3560 666 /usr/local/apache/current/bin/httpd -D SSL 11838 11802 0:35.50 1 0.0 1.1 16 0 S 21496 3660 17m 3556 666 /usr/local/apache/current/bin/httpd -D SSL 11811 11802 0:45.61 0 0.0 1.2 16 0 S 21552 3624 17m 3592 666 /usr/local/apache/current/bin/httpd -D SSL 11810 11802 0:32.76 0 0.0 1.2 16 0 S 21572 3576 17m 3640 666 /usr/local/apache/current/bin/httpd -D SSL 11809 11802 0:47.12 0 0.0 1.1 16 0 S 21452 3560 17m 3656 666 /usr/local/apache/current/bin/httpd -D SSL 11807 11802 0:38.64 1 0.0 1.3 16 0 S 24424 3660 20m 3560 666 /usr/local/apache/current/bin/httpd -D SSL 11806 11802 0:40.73 0 0.0 1.4 16 0 S 24908 3660 20m 3564 666 /usr/local/apache/current/bin/httpd -D SSL 11805 11802 0:33.50 0 0.0 1.1 16 0 S 21228 3640 17m 3572 666 /usr/local/apache/current/bin/httpd -D SSL 11804 11802 0:41.47 1 0.0 1.3 16 0 S 24316 3640 20m 3576 666 /usr/local/apache/current/bin/httpd -D SSL 11803 11802 0:45.92 0 0.0 3.4 16 0 S 69668 16m 51m 3644 666 /usr/local/apache/current/bin/httpd -D SSL 11802 1 0:00.18 1 0.0 0.5 16 0 S 11928 4304 7624 2968 0 /usr/local/apache/current/bin/httpd -D SSL 5874 1 0:00.00 0 0.0 0.0 16 0 S 1504 884 620 540 0 /sbin/agetty 38400 tty1 linux 5804 1 0:00.00 1 0.0 0.0 16 0 S 1500 880 620 540 0 /sbin/agetty 38400 tty3 linux 5803 1 0:00.00 0 0.0 0.0 16 0 S 1504 884 620 540 0 /sbin/agetty 38400 tty2 linux 5775 1 0:08.27 0 0.0 0.1 16 0 S 3412 1876 1536 1260 0 /usr/sbin/sshd 4758 1 0:02.80 0 0.0 0.0 15 0 S 1716 984 732 564 0 /usr/sbin/syslog-ng 3970 1 0:02.63 0 0.0 0.0 15 0 S 0 0 0 0 0 [kjournald] 3552 1 0:00.43 1 0.0 0.0 16 0 S 1716 988 728 616 0 /usr/sbin/cron 766 1 0:00.08 1 0.0 0.0 12 -4 S 1472 984 488 416 0 udevd 711 1 0:00.79 0 0.0 0.0 15 0 S 0 0 0 0 0 [kjournald] 710 1 0:00.00 1 0.0 0.0 15 0 S 0 0 0 0 0 [kirqd] 653 1 0:00.18 0 0.0 0.0 15 0 S 0 0 0 0 0 [kseriod] 64 9 0:00.00 1 0.0 0.0 19 -5 S 0 0 0 0 0 [aio/1] 63 9 0:00.00 0 0.0 0.0 19 -5 S 0 0 0 0 0 [aio/0] 62 1 0:00.00 0 0.0 0.0 25 0 S 0 0 0 0 0 [kswapd0] 61 9 0:00.42 0 0.0 0.0 15 0 S 0 0 0 0 0 [pdflush] 60 9 0:00.00 1 0.0 0.0 20 0 S 0 0 0 0 0 [pdflush] 14 9 0:00.00 1 0.0 0.0 10 -5 S 0 0 0 0 0 [kblockd/1] 13 9 0:00.26 0 0.0 0.0 10 -5 S 0 0 0 0 0 [kblockd/0] 12 1 0:00.52 0 0.0 0.0 6 -10 S 0 0 0 0 0 [vesafb] 9 1 0:00.00 1 0.0 0.0 18 -5 S 0 0 0 0 0 [kthread] 8 1 0:00.59 1 0.0 0.0 10 -5 S 0 0 0 0 0 [khelper] 7 1 0:00.00 1 0.0 0.0 10 -5 S 0 0 0 0 0 [events/1] 6 1 0:00.07 0 0.0 0.0 10 -5 S 0 0 0 0 0 [events/0] 5 1 0:00.00 1 0.0 0.0 34 19 S 0 0 0 0 0 [ksoftirqd/1] 4 1 0:00.41 1 0.0 0.0 RT 0 S 0 0 0 0 0 [migration/1] 3 1 0:00.00 0 0.0 0.0 34 19 S 0 0 0 0 0 [ksoftirqd/0] 2 1 0:00.31 0 0.0 0.0 RT 0 S 0 0 0 0 0 [migration/0] 1 0 0:02.68 1 0.0 0.0 16 0 S 1464 960 504 440 0 init [3]
# netstat -patune | grep httpd
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 0 423856 11802/httpd tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 0 423855 11802/httpd
11.2 PHP
11.3 ModSSL
11.4 ModSecurity
11.5 ModGZip
[http://leknor.com/code/gziped.php http://leknor.com/code/gziped.php}
12 Pitfalls & Caveats
Due to ModSecurity's chroot design there are some pitfalls to be aware of :
| Module Order: | ModSecurity requires a specific Apache module execution order to run properly with all modules you want to include into Apache. Don't mess with ModSecurity's position in the module execution order. |
| Hint: | The parent Apache process needs an ssl-mutex and so does the jailrooted child if handling SSL requests therefore both need to read & write to the same mutex file. To not to make this a jailroot violation you'll have to link it from /www/var/ to /var |
| Hint: | PHP, SSL and some Apache modules ( or even your own code ) will need /tmp to work properly but after chrooting Apache has only access to files and directories below /www so we need a separate /www/tmp directory there. |
| Hint: | If you use ModProxy or any other code which needs /etc/hosts to resolve hostnames be aware of the fact that /etc cannot be seen from inside the jailroot. You can link /www/etc/hosts to /etc/hosts but I recommend to create the directory /www/etc and put your own hosts file in there. This way you can even control the behaviour of ModProxy or your own code differently from what your machine itself would do. |
| Hint: | ModSSL behaves even worse because it needs access to some files IN and OUTside of the jailroot and both path's have to be the same. If Apache refuses to start or missbehaves - have a thorough look into your error_log files. . |
| Hint: | Apache graceful restarting DOES NOT WORK anymore. Once chrooted, Apache cannot access anything located above ChrootDir. For that reason restarting Apache with 'apachectl reload', 'apachectl graceful' or 'kill -HUP apache_pid' will not work as expected. Apache will not be able to read its config file, open logs or load modules once chrooted. |
| Shared libraries: | Shared libraries are libraries which are linked to a program at run-time. Nowadays, most programs require some shared libraries to run - libc.so is most common. You can see a list of shared libraries a program requires by running ldd /path/to/program. Loading of these libraries is done automagically by ld.so at startup. mod_chroot doesn't interfere with this mechanism.
A program may also explicitly load a shared library by calling dlopen() and dlsym(). This might cause troubles in a chrooted environment - after a process is chrooted, libraries (usually stored in /lib) might be no longer accessible. This doesn't happen very often, but if it does - there is a solution: you can preload these libraries before chrooting. Apache has a handy directive for that: LoadFile. This is what people reported on the mailing list: DNS lookups - GNU libc tries to load libnss_dns.so.2 when a first DNS lookup is done. Solution: LoadFile /lib/libnss_dns.so.2Apache 2.0 with mpm_worker on Linux 2.6 - GNU libc tries to load libgcc_s.so.1 when pthread_cancel is called. Solution: LoadFile /lib/libgcc_s.so.1 |
