Linux Consulting: Friday, December 09, 2005

Abusing bots are a problem for all successful sites. When dealing with abusive bots, you can take the WebmasterWorld's senseless route to block all user agents or you can be a bit smart, and install mod_security. That way you won't have to block all the indexing bots and no one will call you stupid.

In this post, I will show you how to secure your Apache 2 installation with mod_security

First, we want to make sure that we have apxs installed. try

which apxs

to see what you get. If apxs is not installed then you will need to install it. Hint: Install the httpd-devel rpm for your machine.

In my case I had to look for it. I ended up finding one here.

Once you have located the RPM, installing it is a breeze:

wget http://vps-demo.freevps.com/up2date/RHEL3/httpd-devel-2.0.46-44.ent.i386.rpm
rpm -ivh httpd-devel-2.0.46-44.ent.i386.rpm
# verify httpd-devel is now installed
rpm -qa | grep httpd

If you are using the wrong RPM, you can expect to get errors similar to the following (abbreviated)

file /usr/sbin/httpd from install of httpd-2.0.46-54.ent conflicts with file from package httpd-2.0.46-44.ent
file /usr/sbin/httpd.worker from install of httpd-2.0.46-54.ent conflicts with file from package httpd-2.0.46-44.ent
file /usr/sbin/rotatelogs from install of httpd-2.0.46-54.ent conflicts with file from package httpd-2.0.46-44.ent
file /usr/sbin/suexec from install of httpd-2.0.46-54.ent conflicts with file from package httpd-2.0.46-44.ent

Now that we have apxs installed, lets continue with installing mod_security

wget http://www.modsecurity.org/download/modsecurity-apache-1.9.1.tar.gz
tar -zxvf modsecurity-apache-1.9.1.tar.gz
cd modsecurity-apache-1.9.1/apache2/
# backup configuration files
cp -p -r /etc/httpd /etc/httpdbk
# make sure that we have mod_security.c in the current dir
ls
# config.m4 Makefile.in makefile.win mod_security.c

Actual Installation (compiling mod_security)

[root@srv30 apache2]# /usr/sbin/apxs -cia mod_security.c
/usr/bin/libtool --silent --mode=compile gcc -prefer-pic -O2 -g -pipe -march=i386 -mcpu=i686 -DSSL_EXPERIMENTAL_ENGINE -I/usr/kerberos/include -DAP_HAVE_DESIGNATED_INITIALIZER -DLINUX=2 -D_REENTRANT -D_XOPEN_SOURCE=500 -D_BSD_SOURCE -D_SVID_SOURCE -D_GNU_SOURCE -pthread -I/usr/include/httpd -c -o mod_security.lo mod_security.c && touch mod_security.slo
/usr/bin/libtool --silent --mode=link gcc -o mod_security.la -rpath /usr/lib/httpd/modules -module -avoid-version mod_security.lo
/usr/lib/httpd/build/instdso.sh SH_LIBTOOL='/usr/bin/libtool' mod_security.la /usr/lib/httpd/modules
/usr/bin/libtool --mode=install cp mod_security.la /usr/lib/httpd/modules/
cp .libs/mod_security.so /usr/lib/httpd/modules/mod_security.so
cp .libs/mod_security.lai /usr/lib/httpd/modules/mod_security.la
cp .libs/mod_security.a /usr/lib/httpd/modules/mod_security.a
ranlib /usr/lib/httpd/modules/mod_security.a
chmod 644 /usr/lib/httpd/modules/mod_security.a
PATH="$PATH:/sbin" ldconfig -n /usr/lib/httpd/modules
----------------------------------------------------------------------
Libraries have been installed in:
/usr/lib/httpd/modules

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
- add LIBDIR to the `LD_LIBRARY_PATH' environment variable
during execution
- add LIBDIR to the `LD_RUN_PATH' environment variable
during linking
- use the `-Wl,--rpath -Wl,LIBDIR' linker flag
- have your system administrator add LIBDIR to `/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
chmod 755 /usr/lib/httpd/modules/mod_security.so
[activating module `security' in /etc/httpd/conf/httpd.conf]

To see what changes exactly were made to the httpd.conf file:

[root@srv30 apache2]# diff /etc/httpd/conf/httpd.conf /etc/httpdbk/httpd/conf/httpd.conf
216d215
< LoadModule security_module /usr/lib/httpd/modules/mod_security.so

On ensim, you may get an error saying that mod_security.so isn't available. If so, the following (or something similar should fix it.

cp -p /usr/lib/apache/mod_security.so /etc/httpd/lib/apache/mod_security.so

Now we need to get some rules.

mkdir modsecurity-rules
cd modsecurity-rules
wget http://www.gotroot.com/downloads/ftp/mod_security/apache2/apache2-gotrootrules-latest.tar.gz

Extract the files

[root@srv30 modsecurity-rules] tar -zxvf apache2-gotrootrules-latest.tar.gz
apache2/proxy.conf
apache2/apache2-rules.conf
apache2/blacklist2.conf
apache2/rules.conf
apache2/blacklist.conf
apache2/rootkits.conf
apache2/exclude.conf
apache2/badips.conf
apache2/useragents.conf
apache2/recons.conf

Create a file /etc/httpd/conf.d/modsecurity.conf and put something like the following (or see httpd.conf.example-minimal file)



# Only inspect dynamic requests
# (YOU MUST TEST TO MAKE SURE IT WORKS AS EXPECTED)
#SecFilterEngine DynamicOnly

SecFilterEngine On

# Reject requests with status 500
SecFilterDefaultAction "deny,log,status:500"

# Some sane defaults
SecFilterScanPOST On
SecFilterCheckURLEncoding On
SecFilterCheckCookieFormat On
SecFilterCheckUnicodeEncoding Off
SecFilterNormalizeCookies On
# enable version 1 (RFC 2965) cookies
SecFilterCookieFormat 1

SecServerResponseToken Off

#If you want to scan the output, uncomment these
#SecFilterScanOutput On
#SecFilterOutputMimeTypes "(null) text/html text/plain"

# Accept almost all byte values
SecFilterForceByteRange 1 255

# Server masking is optional
#fake server banner - NOYB used - no one needs to know what we are using
SecServerSignature "NOYB"

#SecUploadDir /tmp
#SecUploadKeepFiles Off

# Only record the interesting stuff
SecAuditEngine RelevantOnly
SecAuditLog logs/audit_log

# You normally won't need debug logging
SecFilterDebugLevel 0
SecFilterDebugLog logs/modsec_debug_log

#And now, the rules
#Remove any of these Include lines you do not use or have rules for.

#First, add in your exclusion rules:
#These MUST come first!
Include /etc/modsecurity/exclude.conf

#Application protection rules
Include /etc/modsecurity/rules.conf

#Comment spam rules
Include /etc/modsecurity/blacklist.conf

#Bad hosts, bad proxies and other bad players
Include /etc/modsecurity/blacklist2.conf

#Bad clients, known bogus useragents and other signs of malware
Include /etc/modsecurity/useragents.conf

#Known bad software, rootkits and other malware
Include /etc/modsecurity/rootkits.conf

#Signatures to prevent proxying through your server
#only rule these rules if your server is NOT a proxy
Include /etc/modsecurity/proxy.conf

#Additional rules for Apache 2.x ONLY!  Do not add this line if you use Apache 1.x
Include /etc/modsecurity/apache2-rules.conf

Create directory and copy rules

mkdir /etc/modsecurity
cp -p apache2/*.conf /etc/modsecurity/

Now lets restart Apache

service httpd restart

Ban "Abusing" Bots
There are some bots that are going to abuse your servers. Now that you have modsecurity installed, you can easily ban them. For example, on my server, FunWebProducts took 3200+ pages in less than 30 minutes. They can go and kiss my ass while I have fun saving my bandwidth.

To block FunWebProducts, open up your useragents.conf file in /etc/modsecurity directory and add the following line

SecFilterSelective HTTP_USER_AGENT "FunWebProducts"
SecFilterSelective HTTP_USER_AGENT "Ichiro"

Now when FunWebProducts or Ichiro try to visit, we actually greet them with a 500 error

[Fri Dec 09 21:23:27 2005] [error] [client 210.150.10.100] mod_security: Access denied with code 500. Pattern match "ichiro" at HEADER("USER-AGENT") [hostname "domain-store.com"] [uri "/-21460"] [unique_id "Bnn1fM8s8FYAAEISjS4AAAAM"]

Thanks / Also see:

Setting up mod_security
Setting up mod_security rules
Mod Security for Apache Guide
Advanced Web Application Defense with ModSecurity (PDF, Daniel Fernandez Bleda & Christian Martorella)
Securing Web Services with mod_security (Shreeraj Shah - O'Reilly Network)
Firewall de Aplicaciones con Mod_Security (Spanish, Christian Martorella)
Advanced Web Application Defense with ModSecurity (PDF, Daniel Fernandez Bleda & Christian Martorella)
Introduction to ModSecurity (Persian, translated by Alan Baghumian)
Securing Web Services with mod_security (Shreeraj Shah for O'Reilly Network)
Firewall de Aplicaciones con Mod_Security (Spanish, Christian Martorella)
Locking down your Apache Web Server with mod_security (PPT, Hans Kind, FlyingServers)
Protecting your blog with mod_security (Anil, Six Apart)
blacklist_to_modsec.pl (Peter R. Wood)
The 80/20 Rule for Web Application Security (Jeremiah Grossman, WASC)
Defending Web Services using ModSecurity (Shreeraj Shah, InfosecWriters)
An introduction to mod_security (Atomic Playboy)
mod_security rule generator (Noel Jackson)
Enhancing Apache with mod_security (Joel A. Gallegos, FedoraNEWS)
ModSecurity - An Intrusion Prevention Module for Apache (PDF, Ryan C. Barnett, SANS)
Better Living Through Mod Security by Dhillon A. K. Introduces a PHP utility that parses the audit log and puts it into the database.
mod_security and FreeBSD HOWTO in Portuguese by Gleydson Soares.
Locking down your Apache Web Server with mod_security (PPT, Hans Kind, FlyingServers)

Linux Consulting

Friday, December 09, 2005

Securing Apache with mod_security

MySQL

My Blogs

Previous Posts

Archives