Fail2Ban – The Python Based and Popular IPS Framework

Fail2Ban is a commonly used Intrusion Prevention System (IPS) written in Python. With its inception as an open-source Python product in 2004 being a hit in the development community, development has continued to the present.

Fail2Ban's primary use case

Fail2Ban is designed to help servers of all types avoid brute force attacks. It utilizes a variety of customizable features to accomplish this goal, as we'll discuss further. First, there are a few important items to note before you consider employing Fail2Ban on your server. While the program is world-renowned and used on servers around the globe, there are a few things the program does not accomplish that you may expect or need:

• Though its primary purpose is to protect against brute force attacks, the software does not protect servers from distributed brute force attacks.
• Applications with non-standard APIs are not supported by Fail2Ban.

In order to use Fail2Ban, you'll either need to have a firewall locally installed or, at minimum, an interface to a packet management and control system on the server.

How does Fail2Ban work?

Fail2Ban takes the log-monitoring approach to preventing server intrusions. Common log file paths, such as /var/log/apache/access.log for Apache servers, are generally automatically detected and added to the list of text files to monitor, unless the user specifies otherwise.

At the most rudimentary level, Fail2Ban works as a simple IP address blocker. Administrators may manually add undesired IP addresses to a permanent connection ban list. Both IPv4 and IPv6 addresses are supported. Alternatively, administrators may define “undesired behaviors” and block IP addresses based on violations of this rule list. For example, most Fail2Ban administrators make attempting a failed login more than three times within a certain number of minutes an “undesired behavior”.

Built-in safeguards

During a DDoS attack, potentially millions of IP addresses could attempt to connect to a server. Of course, Fail2Ban works to prevent DDoS attacks by blocking blocks of IP addresses that are flooding a server; however, by default, these bans are temporary bans.

As a server administrator, it may be tempting to permanently ban all IPs who were members of the attack. However, no software can perfectly isolate only offending IP addresses, and you'd likely end up blocking some of the server's user base permanently, so this safeguard shouldn't usually be bypassed.

Defining custom actions

Once an IP address is caught breaking a set rule or trying to connect when it's prohibited, Fail2Ban will perform whatever action the administrator has chosen. These can be from an array of options offered by the program, such as adding it to a firewall's “deny” list. Additionally, more intricate steps may be taken at the administrator's discretion via any custom Python script.

What's supported by default?

Due to the open-source nature of Fail2Ban, many less common server software packages are supported via third-party modules on repositories, even if they aren't supported by default. Here are the types of servers that come with out of the box filters shipped with a standard Fail2Ban installation:

• Apache
• Lighttpd
• sshd
• vsftpd
• qmail
• Postfix
• Courier Mail Server

The mechanism of action

Fail2Ban follows a simple set of steps to quickly determine whether to take action, and if so, which action, upon each connection to a server. When there's a new entry to a server's connection log, it means a connection is being attempted. Fail2Ban immediately detects and reviews the connection's parameters.

Upon receiving a connection request, the connection attempt is run through a “filter” to determine if its behavior is defined as unwanted or the address is disallowed. If it does match a filter, there is a corresponding “action” that contains the steps to be taken if the filter is matched. When a connection request hits a filter and triggers an action that leads to a ban, that IP address is “jailed”. Exactly what occurs when an IP address is “jailed” is defined in the jail.conf file.

Getting started with Fail2Ban

Depending on the distribution of Linux you're using, the syntax of these processes will vary slightly. All example terminal commands are given for use with Ubuntu.

First, we'll need to update our repositories to ensure we get the latest editions, then we'll need to install the package using:

sudo apt-get update
sudo apt-get install fail2ban

Configuring Fail2Ban

Once the installation is completed, head over to the directory in /etc/fail2ban. First, we'll configure our “jail” settings. These are kept in this directory in the file jail.conf. Do not make changes directly to this file! Each time there's a package upgrade, this file gets modified. If you'd like your own custom “jail” settings, you'll need to copy its contents to a file called jail.local.

By creating this file, we'll have the .conf and .local editions of this file working together. The .conf file is what has all default settings. However, if a setting is also defined in the .local file, that setting takes precedence, so that's the only file that we'll be modifying. First, we need to create the .local edition by copying the contents of the .conf file using:

awk '{ printf "# "; print; }' /etc/fail2ban/jail.conf | sudo tee /etc/fail2ban/jail.local

Starting an email server?

If you'd like to use Fail2Ban with an email service, the open-sourced “Sendmail” is a great option. Fail2Ban has built-in support for this server software. To install Sendmail, use:

apt-get install sendmail

Linking to a firewall

Remember that Fail2Ban is designed to monitor server logs and it interacts with either a firewall or packet controller. Without one of these items, Fail2Ban would be useless in itself. For the purposes of our tutorial, we'll assume you're linking it to a firewall.

Fail2Ban doesn't support software APIs that aren't routinely used, so we'll stick with the widely used firewall called “Universal Firewall”. In case you need to install this, just use:

sudo apt-get install ufw

ufw allow ssh
ufw enable

The important variables in fail2ban.local

Since the file with the .local extension is the one that ultimately takes precedence over the file ending in .cfg (and we shouldn't ever edit the .cfg version), let's take a look at the more important variables we can edit. We'll look at their names and some of the most common options server administrators use.

1. loglevel - This sets the verbosity of Fail2Ban's logging mechanism. It has four integer options. Setting it to 1 means only log errors, 2 means log errors and warnings, 3 means log errors, warnings, and informational messages, and 4 means log errors, warnings, informational messages, and advanced debugging material.

   In short, the options we have for loglevel are cumulative; 1 means to log the least (only errors), while 4 means to log pretty much everything that comes through. Most administrators will make this value 4 on development and test servers. However, on actual production servers, loglevel should usually be set to 2; this limits what needs to be reviewed to only the important things.

2. logtarget - This is where actions taken by Fail2Ban are stored. By default, you'll see this is /var/log/fail2ban.log. However, you can change it to four different values.

   You may set this variable to STDOUT to use your standard output, STDERR to print out through your standard error printer, SYSLOG to utilize the system event log, or FILE. Note that if you use FILE, you will need to define the FILE variable as a full path within fail2ban.local.

3. socket - This is simply the location of where you want your socket file and is set by default; it must be a full file path.

4. pidfile - This variable sets where the Process ID log is put; it also must be a full file path.

Making a jail

Now that we've configured the basics of our general Fail2Ban settings, it's time to make our “jail” configuration. Unlike the fail2ban.cfg file, we can edit our jail.cfg file (or make one if it doesn't already exist) directly without harm. However, it's still wise to make a separate jail.local file and then transfer its contents to jail.cfg once we're positive that we want to keep our changes.

If you're using Fail2Ban on CentOS, all jails are disabled by default. You will need to uncomment out lines in the .cfg file in order for Fail2Ban to work as intended. This file is quite extensive, so we can't go through each variable and what it does. Additionally, unlike the general settings, the names of each variable are generally self-explanatory.

Note: In general, iptables is a great choice to use in tandem with Fail2Ban to ensure that “jails” successfully drop the correct connections.

Defining when to ban

Depending on your security needs, you'll need to make your own determinations on these settings. There are three variables that you should set that determine when we should kick an IP off our server. Remember, bans should never be permanent on IP addresses unless there's an extenuating circumstance that uniquely requires this.

# Number in seconds that we ban “jailed” addresses.
bantime = 600

# This variable is the amount of time during which our maximum number of login attempts must occur for a ban to happen.
findtime = 600

# The number of times someone must try to log in during the time we set in 'findtime'.
maxretry = 3

If you want to exempt specific IP addresses from ever being banned (or an IP range), you can add it to the array held in the ignoreip variable. The separator between entries is simply a space.

Want to use the command terminal to add an IP to this list? Here's how to do it:

fail2ban-client set JAIL addignoreip 192.0.0.1

More advanced components of Fail2Ban

Beyond these variables, there are a host of other configuration files that do other things. As you set it up, you'll be prompted to manually set these variables and settings.

One skill you may wish to learn if you continue to use Fail2Ban is regex, which is short for regular expressions. This is a formatted way to specify conditions and it's how filters are specified in Fail2Ban. These get very complicated, but there are some default filter settings that are easy enough to decipher even for those unfamiliar with regex.

Using the terminal Fail2Ban client

Once you've gotten everything configured on your test server and a firewall or packet interface connected to Fail2Ban, it's time to fire up the daemon and the client. Predictably, the client program's name is simply fail2ban-client.

All commands for Fail2Ban can be run in the syntax of fail2ban-client COMMAND. Here are the basic commands you'll need to know to utilize the Fail2Ban client.

• start: This simply fires up the client and is what you'll need to either have in your startup script or manually run on server start to keep it safe from attackers.

• reload: This reloads all configuration files of Fail2Ban.

• reload <JAIL>: You can also use the reload command in conjunction with the name of a defined jail.

• stop: This kills the daemon and client of Fail2Ban.

• status: This shows a detailed status of Fail2Ban and will enable jails.

• status <JAIL>: This will show the current status of the jail you enter and any IPs that have been banned by that specific jail.

  Note that if you're using CentOS, you'll need to use these two commands to fully kill off the daemon and client processes:

  systemctl stop fail2ban
  
  systemctl disable fail2ban
  

Wrapping up

While Fail2Ban is certainly a powerful tool, it requires some time and skill to get used to in order for it to be effective. Learning Python and regular expressions if you haven't already is a great idea. There's lots of learning materials online for advanced Fail2Ban topics, and it's a technology common enough that it could help obtain certain IT administration jobs.

Finally, Fail2Ban is as intricate as you want it to be. With the right knowledge, you can go from being a beginner and following this guide to being a guru in almost no time at all.