Banning Bad Bots - A Short But Effective Script

Introduction


Coding

Formatting Your Code
Why style matters

Universal Programmers Toolkit
Care and feeding of your code collection

Effective Proactive Debugging Techniques
It's all about the tools

Good Programming Practices
What to do (or not)

Banning Bad Bots
A short but effective script


Management

The Joy of Specs
How to (almost) guarantee a successful project

Habits of Successful Freelancers
Advice for success

How to Become a Great Programmer
One easy lesson!

Bidding on a Stranger's Project
The basics

Freelancing 101 - Don't Send That Email!
Pick up the phone instead

Ensuring Your Web Site Project Succeeds
Advice for clients


Photography

How to Take Great Photos (And Fix Lousy Ones), Part 1
Composing and shooting your photos

How to Take Great Photos (And Fix Lousy Ones), Part 2
Editing and postproduction

Banning Bad Bots: A Short But Effective Script

Introduction

Lots of malicious machines are out there trying to find exploits on your web site. From the occasional armchair script kiddie looking for an unpached IIS to a full-scale botnet bent on finding and exploiting email scripts, chances are your site is being hit with far more useless—not to mention potentially dangerous—traffic than necessary.

 

Because you’re a good programmer you’re always on top of the latest patches for your OS, Web server, and apps; or if you’re on a shared server you’re smart enough to go with a company that does this for you, right? Right.

 

But surely there’s an easy and fun way to stop all those bogus requests, if for no other reason than to slam the door shut on all those Web equivalents of door-to-door snake oil salesmen. If they’re looking for exploits, who’s to say they aren’t also scraping your site for email addresses?

Overview

For those of you who want to cut to the chase, here’s the general idea: for every URL that shouldn’t ever get legitimate traffic, map it (e.g. via Apache’s mod_rewrite) to a CGI that bans that IP address by adding a “deny from x.x.x.x” line to .htaccess.

 

In other words, the moment a host attempts to retrieve a dangerous URL your script adds the host’s IP address to your site’s list of banned hosts, thereby locking them out of retrieving anything on your site.

 

I’ll give you Perl and PHP versions of this script, as well as some ideas for modifying it to make it more robust.

Perl

Here’s a Perl script that does the job:

 

#!/usr/local/bin/perl -w

 

use Fcntl qw(:flock);

use CGI qw(:standard);

use IO::File;

 

my $fp = new IO::File();

 

$filename = '/path/to/.htaccess'; # CHANGE IF NECESSARY

 

if (open($fp, ">>$filename")) {

      eval { flock($fp, LOCK_EX); };

      print $fp "deny from $ENV{'REMOTE_HOST'}\n";

 

      eval { flock($fp, LOCK_UN); };

      close($fp);

}

 

print header(-status => "HTTP/1.1 403 Forbidden");

 

exit();

 

# EOF

PHP

Here’s the PHP equivalent:

 

<?php

 

$filename = '/path/to/.htaccess'; // CHANGE IF NECESSARY

 

if ($fp = fopen($filename, 'a')) {

      if (flock($fp, LOCK_EX)) {

            $remote_addr = $_SERVER['REMOTE_ADDR'];

 

            fwrite($fp, "deny from $remote_addr\n");

      }

 

      flock($fp, LOCK_UN);

 

      fclose($fp);

}

 

header('HTTP/1.1 403 Forbidden');

 

echo "Forbidden!"; // Add any other HTML here

 

exit();

 

// EOF

 

?>

 

Customizing Your Script

I’m assuming you know enough Perl or PHP to properly configure the line in the script that contains the path to .htaccess on your server (“CHANGE IF NECESSARY”). You’ll also need to ensure .htaccess is writable by your web server.

Linking to Your Script

Next, upload your script (let’s call it denied.php) to your server. Theoretically you can place it in any directory, but I’ll assume you’ve placed it in the root.

 

Now we need to link potentially malicious URLs to your script. One easy way is to use Apache’s .htaccess:

 

RewriteEngine on

RewriteRule     bbs/skins$ denied.php [L]

RewriteRule     email.cgi$ denied.php [L]

#... add more rewrite rules here ...

 

In this case I’ve redirected requests for “bbs/skins” and “email.cgi” to denied.php. Obviously you’ll want to customize this for your site.

Testing It Out

Test it out by trying to navigate to one of your banned URLs. For example, I would surf to:

 

http://www.my-site-here.com/bbs/skins

 

Of course, once you’ve successfully tested it by banning yourself you’ll want to unban yourself. Just remove the “deny from x.x.x.x” line that was added to .htaccess.

Extras

Extra features you might want to add:

 

 

 

Potential Dangers

The real pitfalls here are that you may end up banning traffic you don’t want to ban:

 

 

Conclusion

Banning malicious hosts from your site is not only practical, but also it’s lots of fun. Now all we need to do is find out how to profit from it.

 

===END===



Return to Kim Moser's Generic Home Page.
Copyright © 2017 by Kim Moser (email)
Last modified: Thu 10 January 2008 03:13:27