As my framework of choice is Django, we’ll focus on that.

To beign, you’ll need to request your http:BL API Access Key here. It is also suggested, but not required, that you configure your QuickLink (more on this below).

Now, in your settings.py you’ll want to add your http:BL API Key ‘HTTPBLKEY’.

HTTPBLKEY = 'opqrstuvwxyz'

Next you’ll need to include your httpbl middleware in MIDDLEWARE_CLASSES

MIDDLEWARE_CLASSES = (
    ....
    'projectname.middleware.httpbl.HttpBLMiddleware',
    ....
)

You’ll want to place the middleware file, named httpbl.py in your project/middleware directory. If this does not exist, you will need to create it and place an empty __init__.py file in it.

project/middleware/httpbl.py(download)

from django.conf import settings
from django.http import HttpResponseNotFound, HttpResponsePermanentRedirect
import socket

class HttpBLMiddleware(object):
   """
   "HttpBL" Middleware by iamtgc@gmail.com
   """
   def __init__(self, age=None, threat=None, classification=None):
      if age is None:
         self.age = getattr(settings, 'HTTPBLAGE', 14)
      else:
         self.age = age
      if threat is None:
         self.threat = getattr(settings, 'HTTPBLTHREAT', 30)
      else:
         self.threat = threat
      if classification is None:
         self.classification = getattr(settings, 'HTTPBLCLASS', 7)
      else:
         self.classification = classification

   def process_request(self, request):

      if settings.HTTPBLKEY:
         ip = request.META.get('REMOTE_ADDR')
         iplist = ip.split('.')
         iplist.reverse()

         domain = 'dnsbl.httpbl.org'

         query = settings.HTTPBLKEY + "." + ".".join(iplist) + "." + domain

         try:
            result = socket.gethostbyname(query)
         except socket.gaierror:
            return None

         resultlist = result.split('.')

         if (int(resultlist[1]) <= self.age and int(resultlist[2]) >= self.threat and int(resultlist[3]) & self.classification > 0):
            if settings.HTTPBLREDIRECT:
               return HttpResponsePermanentRedirect(settings.HTTPBLREDIRECT)
            else:
               return HttpResponseNotFound('
Not Found

')

      return None

This should be all you need to be on your way and protecting your Django site from suspicious hosts, email harvesters, and comment spammers. But who am I to tell you what your settings should be? Here are the additional settings you can define in settings.py. The octets that these variables correspond with are fully documented in Http:BL API Specification – Query Responses.

# HTTBLAGE - represents the number of days since activity was seen on the Honey Pot network.  Defaults to 14
HTTPBLAGE = 14
# HTTPBLTHREAT = threat score assigned by Project Honey Pot, higher number is more of a threat.  Defaults to 30
HTTPBLTHREAT = 30
# HTTPBLCLASS = bitset category, see API doc for more details.  Defaults to 7 = Suspicious & Harvester & Comment Spammer
HTTPBLCLASS = 7

As mentioned above, it is suggested that you configure a QuickLink. If configured, you should set HTTPBLREDIRECT to your QuickLink URL to redirect any “bad” traffic away from your site and into a honeypot. Again, this would be defined in settings.py.

# HTTPBLREDIRECT = QuickLink Honey Pot URL that we direct the bad traffic to.  Default = "Not Found" response, no redirection.
HTTPBLREDIRECT = 'http://some.honeypot.url/goes/here'

The Mailhide API is well documented, information on it’s inner workings can be found here.

In order to leverage Mailhide, you will need to request your API Key.

Additionally, you’ll also need amk’s Python Cryptography Toolkit. The version of the Python Cryptography Toolkit tested is 2.0.1, but any version that supports AES should work.

Now, once you have installed the necessary packages and requested your API Key, it’s time to put them to use.

In your settings.py you’ll want to add your API Keys in their respective variables, MAILHIDE_PUB_KEY is your Public Key, and MAILHIDE_PRIV_KEY is your Private Key.

MAILHIDE_PUB_KEY = '01QvLWux5x-zHdXXYv-VumOg=='
MAILHIDE_PRIV_KEY = '41F69FFAF768AE6ED69A96C15A13252A'

Next, you’ll want to add the filter, I’ve named this mailhide.py and it should reside in your app/templatetags/ directory. If this does not exist, you will need to create it and place an empty __init__.py file in it. Additional information on Extending the template system can be found here.

app/templatetags/mailhide.py (download)

import base64
import binascii

from Crypto.Cipher import AES
from django.conf import settings
from django import template

register = template.Library()

def pad_string (str, block_size):
    numpad = block_size - (len (str) % block_size)
    return str + numpad * chr (numpad)

def enc_string(str):
    key = binascii.unhexlify(settings.MAILHIDE_PRIV_KEY)
    mode = AES.MODE_CBC
    iv = '\000' * 16
    obj = AES.new(key, mode, iv)
    return obj.encrypt(str)

@register.filter()
def mailhide(value):
    args = {}
    padded_value = pad_string(value, 16)
    encrypted_value = enc_string(padded_value)

    args['public_key'] = settings.MAILHIDE_PUB_KEY
    args['encrypted_email'] = base64.urlsafe_b64encode(encrypted_value)
    args['domain'] = value[value.index('@')+1:]

    return '''...@%(domain)s''' % args

This filter expects an email address, fred.flinstone@domain.tld, for example, and will return a link to the captcha titled …@domain.tld.

Here is an example of how you would use this in your template.
templates/app/index.html

{% load mailhide %}

{{ user_obj.email|mailhide }}

Enjoy!

Please leave any feedback in the comments.

For scenarios when you would like to execute stored procedures, or any other custom SQL, you can use the Python DB API as suggested here in the Django documentation.

So, to begin, let’s examine the following stored procedure. This example was taken from the Postgres documentation on Cursors.

CREATE FUNCTION reffunc(refcursor) RETURNS refcursor AS '
BEGIN
    OPEN $1 FOR SELECT col FROM test;
    RETURN $1;
END;
' LANGUAGE plpgsql;

This stored procedure takes the name of a refcursor as an argument, executes a SELECT, and returns a refcursor containing the results of the SELECT. Using the tool psql the stored procedure can be executed in the following way.

BEGIN;
SELECT reffunc('funccursor');
FETCH ALL IN funccursor;
COMMIT;

It is important that this stored procedure is wrapped in a transaction to be able to FETCH the contents of the returned refcursor.

Now, let’s examine how you’d accomplish this in Django

from django.db import connection
cursor = connection.cursor()
cursor.execute("BEGIN")
cursor.execute("SELECT reffunc('funccursor')")
cursor.execute("FETCH ALL IN funccursor")
results = cursor.fetchall()
cursor.execute("COMMIT")

Simple! Now, the contents of the refcursor, in this case funccursor are in results.

In this article I outline the steps which I took that lead to a successful configuration. This article is not explanation on how to install Django, lighttpd or any of the associated components. It is, however, my hope that this article will help you get Django running with lighttpd and fastcgi without the frustration.


This example assumes you, jdoe, will set up a django-projects directory to contain multiple django projects, and your first project is a blog.

$ mkdir /home/jdoe/django-projects

$ cd /home/jdoe/django-projects

$ django-admin.py startproject blog

Now we need to setup your fastcgi file, we’ll place this in /home/jdoe/www/blog/mysite.fcgi, this is assuming your document-root is /home/jdoe/www/blog.

#!/usr/pkg/bin/python2.4
import sys, os

# Add a custom Python path.
sys.path.insert(0, "/home/jdoe/django-projects")

# Switch to the directory of your project. (Optional.)
# os.chdir("/home/jdoe/django-projects/blog")

# Set the DJANGO_SETTINGS_MODULE environment variable.
os.environ['DJANGO_SETTINGS_MODULE'] = "blog.settings"

from django.core.servers.fastcgi import runfastcgi
runfastcgi(method="threaded", daemonize="false")

Okay, now that you’ve set up your fcgi file, let’s test it out

$ python2.4 ./mysite.fcgi
WSGIServer: missing FastCGI param REQUEST_METHOD required by WSGI!
WSGIServer: missing FastCGI param SERVER_NAME required by WSGI!
WSGIServer: missing FastCGI param SERVER_PORT required by WSGI!
WSGIServer: missing FastCGI param SERVER_PROTOCOL required by WSGI!
Traceback (most recent call last):
  File "/usr/pkg/lib/python2.4/site-packages/flup-0.5-py2.4.egg/flup/server/fcgi_base.py", line 558, in run
    protocolStatus, appStatus = self.server.handler(self)
  File "/usr/pkg/lib/python2.4/site-packages/flup-0.5-py2.4.egg/flup/server/fcgi_base.py", line 1112, in handler
    result = self.application(environ, start_response)
  File "/usr/pkg/lib/python2.4/site-packages/Django-0.95.1-py2.4.egg/django/core/handlers/wsgi.py", line 148, in __call__
    response = self.get_response(request.path, request)
  File "/usr/pkg/lib/python2.4/site-packages/Django-0.95.1-py2.4.egg/django/core/handlers/base.py", line 59, in get_response
    response = middleware_method(request)
  File "/usr/pkg/lib/python2.4/site-packages/Django-0.95.1-py2.4.egg/django/middleware/common.py", line 40, in process_request
    if settings.APPEND_SLASH and (old_url[1][-1] != '/') and ('.' not in old_url[1].split('/')[-1]):
IndexError: string index out of range
Content-Type: text/html

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>Unhandled Exception</title>
</head><body>
<h1>Unhandled Exception</h1>
<p>An unhandled exception was thrown by the application.</p>
</body></html>

This is the output you can expect to see, if you happen to get the following output instead

$ python2.4 ./mysite.fcgi
Traceback (most recent call last):
  File "./mysite.fcgi", line 15, in ?
    runfastcgi(method="threaded", daemonize="false")
TypeError: runfastcgi() got an unexpected keyword argument 'method'

You should replace

runfastcgi(method="threaded", daemonize="false")

in mysite.fcgi with

runfastcgi(["method=threaded", "daemonize=false"])

Now the last step is setting up lighttpd, the following sets up django in your domain blog.domain.tld, but the important part is the fastcgi.server block. The primary change that I had to make from the Django documentation is the addition of the bin-path variable.

$HTTP["host"] =~ "(^|\.)blog\.domain\.tld$" {
        server.document-root = "/home/jdoe/www/blog"
        accesslog.filename = "/var/log/lighttpd/blog.domain.tld-access.log"

        fastcgi.server = (
                ".fcgi" => (
                        "localhost" => (
                                "bin-path" => "/home/jdoe/www/blog/mysite.fcgi",
                                "socket" => "/home/jdoe/tmp/mysite.sock",
                                "check-local" => "disable",
                                "min-procs" => 2,
                                "max-procs" => 4,
                        )
                ),
        )

        alias.url = (
                "/media/" => "/home/jdoe/www/blog/media/",
        )
        url.rewrite-once = (
                "^(/media.*)$" => "$1",
                "^/favicon\.ico$" => "/media/favicon.ico",
                "^(/.*)$" => "/mysite.fcgi$1",
        )
}