Announcing django-ittybitty 0.1.0-pre2

I'd like to take this opportunity to officially announce my latest little side project: django-ittybitty! Some of you out there might not find this to be a useful application, but I hope others will enjoy it.

Many of you are familiar with the URL-shortening sites like http://tinyurl.com/, http://is.gd/, http://cli.gs/, and whole slew of others. These sites are all fine and dandy, right? Wrong! What happens when those sites have downtime and potential visitors to your site never get to your site because the URL-shortening site is down? You lose traffic. That's not good, in case you were unsure about it.

That is why I made this application. It allows you to have short URLs for any and every page on your Django site. No more need to rely on 3rd party servers to translate short URLs to real URLs on your site. So long as your pony-powered site is up and running, your visitors will be able to use URLs generated by this application to get anywhere on your site. All you need to do to make this work is download and install the application, add a middleware class to your MIDDLEWARE_CLASSES, and then use a simple template tag to generate a short URL for any given page.

django-ittybitty will keep track of the number of times a particular "itty bitty URL" has been used to access your site. I suppose some people will find that useful, but it's hardly a true metric for your "most popular" pages.

The algorithm behind this application is very simple, but it can potentially handle around 18,446,744,073,709,551,615 shortened URLs in 64 characters or fewer, neglecting the 'http://www.....' for your site (good luck getting your database to play well with that many records, much less storing them on a server :)).

For more information, please check out the project pages and enjoy:

For those who are interested, here are some code samples for how to use django-ittybitty:

{% extends 'base.html' %}
{% load ittybitty_tags %}

{% block content %}
<a href="{% ittybitty_url %}">Link to this page!</a>
{% endblock %}

or:

{% extends 'base.html' %}
{% load ittybitty_tags %}

{% block content %}
{% ittybitty_url as ittybitty %}
<a href="{{ ittybitty.get_shortcut }}">Link to this page!</a>
{% endblock %}

or:

{% extends 'base.html' %}
{% load ittybitty_tags %}

{% block content %}
{% ittybitty_url as ittybitty %}
{% with ittybitty.get_shortcut as short_url %}
<a href="{{ short_url }}">Link to this page!</a>
<a href="{{ short_url }}">Link to this page again!</a>
<a href="{{ short_url }}">Link to this page one more time!</a>
{% endwith %}
{% endblock %}

Enjoy!

Pony Power + Django Critter = Sheer Genius

I just love the Django community. So many good times. I hope that the rest of you find some form of entertainment in the next part of this article.

This morning I woke up to a humorous post by Eric Walstad on the django-users mailing group. It discussed a story about his 9-year-old daughter who has seen the light with Django. She apparently fully understands what Django is capable of and how amazing it truly is. Here is her version of what Django, embodied as a "critter," can do:

Django is a computer programming critter. He is loyal only to computer programmers and does all their work. He types with the ball on the end of his tail, at the speed of light. He beeps when his work is done and when you take him home, he flies around the house, doing all your chores. He's a helpful little fellow.

That just about sums it all up! Django rocks. We already have Pony Power to get us through the day, but when you put Pony Power and the Django Critter together, this is what you get:

Django Pony + Django Critter

Oh man!! Can you feel it? I sure can. Django is amazing, and anyone who's not using it is missing out.

(disclaimer: the characters in the image above remain the property of their respective owners)

Model Relationships and "list_display"

Yesterday I had one of my coworkers ask me what I thought to be a simple Django question: "in the admin pages im trying to show fields from different tables but it wont let me." I clarified the problem with this chap, and eventually suggested using something like this:

from django.contrib import admin
from project.app.models import AwesomeModel

class AwesomeModelAdmin(admin.ModelAdmin):
    list_display = ('fk_field__fk_attr1', 'fk_field2__fk_attr')

admin.site.register(AwesomeModel, AwesomeModelAdmin)

The Problem

As it just so happens, that does not work with Django as of SVN revision 9907 (or any previous versions I presume). You cannot span relationships in your Django models from the list_display item in your model admin classes. This completely caught me off guard, so I looked through the docs a bit. I found some work-arounds for the issue, but they all seemed pretty ridiculous and, more importantly, violations of the DRY principle. It also surprised me that I hadn't noticed this problem before! I guess that's why it's not fixed yet--it's not really required by all that many people?

Anyway, I did a bit of research into the issue. I stumbled upon a ticket which appears to be aimed at resolving this problem. The ticket is pretty old, and it looks like it's still up in the air as to whether or not the patches will be applied to trunk. That wasn't very encouraging.

A Solution

Being the nerd that I am, I set out to find an "efficient" solution of my own for the problem, without meddling with the Django codebase itself. Below you will find my attempt at some pure Python hackery (no Django involved other than overriding a method) to make our lives easier until someone with some pull in the Django community gets something better into Django's trunk.

Disclaimer: this might well be the absolute worst way to approach the problem. I'm okay with that, because I still like the results and I learned a lot while producing them. I don't have any benchmarks or anything like that, but I wouldn't complain if someone else came up with some and shared them in the comments.

from django.contrib import admin

def mygetattr(obj, hier):
    """
    Recursively attempts to find attributes across Django relationships.
    """
    if len(hier):
        return mygetattr(getattr(obj, hier[0]), hier[1:])
    return obj

def dynamic_attributes(self, attr, *args, **kwargs):
    """
    Retrieves object attributes.  If an attribute contains '__' in the name,
    and the attribute doesn't exist, this method will attempt to span Django
    model relationships to find the desired attribute.
    """
    try:
        # try to get the attribute the normal way
        return super(admin.ModelAdmin, self).__getattribute__(attr, *args, **kwargs)
    except AttributeError:
        # the attribute doesn't exist for the object.  See if the attribute has
        # two underscores in it (but doesn't begin with them).
        if attr and not attr.startswith('__') and '__' in attr:
            # it does!  make a callable for the attribute
            new_attr = lambda o: mygetattr(o, attr.split('__'))

            # add the new callable to the object's attributes
            setattr(self, attr, new_attr)

            # return the callable
            return new_attr

# override the __getattribute__ method on the admin.ModelAdmin class
admin.ModelAdmin.__getattribute__ = dynamic_attributes

This code could be placed, for example, in your project's root urls.py file. That would make it so that all of the apps in your project could benefit from the relationship spanning. Alternatively, you could place it in the admin.py module for a specific application. It would just need to be someplace that was actually processed when your site is "booted up."

Basically this code will override the built-in __getattribute__ method for the django.contrib.admin.ModelAdmin class. When an attribute such as fk_field__fk_attr1 is requested for an object, the code will check to see if an attribute already exists with that name. If so, the existing attribute will be used. If not, it chops up the attribute based on the __ (double underscores) that it can find. Next, the code does some recursive getattr() calls until it runs out of relationships to hop across so it can find the attribute you really want.

Once all of that is done, the end result is placed in a callable attribute for the respective admin.ModelAdmin subclass so it won't have to be built again in the future. The new callable attribute is what is returned by the __getattribute__ function.

Caveats

Now, there are some funky things that you must be aware of before you go an implement this code on your site. Very important things, I might add. Really, I've only found one large caveat, but I wouldn't be surprised if there are others. The biggest issue is that you have to define the list_display attribute of your ModelAdmin class after you register the model and the admin class with the admin site (see below for an example). Why? Because when the Django admin validates the model's admin class, it checks the items in the list_display. If it can't find a callable attribute called fk_field__fk_attr1 during validation, it will complain and the model won't be registered in the Django admin site. The dynamic attributes that are built by my hackery are added to the object after it is validated (more accurately, when the list page is rendered, from what I have observed).

This is by far the most disgusting side effect of my hackery (at least that I have observed so far). I don't like it, but I do like it a lot more than defining loads of callables in several ModelAdmin classes just to have something simple show up in the admin's list pages. You're free to form your own opinions.

Using the original code I offered to my coworker, this is how it would have to look in order for my hack to work properly.

from django.contrib import admin
from project.app.models import AwesomeModel

class AwesomeModelAdmin(admin.ModelAdmin):
    pass

admin.site.register(AwesomeModel, AwesomeModelAdmin)
AwesomeModelAdmin.list_display = ('fk_field__fk_attr1', 'fk_field2__fk_attr')

See, I told you it was funky. But again, I'll take it.

If any of you have thoughts for how this could be improved, please share. Constructive criticism is very welcome and encouraged. Please also consider reviewing the ticket that is to address this problem.

Pluggable Django Apps and Django's Admin

There are a lot of us out there who build these reusable, pluggable Django applications and share them with the world. Some of the applications are more useful than others. Personally, I think my applications tend to fall in the less useful category, but I like to build and release them nonetheless.

The Background

The other day, I received a suggestion from one of the users of django-pendulum for how to make it a little more user-friendly. The problem was that he had not yet configured Pendulum for his current Site. As soon as he tried to access his site after installing django-pendulum, he was greeted with a nasty Django exception page, admonishing him to configure Pendulum for the site. It was pretty dirty and effective, but not very pleasant at all.

This user suggested that, instead of displaying the nasty exception page, I redirect the user to a place where they could in fact configure Pendulum. I thought this was a grand idea, and I set out to make it happen last night.

The Dilemma

That's when I realized I had a problem. django-pendulum is intended to be a reusable Django application. Most of the time it might be safe to assume that everyone uses /admin/ to get to the Django administration utility. However, there's also a good chance that this will not be the prefix used to access the administration utility. What do you do when you hardcode your applications to redirect a user to /admin/ and they use something more secure like /milwaukee/ for their administration utility?

So the question is this: how do you determine what URL will bring a user to the Django administration page for a particular Django model based on the specific site's URLconf setup?

The Solution

It turns out that one solution (not sure if it's safe to say "the solution" necessarily) with Django 1.1+ is to have something like this:

from django.core.urlresolvers import reverse
admin_url = reverse('admin_pendulum_pendulumconfiguration_add')

The named URLconf used in the reverse function should ensure that the appropriate administration URL prefix is used for whatever site your application is installed on.

I did a bit of browsing through the Django documentation, but I've never seen it stated anywhere what the names are for administration URLs actually are. I found out about this little nugget by examining the output of the django.contrib.admin.ModelAdmin.get_urls method (see get_urls(self)). From the looks of it, pretty much any Django model registered with your Django administration will have URLconf items similar to the following:

[<RegexURLPattern admin_pendulum_pendulumconfiguration_changelist ^$>,
 <RegexURLPattern admin_pendulum_pendulumconfiguration_add ^add/$>,
 <RegexURLPattern admin_pendulum_pendulumconfiguration_history ^(.+)/history/$>,
 <RegexURLPattern admin_pendulum_pendulumconfiguration_delete ^(.+)/delete/$>,
 <RegexURLPattern admin_pendulum_pendulumconfiguration_change ^(.+)/$>]

In other words, you get 5 URLconf patterns per model: changelist, add, history, delete, and change. These patterns are named, and they all appear to be prefixed with admin_[app_label]_[model]_. In the case of django-pendulum, the application is called pendulum so [app_label] becomes pendulum. The model in question is called PendulumConfiguration so the [model] becomes pendulumconfiguration. I assume this same pattern will be followed for any registered model. Please correct me if I'm wrong.

More Details

For those of you interested in more details, django-pendulum uses a middleware class to redirect users to the screen where they can configure Pendulum for the site in question (unless it's already been configured). Here's my middleware as of revision 18:

from django.contrib.auth.views import login
from django.contrib.sites.models import Site
from django.core.urlresolvers import reverse
from django.http import HttpResponseRedirect
from django.conf import settings
from pendulum.models import PendulumConfiguration

SITE = Site.objects.get_current()
admin_url = reverse('admin_pendulum_pendulumconfiguration_add')
login_url = getattr(settings, 'LOGIN_URL', '/accounts/login/')

class PendulumMiddleware:
    """
    This middleware ensures that anyone trying to access Pendulum must be
    logged in.  If Pendulum hasn't been configured for the current site, the
    staff users will be redirected to the page to configure it.
    """
    def process_request(self, request):
        try:
            SITE.pendulumconfiguration
        except PendulumConfiguration.DoesNotExist:
            # this will force the user to configure pendulum if they're staff
            if request.user.has_perm('add_pendulumconfiguration') and \
                request.path not in (admin_url, login_url):
                # leave the user a message
                request.user.message_set.create(message='Please configure Pendulum for %s' % SITE)

                return HttpResponseRedirect(admin_url)
        else:
            entry_url = reverse('pendulum-entries')

            if request.path[:len(entry_url)] == entry_url and request.user.is_anonymous():
                if request.POST:
                    return login(request)
                else:
                    return HttpResponseRedirect('%s?next=%s' % (login_url, request.path))

This checks to see if Pendulum has been configured for the site. If not, the middleware will check to see if the current user has permission to add a configuration for Pendulum. If so, they will be redirected to the appropriate configuration screen.

If Pendulum has been configured for the site or the current user does not have permission to add a configuration, the user should be unaffected. If the user is trying to access Pendulum on the site's front-end, the middleware will ensure that they're actually logged in.

As always, suggestions are welcome!

Firebug for !Firefox

Pretty much anyone who's been doing any Web development in the last few years probably prefers to use Firefox because of the incredibly powerful extensions it offers. Among the extensions I hear most Web developers complain about not having in other browsers are Web Developer and Firebug. Several people feel that they could get by with another browser (such as Google Chrome) if it only had Firebug.

Well, my friends, the trusty folks who built Firebug actually offer their amazing product for other browsers! It goes by the name of "Firebug Lite." I'm not sure exactly how long this has been around, but the earliest date I can find suggests that it was released to the public in July of 2008.

I happened upon this utility while perusing Django Snippets the other day. A member by the username of jfw posted a middleware which injects the Firebug Lite utility into a response when you're not using Firefox and when your site is in debug mode. I've found it to be quite useful. I hope you all do too!!

Custom Django Settings and Default Values

This article is more about basic Python than anything else, but I hope it will help some newbies out there get further along faster. I should note that I learned Python by learning Django, so I don't exactly have the best foundation in Python. I think it's getting better though :)

Anyway, there are many occasions when I find myself creating applications that have custom settings which can be defined in a Django project's settings.py file. Sometimes I want to force the developer to define the setting (such as when a third-party API key is required). Other times I want to have sensible defaults which can easily be overridden by the developer if they desire. I struggled for a while to find an elegant solution for providing such default values for my custom settings.

I first tried something like this:

1
2
3
4
5
6
from django.conf import settings

try:
    MY_CUSTOM_SETTING = settings.MY_CUSTOM_SETTING
except AttributeError:
    MY_CUSTOM_SETTING = 'default value'

That seemed to work well enough, but it just felt dirty to me. After a while of doing thing that way, I started doing it like this:

1
2
3
4
5
6
from django.conf import settings

if hasattr(settings, 'MY_CUSTOM_SETTING'):
    MY_CUSTOM_SETTING = settings.MY_CUSTOM_SETTING
else:
    MY_CUSTOM_SETTING = 'default value'

Despite being essentially the same, it felt slightly better than the try..except method. I think I stuck with this method for a bit longer than the first method I mentioned. Finally, I found out that Python's built-in getattr function allows you to specify a default value if it fails to retrieve the value of the attribute you want:

1
2
3
from django.conf import settings

MY_CUSTOM_SETTING = getattr(settings, 'MY_CUSTOM_SETTING', 'default value')

So far, this is the most elegant and efficient method I've found for allowing custom settings with default values. Perhaps there is a better way, and I'd be delighted if someone would enlighten me and the rest of the readers.

New Site Design

I don't know how many of you have noticed this, but I just published a new design for Code Koala. Hooray!!

Along with the change in design came several improvements to the site, including a contact me form and some code optimizations.

Here are some screenshots to illustrate the difference:

Version 1.0

Version 1

Version 2.0

Version 2

Downtime and django-tracking 0.2.7

The Foul Side

Some of you may have noticed the ~11 hours of intermittent downtime that codekoala.com experienced from early on the 24th of January to just a little while ago. I was doing some work on my django-tracking application, which somehow seemed to break my site. CodeKoala.com uses PostgreSQL as the database backend, and as soon as I tried to apply the changes to django-tracking to my site, everything just seemed to die.

The weird thing was that the site would work if I put it on a sqlite or MySQL backend. I didn't change the database schema at all as part of my changes to django-tracking, so it made absolutely no sense. I was in touch with WebFaction's awesome support squad for a good deal of today trying to get things sorted out. We tried just about everything we could think of, short of porting the entire site to a different backend or restoring a recent backup.

Just as things were looking very grim, I tried this command: ./manage.py reset tracking. Voilà! The site started working again. I guess I just had some super funky junk in my tracking application's tables.

On the Brighter Side

As a result of all this work and toil, you all can now enjoy django-tracking 0.2.7! There were a lot of minor code optimizations that went into this release. The biggest change, however, is the fancy "active users map" that you see here.

This feature allows you to display a map of where your recently active users are likely to be based upon their IP address. A list is also available below the map with displays further information about each active visitor. The page updates itself every 5 seconds or so, which means that if a visitor hasn't been active for 10 minutes (or whatever your timeout happens to be), their marker will disappear from the map and their entry in the last will go away too! Pretty dang fancy if you ask me!

If you're interested in downloading and using django-tracking, please check out the links at the end of the article. The Google Code link explains what you need to do and how to configure things.

So folks!! Please play with it!

Django + Aggregation = w00t

One of my good friends has just notified me of some great news in Django land: aggregates are upon us! As of revision 9742, Django includes two new operations: annotate() and aggregate().

Ok, ok, so I haven't actually played with Django's new aggregation functionality yet, but I definitely will!! And I'm sure it will rock.

More information:

jQuery 1.3.0 Released

Today the jQuery team has released version 1.3.0 of their amazing JavaScript toolkit. It boasts a lot of great improvements, such as better selector performance, "live events" (YAY!!), drastically improved delegation filtering, overhauled HTML injection functions, et cetera. I haven't yet had a chance to play with it, but I think it will be quite fun once I do. I have a lot of Django-powered apps that are running on older versions of jQuery.

For anyone who's interested in learning more, check out the jQuery 1.3.0 release notes.