Today I decided that I would deliberately put myself outside of my comfort zone. No, not by intentionally putting myself on a telephone for more than 5 minutes this month... I will need a lot more preparation before I can attempt that one. No no, today's experiment has to do with Linux. If you're new around here, I am a very big fan of Linux. It has been my primary operating system for over 8 years (but I still use Windows and Mac occasionally, when I need to test my programs and the cross-platform behavior).

I just thought I would share this little screenshot with you all:

Can anyone guess what my favorite filesystem is and why?

That's right, I'm a fan of reiserfs even if its daddy is a murderer. The filesystem is fast. Really fast. I have no definitive benchmarks, I just know what I've personally experienced over the last 8 years or so. I've dabbled with the major filesystems that Linux supports like xfs, ext2, ext3, and even a little with ext4. Out of the bunch, reiserfs has always come out on top for me.

I'm not trying to start any flame wars, and I don't really care to hear why my choice is wrong--it's my choice. A few people have asked me what format I prefer, and this is just a small demonstration of why I prefer what I do. Done.

Today I have another Django goodie to share with you. For the past few days, I've been struggling to come up with a way to only load certain Django template tag libraries when a particular Django application is installed. There may well be other, more elegant solutions for this particular problem, but it can't hurt to add my findings to the pile.

We have several templates which need to display certain information only when a particular application (Satchmo, in this case) is installed in the site. A lot of these templates are global for our 100+ Django-powered sites, such as customized admin templates and the like. It's much easier for us to maintain our code this way, as opposed to overriding templates on a per-site basis.

from django import template
from django.core.exceptions import ImproperlyConfigured
from django.db.models import get_app
from django.template.loader_tags import ConstantIncludeNode, IncludeNode

register = template.Library()

def do_include_ifapp(parser, token):
    """
    Loads a template and renders it with the current context if the specified
    application is in settings.INSTALLED_APPS.

    Example::

        {% includeifapp app_label "foo/some_include" %}
    """
    bits = token.split_contents()
    if len(bits) != 3:
        raise TemplateSyntaxError, "%r tag takes two argument: the application label and the name of the template to be included" % bits[0]

    app_name, path = bits[1:]
    app_name = app_name.strip('"\'')
    try:
        models = get_app(app_name)
    except ImproperlyConfigured:
        return template.Node()

    if path[0] in ('"', "'") and path[-1] == path[0]:
        return ConstantIncludeNode(path[1:-1])
    return IncludeNode(path)
register.tag('includeifapp', do_include_ifapp)

{% extends 'base.html' %}
{% load our_global_tags %}

{% block content %}
<h2>Global Content Header</h2>
Bla bla

{% includeifapp foo 'foo_specific_junk.html' %}
{% endblock %}

I thought I would share with all of you a little trick that I've been using for quite some time in my Django applications. Personally, I find it to be very convenient and simple.

Django allows you to specify an AUTH_PROFILE_MODULE setting if you wish to maintain information about a user beyond the basic username, password, email, etc. To access the profile for a given User instance, you must do something like:

from django.contrib.auth.models import User
user = User.objects.get(pk=1)
user.get_profile().additional_info_field

That seems all find and dandy, right? Just a simple call to get_profile() isn't that difficult. However, if there is not yet an instance of whatever you set AUTH_PROFILE_MODULE to for the user in question, you'll get an error about it when you call get_profile().

My simple-minded way around this is to do something like this:

from django.db import models
from django.contrib.auth.models import User

class UserProfile(models.Model):
    user = models.ForeignKey(User, unique=True)
    additional_info_field = models.CharField(max_length=50)

User.profile = property(lambda u: UserProfile.objects.get_or_create(user=u)[0])

The magic is in the property() and get_or_create. Using the property() feature in Python, means you can just do something like:

from django.contrib.auth.models import User
user = User.objects.get(pk=1)
user.profile.additional_info_field

(with no parentheses after profile) The get_or_create method tells Django to look for any UserProfile objects whose user attribute is the user from which you are accessing the profile property. If no matches are found, an instance of UserProfile is created for you. The lambda function returns the UserProfile instance in both cases.

This trick is very simple. It's also very effective in my experience. I'm sure there are other ways of doing the same thing, but this works for me, and it's just one line of code--no need to even specify the AUTH_PROFILE_MODULE setting! You can apply the same trick to pretty much anything if you'd like. It doesn't have to be just for user profiles. Enjoy!

I suppose I should update everyone out there about what I've been up to lately. It seems strange to me that I post article much less frequently now than I did when I was a full-time university student. You'd think I'd have a whole lot more time to blog about whatever I've been working on. I suppose I do indeed have that time, it's just that I usually like to wait until my projects are "ready" for the public before I write about them.

The biggest reason I haven't posted much of anything lately is a small Twitter client I've been working on. Its purpose is to be a simple, out-of-the-way Twitter client that works equally well on Windows, Linux, and OSX. The application is written in Python and wxPython, and it has been coming along quite well. It works great in Linux (in GNOME and KDE at least), but Windows and OSX have issues with windows stealing focus when I don't want them to. I'm still trying to figure it out--any advice would be greatly appreciated.

Chirpy currently does nothing more than check your Twitter accounts for updates periodically. It notifies you of new updates using blinking buttons (which can be configured to not blink). I think the interface is pretty nice and easy to use, but I am its developer so it's only proper that I think that way.

Anyway, that project has been sucking up a lot of my free time. It's been frustrating as I build it in Linux only to find that Windows and OSX both act stupidly when I go to test it. That frustration inspired me to tinker with a different approach to a Twitter client. I began fooling around with it last night, and I think the idea has turned out to be more useful than Chripy is after a month of development!

I'm calling this new project "Tim", which is short for "Twitter IM". This one also periodically checks your Twitter account(s) for updates (of course). However, Tim will send any Twitter updates to any Jabber-enabled instant messenger client that you are signed into. If you're like me, you have Google Talk open most of the day, so you can just have Twitter updates go straight there! You can also post updates to Twitter using your Jabber instant messenger when Tim is running by simply sending a message back!!

The really neat stuff comes in when you start to consider the commands that I've added to Tim tonight. I've made it possible for you to filter out certain hashtags, follow/unfollow users, and specify from which Twitter account to post updates (when you have multiple accounts enabled). I hate all of those #FollowFriday tweets... they drive me crazy. So all I have to do is type ./filter followfriday and no tweet that contains #FollowFriday will be sent to my Jabber client. I love it.

More commands are on the way. Also on the way is a friendly interface for configuring Tim. Getting it up and running the first time is... a little less than pleasant :) Once you have it configured it seems to work pretty well though.

If you're interested in trying it out, just head on over to the project's page (http://bitbucket.org/codekoala/twitter-im/). Windows users can download an installer from the Downloads tab. I plan on putting up a DMG a little later tonight for OSX users. Linux users can download the .tar.gz file and install the normal Python way :) Enjoy!

Update: The DMG for OSX is a little bigger than I thought it would be, so I won't be hosting it on bitbucket. Instead, you can download it from my server.

Don't forget to read the README !!!

It's been a while since I wrote a blog article, so I'm using this one-liner as an excuse. In case you're new here, I do a lot of Python development. In the world of Python, you need to have a special file in a directory before you can use Python code within that directory. Yeah, yeah... that's not exactly the clearest way to explain things, but it'll have to do.

This special file is called __init__.py. Having this file in a directory that contains Python code turns that directory into what's called a "python package." We like Python packages. They make our lives so much fun!

Anyhoo, I was working on a project last night, and I wanted to create a bunch of placeholder directories that I plan to use later on. I plan on keeping Python code in these directories, so putting the special __init__.py file in them is what I was looking to do. I didn't want to have to create the __init__.py file in each directory manually, or copy/paste the file all over the place, so I investigated a way to do it quickly from the command line.

One of my buddies brought an interesting command to my attention recently: xargs. I had seen it before in various tutorials online, but I never bothered to learn about it. This seemed like as good a time as any, so I started playing. The result of my efforts follows:

find . -type d | xargs -I {} touch {}/__init__.py

What it does is:

• recursively finds (find) all directories (-type d) within the current directory (.)
• pipes (|) each directory to xargs, which makes sure that the __init__.py file exists in each one (touch {}/__init__.py)
• the -I {} tells xargs what to use as a placeholder when considering each directory found by the find command

Turns out that xargs can be used for all sorts of good stuff. My friend brought it up as a way to get rid of those nasty .svn directories on his path to "Mercurial bliss."

find . -name ".svn" -type d | xargs -I {} rm -Rf {}

How beautiful!

I spent a little bit of time last night and this morning trying to find some examples for AES encryption using Python and PyCrypto. To my surprise, I had quite a difficult time finding an example of how to do it! I posted a message on Twitter asking for any solid examples, but people mostly just responded with things I had seen before--the libraries that do the encryption, not examples for how to use the libraries.

It wasn't long after that when I just decided to tackle the problem myself. My solution ended up being pretty simple (which is probably why there weren't any solid examples for me to find). However, out of respect for those out there who might still be looking for a solid example, here is my solution:

#!/usr/bin/env python

from Crypto.Cipher import AES
import base64
import os

# the block size for the cipher object; must be 16 per FIPS-197
BLOCK_SIZE = 16

# the character used for padding--with a block cipher such as AES, the value
# you encrypt must be a multiple of BLOCK_SIZE in length.  This character is
# used to ensure that your value is always a multiple of BLOCK_SIZE
PADDING = '{'

# one-liner to sufficiently pad the text to be encrypted
pad = lambda s: s + (BLOCK_SIZE - len(s) % BLOCK_SIZE) * PADDING

# one-liners to encrypt/encode and decrypt/decode a string
# encrypt with AES, encode with base64
EncodeAES = lambda c, s: base64.b64encode(c.encrypt(pad(s)))
DecodeAES = lambda c, e: c.decrypt(base64.b64decode(e)).rstrip(PADDING)

# generate a random secret key
secret = os.urandom(BLOCK_SIZE)

# create a cipher object using the random secret
cipher = AES.new(secret)

# encode a string
encoded = EncodeAES(cipher, 'password')
print 'Encrypted string:', encoded

# decode the encoded string
decoded = DecodeAES(cipher, encoded)
print 'Decrypted string:', decoded

Edit: thanks to John and Kaso for their suggestions, though John's didn't seem to work for me (?)

Edit 2015.12.14: thanks to Stephen for pointing out that the block size for AES is always 16, and the key size can be 16, 24, or 32. See FIPS-197 for more details.

If you plan to use this script, you'll need to have PyCrypto installed on your computer. I have had a difficult time finding this for Windows in the past, so I will mirror the installer that I found over here: http://jintoreedwine.wordpress.com/2008/07/20/python-25-and-encryption-pycrypto-under-windows/. I haven't tried it on Mac OS X yet, but it should be fairly simple to install it. Same goes for Linux.

The output of the script should always change with each execution thanks to the random secret key. Here's some sample output:

$ python aes_encryption.py
Encrypted string: aPCQ8v9WzLM/JusrJPS19K8uUA/34Xiu/ZR+arzl1oM=
Decrypted string: password

$ python aes_encryption.py
Encrypted string: F0cp4hMk8RXjcww270leHnigH++yqysIyPy8Em/qEbI=
Decrypted string: password

$ python aes_encryption.py
Encrypted string: 7gH2QCIPOxXVBjTXrMmdgU2l7Iku5Lch5jpG9OScGZw=
Decrypted string: password

$ python aes_encryption.py
Encrypted string: oJUq0/XHdmYgC3ILgFgF6Tpuo8ZhoEHN9wmnuYvV58Y=
Decrypted string: password

If the comments in the script aren't explanatory enough, please comment and ask for clarification. I will offer any that I am capable of, and I invite others to do the same.

I'd like to take this opportunity to officially announce my latest little side project: django-ittybitty! Some of you out there might not find this to be a useful application, but I hope others will enjoy it.

Many of you are familiar with the URL-shortening sites like http://tinyurl.com/, http://is.gd/, http://cli.gs/, and whole slew of others. These sites are all fine and dandy, right? Wrong! What happens when those sites have downtime and potential visitors to your site never get to your site because the URL-shortening site is down? You lose traffic. That's not good, in case you were unsure about it.

That is why I made this application. It allows you to have short URLs for any and every page on your Django site. No more need to rely on 3rd party servers to translate short URLs to real URLs on your site. So long as your pony-powered site is up and running, your visitors will be able to use URLs generated by this application to get anywhere on your site. All you need to do to make this work is download and install the application, add a middleware class to your MIDDLEWARE_CLASSES, and then use a simple template tag to generate a short URL for any given page.

django-ittybitty will keep track of the number of times a particular "itty bitty URL" has been used to access your site. I suppose some people will find that useful, but it's hardly a true metric for your "most popular" pages.

The algorithm behind this application is very simple, but it can potentially handle around 18,446,744,073,709,551,615 shortened URLs in 64 characters or fewer, neglecting the 'http://www.....' for your site (good luck getting your database to play well with that many records, much less storing them on a server :)).

For more information, please check out the project pages and enjoy:

• http://code.google.com/p/django-ittybitty/
• http://pypi.python.org/pypi/django-ittybitty

For those who are interested, here are some code samples for how to use django-ittybitty:

{% extends 'base.html' %}
{% load ittybitty_tags %}

{% block content %}
<a href="{% ittybitty_url %}">Link to this page!</a>
{% endblock %}

or:

{% extends 'base.html' %}
{% load ittybitty_tags %}

{% block content %}
{% ittybitty_url as ittybitty %}
<a href="{{ ittybitty.get_shortcut }}">Link to this page!</a>
{% endblock %}

or:

{% extends 'base.html' %}
{% load ittybitty_tags %}

{% block content %}
{% ittybitty_url as ittybitty %}
{% with ittybitty.get_shortcut as short_url %}
<a href="{{ short_url }}">Link to this page!</a>
<a href="{{ short_url }}">Link to this page again!</a>
<a href="{{ short_url }}">Link to this page one more time!</a>
{% endwith %}
{% endblock %}

Enjoy!

I just love the Django community. So many good times. I hope that the rest of you find some form of entertainment in the next part of this article.

This morning I woke up to a humorous post by Eric Walstad on the django-users mailing group. It discussed a story about his 9-year-old daughter who has seen the light with Django. She apparently fully understands what Django is capable of and how amazing it truly is. Here is her version of what Django, embodied as a "critter," can do:

Django is a computer programming critter. He is loyal only to computer programmers and does all their work. He types with the ball on the end of his tail, at the speed of light. He beeps when his work is done and when you take him home, he flies around the house, doing all your chores. He's a helpful little fellow.

That just about sums it all up! Django rocks. We already have Pony Power to get us through the day, but when you put Pony Power and the Django Critter together, this is what you get:

Oh man!! Can you feel it? I sure can. Django is amazing, and anyone who's not using it is missing out.

(disclaimer: the characters in the image above remain the property of their respective owners)

I just had a buddy of mine ask me to write a script that would send an e-mail to you whenever you get an "@reply" on Twitter. I've recently been doing some work on a Twitter application, so I feel relatively comfortable with the python-twitter project to access Twitter. It didn't take very long to come up with this script, and it appears to work fine for us (using a cronjob to run the script periodically).

I thought others on the Internets might enjoy the script as well, so here it is!

#!/usr/bin/env python
# -*- coding: utf-8 -*-

"""
A simple script to check your Twitter account for @replies and send you an email
if it finds any new ones since the last time it checked.  It was developed using
python-twitter 0.5 and Python 2.5.  It has been tested on Linux only, but it
should work fine on other platforms as well.  This script is intended to be
executed by a cron manager or scheduled task manager.

Copyright (c) 2009, Josh VanderLinden
All rights reserved.

Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:

- Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright notice, this
list of conditions and the following disclaimer in the documentation and/or
other materials provided with the distribution.
- Neither the name of the organization nor the names of its contributors may
be used to endorse or promote products derived from this software without
specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""

import twitter
import ConfigParser
import os
import sys
from datetime import datetime
import smtplib
from email.MIMEMultipart import MIMEMultipart
from email.MIMEText import MIMEText
from email.Utils import formatdate

# get the user's "home" directory
DIRNAME = os.path.expanduser('~')
CONFIG = os.path.join(DIRNAME, '.twitter_email_replies.conf')
FORMAT = '%a %b %d %H:%M:%S +0000 %Y'
REPLY_TEMPLATE = """%(author)s said: %(text)s
Posted on %(created_at)s
Go to http://twitter.com/home?status=@%(screen_name)s%%20&in_reply_to_status_id=%(id)s&in_reply_to=%(screen_name)s to post a reply
"""

# sections
AUTH = 'credentials'
EXEC = 'exec_info'
EMAIL = 'email_info'

# make the code a bit "cleaner"
O = lambda s: sys.stdout.write(s + '\n')
E = lambda s: sys.stderr.write(s + '\n')
str2dt = lambda s: datetime.strptime(s, FORMAT)

def get_dict(status):
    my_dict = status.AsDict()
    my_dict['screen_name'] = my_dict['user']['screen_name']
    my_dict['author'] = my_dict['user']['name']
    return my_dict

def main():
    O('Reading configuration from %s' % CONFIG)
    parser = ConfigParser.SafeConfigParser()
    config = parser.read(CONFIG)

    # make sure we have the proper sections
    if not parser.has_section(AUTH): parser.add_section(AUTH)
    if not parser.has_section(EMAIL): parser.add_section(EMAIL)
    if not parser.has_section(EXEC): parser.add_section(EXEC)

    try:
        # get some useful settings from the configuration file
        username = parser.get(AUTH, 'username')
        password = parser.get(AUTH, 'password')

        to_address = parser.get(EMAIL, 'to_address')
        from_address = parser.get(EMAIL, 'from_address')
        smtp_server = parser.get(EMAIL, 'smtp_server')
        smtp_user = parser.get(EMAIL, 'smtp_user')
        smtp_pass = parser.get(EMAIL, 'smtp_pass')

        if '' in [username, password, to_address, from_address, smtp_server]:
            raise Exception('Not configured')
    except Exception:
        E('Please configure your credentials and e-mail information in %s!' % CONFIG)

        # create some placeholders in the configuration file to make it easier
        sections = {
            AUTH: ('username', 'password'),
            EMAIL: ('to_address', 'from_address', 'smtp_server', 'smtp_user', 'smtp_pass')
        }

        for section in sections.keys():
            for opt in sections[section]:
                if not parser.has_option(section, opt):
                    parser.set(section, opt, '')
    else:
        # determine the last time we checked for replies
        try:
            last_check = str2dt(parser.get(EXEC, 'last_run'))
        except ConfigParser.NoOptionError:
            last_check = datetime.utcnow()
        last_check_str = last_check.strftime(FORMAT)

        info = 'Fetching updates for %s since %s' % (username,
                                                       last_check_str)
        O(info)

        # attempt to connect to Twitter
        api = twitter.Api(username=username, password=password)

        # not using the `since` parameter for more backward-compatibility
        timeline = api.GetReplies()
        new_replies = []
        for reply in timeline:
            post_time = str2dt(reply.GetCreatedAt())
            if post_time > last_check:
                new_replies.append(reply)

        count = len(new_replies)
        if count:
            # send out an email for this user
            O('Found %i new replies... sending e-mail to %s' % (count, to_address))
            reply_list = '\n\n'.join([REPLY_TEMPLATE % get_dict(r) for r in new_replies])
            is_are = 'is'
            plural = 'y'
            if count != 1:
                is_are = 'are'
                plural = 'ies'

            params = {
                'is_are': is_are,
                'count': count,
                'replies': plural,
                'username': username,
                'reply_list': reply_list,
                'last_check': last_check_str
            }

            text = """There %(is_are)s %(count)i new @repl%(replies)s for %(username)s on Twitter since %(last_check)s:

%(reply_list)s""" % params

            # compose the e-mail
            msg = MIMEMultipart()
            msg['From'] = from_address
            msg['To'] = to_address
            msg['Date'] = formatdate(localtime=True)
            msg['Subject'] = 'New @Replies for %s' % username
            msg.attach(MIMEText(text))

            # try to send the e-mail message out
            email = smtplib.SMTP(smtp_server)
            if smtp_user and smtp_pass:
                email.login(smtp_user, smtp_pass)
            email.sendmail(from_address,
                           to_address,
                           msg.as_string())
            email.close()

        # save the current time so we know where to pick up next time
        parser.set(EXEC, 'last_run', datetime.utcnow().strftime(FORMAT))

    # write the config
    O('Saving settings...')
    out = open(CONFIG, 'wb')
    parser.write(out)
    out.close()

if __name__ == '__main__':
    main()

Feel free to copy this script and modify it to your desires. Also, please comment if you have issues using it.