The other day at work, I found myself needing to see a list of Subversion commits by a specific user. I spent a few minutes looking at the svn log help, but nothing seemed to be designed to show commits by user. It took me a while to find something to do the trick, but this is it:

svn log | sed -n '/username/,/-----$/ p'

Gotta love sed!

Yesterday, one of the QA guys at work approached me with a question that turned out to be much more interesting to me than I think he had planned. He's been doing some unit testing using Selenium, exporting his test cases to Python. His question was this: how can I run the same unit tests using multiple browsers and multiple target servers?

I'm pretty sure he expected a simple 3-step answer or something like that. Instead, he got my crazy wide-eyed "ohhh... that's something I want to experiment with!" look. I started rambling on about inheritance, dynamic class creation, and nested for loops. His eyes started to look a little worried. He didn't really appreciate the nerdy lingo that much. I told him to pull up a chair and get comfortable.

Since I already had some other work I needed to pay attention to, I didn't want to spend too much time trying to figure out a good way to solve his problem. After about 20 minutes of devilish chuckles and frantic rustling through Python documentation, I came up with the following code:

from types import ClassType
from selenium import selenium
import unittest

IPS = ['192.168.0.1', '192.168.0.2']
BROWSERS = ['safari', 'chrome']

class SomeUnitTest(object):

    def test_something(self):
        sel = self.selenium
        # test code

def main(base):
    suites = []
    results = unittest.TestResult()

    for iidx, ip in enumerate(IPS):
        for bidx, browser in enumerate(BROWSERS):
            def setUp(self):
                self.verificationErrors = []
                self.selenium = selenium("localhost", 4444, "*%s" % self.browser, "http://%s/" % self.ip)
                self.selenium.start()

            def tearDown(self):
                self.selenium.stop()
                self.assertEqual([], self.verificationErrors)

            ut = ClassType('UT_%i_%i' % (iidx, bidx), (unittest.TestCase, base), {'ip': ip, 'browser': browser})
            ut.setUp = setUp
            ut.tearDown = tearDown

            suites.append(unittest.TestLoader().loadTestsFromTestCase(ut))

    unittest.TestSuite(suites)(results)
    for obj, error in results.errors:
        print 'In: ', obj
        print error

if __name__ == "__main__":
    main(SomeUnitTest)

I know, I know... it's got some dirty rotten tricks in it, and there are probably more efficient ways of doing what I've done. If the code offends you, look up at my previous disclaimer: I had other things I needed to be working on, so I didn't spend much time refining this. One thing I'm almost certain could be done better is not monkey patching the dynamic classes with the setUp and tearDown methods. Also, the output at the end of the test execution could definitely use some love. Oh well. Perhaps another day I'll get around to that.

Basically, you just set the servers you need to test and the browsers you want Selenium to run the tests in. Those are at the top of the script: IPS and BROWSERS. Then a new unittest.TestCase class is created for each combination of IP/server+browser. Finally, each of the test cases is thrown into a TestSuite, and the suite is processed. If there were any errors during the tests, they'll be printed out. We weren't really concerned with printing out other information, but you can certainly make other meaningful feedback appear.

Anyway, I thought that someone out there might very well benefit from my little experiment on my co-worker's question. Feel free to comment on your personal adventures with some variation of the code if you find it useful!

I've spent a little more time lately adding new features to django-articles. There are two major additions in the latest release (2.0.0-pre2).

• Article attachments
• Article statuses

That's right folks! You can finally attach files to your articles. This includes attachments to emails that you send, if you have the articles from email feature properly configured. To prove it, I'm going to attach a file to this article (which I'm posting via email).

Next, I've decided that it's worth allowing the user to specify different statuses for their articles. One of the neat things about this feature is that if you are a super user, you're logged in, and you save an article with a status that is designated as "non-live", you will still be able to see it on the site. This is a way for users to preview their work before making it live. Out of the box, there are only two statuses: draft and finished. You're free to add more statuses if you feel so inclined (they're in the database, not hardcoded).

The article status is still separate from the "is_active" flag when saving an article. Any article that is marked as inactive will not appear on the site regardless of the article's "status".

On a slightly less impressive note (although still important), this release includes some basic unit tests. Most of the tests currently revolve around article statuses and making sure that the appropriate articles appear on the site.

I learned something very interesting today regarding JavaScript. Back in the day, I used to put something like this in my HTML when I wanted to include some JS:

<script language="javascript">
...
</script>

Then I learned that I should be using something like this instead:

<script type="text/javascript">
...
</script>

I've been doing that for years and years now. Turns out I've been wrong all this time. Well, at least for 4 years of that time. I stumbled upon RFC4329 today for whatever reason and noticed that it said the text/javascript mimetype is obsolete. I dug into the RFC a bit and found this:

Various unregistered media types have been used in an ad-hoc fashion
to label and exchange programs written in ECMAScript and JavaScript.
These include:

   +-----------------------------------------------------+
   | text/javascript          | text/ecmascript          |
   | text/javascript1.0       | text/javascript1.1       |
   | text/javascript1.2       | text/javascript1.3       |
   | text/javascript1.4       | text/javascript1.5       |
   | text/jscript             | text/livescript          |
   | text/x-javascript        | text/x-ecmascript        |
   | application/x-javascript | application/x-ecmascript |
   | application/javascript   | application/ecmascript   |
   +-----------------------------------------------------+

Use of the "text" top-level type for this kind of content is known to
be problematic.  This document thus defines text/javascript and text/
ecmascript but marks them as "obsolete".  Use of experimental and
unregistered media types, as listed in part above, is discouraged.
The media types,

   * application/javascript
   * application/ecmascript

which are also defined in this document, are intended for common use
and should be used instead.

So yeah. It's time to go update all of my JavaScript stuff I guess. I thought the rest of you who are/were in the same boat as me might like to know about this...

Some of you who prefer to use git for your version control needs and were following the django-articles mirror on GitHub may have noticed some strange activity recently. I noticed today that the GitHub mirror was out of sync with the other mirrors, and I took a bit of time to investigate the problem.

I thought, for some reason, that I might be able to quickly and easily bring it back into sync if I just deleted the repo, recreated it, and pushed my changes to it. That didn't work. This means that all of you who were once following the project there are no longer following it, and I only realized that side effect after I had clicked the delete button. I apologize for this inconvenience.

In the end, it turned out that I had some things misconfigured with git on my box. I have resolved the problems and have brought the mirror back into sync. Please let me know if you run into any problems with it!

One of the features that I really like about sites like posterous and tumblr is that they allow you to send email to a special email address and have it be posted as a blog article. This is a feature I've been planning to implement in django-articles pretty much since its inception way back when. I finally got around to working on it.

The latest release of django-articles allows you to configure a mailbox, either IMAP4 or POP3, to periodically check for new emails. A new management command check_for_articles_from_email can be used to process the messages found in the special mailbox. If any emails are found, they will be fetched, parsed, and posted based on your configuration values. Only articles whose sender matches an active user in your Django site will be turned into articles. You can configure the command to mark such articles from email as "inactive" so they don't appear on the site without moderation. The default behavior, actually, is to mark the articles inactive--you must explicitly configure django-articles to automatically mark the articles as active if you want this behavior.

One of the biggest things that you should keep in mind with this new feature, though, is that it does not currently take your attachments into account. In time I plan on implementing this functionality. For now, only the plain text content of your email will be posted. Please see the project's README for more information about this new feature.

Please keep in mind that this is brand new functionality and it's not been very well tested in a wide variety of situations. Right now, it's in the "it works for me" stage. If you find problems with it, please create a ticket or update any similar existing tickets using the ticket tracker on bitbucket.org.

You can install or update django-articles using the following utilities:

• pip install -U django-articles
• easy_install -U django-articles
• hg clone http://bitbucket.org/codekoala/django-articles/ or just hg pull -u if you have already cloned it
• git clone git://github.com/codekoala/django-articles.git

Enjoy!

P.S. This article was posted via email

There has been quite a bit of recent activity in my 2ze.us project since I first released it nearly a year ago. My intent was not to become a competitor with bit.ly, is.gd, or anyone else in the URL-shortening arena. I created the site as a way for me to learn more about Google's AppEngine. It didn't take very long to get it up and running, and it seemed to work fairly well.

A recent project at work has renewed my aversion to Python's exec statement--particularly when you want to use it with arbitrary, untrusted code. The project requirements necessitated the use of exec, so I got to do some interesting experiments with it. I've got a few friends who, until I slapped some sense into them, were seemingly big fans of exec (in Django projects, even...). This article is for them and others in the same boat.

Take this example:

#!/usr/bin/env python

import sys

dirname = '/usr/lib/python2.6/site-packages'

print dirname, 'in path?', (dirname in sys.path)

exec """import sys

dirname = '/usr/lib/python2.6/site-packages'
print 'In exec path?', (dirname in sys.path)

sys.path.remove(dirname)

print 'In exec path?', (dirname in sys.path)"""

print dirname, 'in path?', (dirname in sys.path)

Take a second and examine what the script is doing. Done? Great... So, the script first makes sure that a very critical directory is in my PYTHONPATH: /usr/lib/python2.6/site-packages. This is the directory where all of the awesome Python packages, like PIL, lxml, and dozens of others, reside. This is where Python will look for such packages when I try to import and use them in my programs.

Next, a little Python snippet is executed using exec. Let's say this snippet comes from an untrusted source (a visitor to your website, for example). The snippet removes that very important directory from my PYTHONPATH. It might seem like it's relatively safe to do within an exec--maybe it doesn't change the PYTHONPATH that I was using before the exec?

Wrong. The output of this script on my personal system says it all:

$ python bad.py
/usr/lib/python2.6/site-packages in path? True
In exec path? True
In exec path? False
/usr/lib/python2.6/site-packages in path? False

From this example, we learn that Python code that is executed using exec runs in the same context as the code that uses exec. This is a critical concept to learn.

Some people might say, "Oh, there's an easy way around that. Give exec its own globals dictionary to work with, and all will be well." Wrong again. Here's a modified version of the above script.

#!/usr/bin/env python

import sys

dirname = '/usr/lib/python2.6/site-packages'

print dirname, 'in path?', (dirname in sys.path)

context = {'something': 'This is a special context for the exec'}

exec """import sys

print something
dirname = '/usr/lib/python2.6/site-packages'
print 'In exec path?', (dirname in sys.path)

sys.path.remove(dirname)

print 'In exec path?', (dirname in sys.path)""" in context

print dirname, 'in path?', (dirname in sys.path)

And here's the output:

$ python also_bad.py
/usr/lib/python2.6/site-packages in path? True
This is a special context for the exec
In exec path? True
In exec path? False
/usr/lib/python2.6/site-packages in path? False

How can you get around this glaring risk in the exec statement? One possible solution is to execute the snippet in its own process. Might not be the best way to handle things. Could be the absolute worst solution. But it's a solution, and it works:

#!/usr/bin/env python

import multiprocessing
import sys

def execute_snippet(snippet):
    exec snippet

dirname = '/usr/lib/python2.6/site-packages'

print dirname, 'in path?', (dirname in sys.path)

snippet = """import sys

dirname = '/usr/lib/python2.6/site-packages'
print 'In exec path?', (dirname in sys.path)

sys.path.remove(dirname)

print 'In exec path?', (dirname in sys.path)"""

proc = multiprocessing.Process(target=execute_snippet, args=(snippet,))
proc.start()
proc.join()

print dirname, 'in path?', (dirname in sys.path)

And here comes the output:

$ python better.py
/usr/lib/python2.6/site-packages in path? True
In exec path? True
In exec path? False
/usr/lib/python2.6/site-packages in path? True

So the PYTHONPATH is only affected by the sys.path.remove within the process that executes the snippet using exec. The process that spawns the subprocess is unaffected, and can continue with life, happily importing all of those wonderful packages from the site-packages directory. Yay.

With that said, exec isn't always bad. But my personal point of view is basically, "There is probably a better way." Unfortunately for me, that does not hold up in my current situation, and it might not work for your circumstances too. If no one is forcing you to use exec, you might investigate alternatives in all of that free time you've been wondering what to do with.

I recently found myself in a situation where knowing the execution context of a function became necessary. It took me several hours to learn about this functionality, despite many cleverly-crafted Google searches. So, being the generous person I am, I want to share my findings.

My particular use case required that a function behave differently depending on whether it was called in an exec call. Specifics beyond that are not important for this article. Here's an example of how I was able to get my desired behavior.

import inspect

def is_exec():
    caller = inspect.currentframe().f_back
    module = inspect.getmodule(caller)

    if module is None:
        print "I'm being run by exec!"
    else:
        print "I'm being run by %s" % module.__name__

def main():
    is_exec()

    exec "is_exec()"

if __name__ == '__main__':
    main()

The output of such a script would look like this:

$ python is_exec.py
I'm being run by __main__
I'm being run by exec!

It's also interesting to note that when you're using the Python interactive interpreter, calling the is_exec function from the code above will tell you that you are indeed using exec.

Some may argue that modifying behavior as I needed to is dirty, and that if your system requires such code, you're doing it wrong. Well, you could apply this sort of code to situations that have nothing to do with exec. Perhaps you want to determine which part of your product is using a specific function the most. Perhaps you want to get additional debugging information that isn't immediately obvious.

Just like always, I want to add the disclaimer that there may be other ways to do this and there probably are. However, this is the way that worked for me. I'd still be interested to here about other solutions you may have encountered for this problem.

On a side note, if you're up for some slightly advanced Python mumbo jumbo, I suggest diving into the inspect documentation.