Installing Python 3.0 Alongside an Existing Python

With the recent release of Python 3.0 final, I've had a crazy itch that needed to be scratched. That itch, my friends, was to do a write-up of how to install Python 3.0 alongside my existing Python 2.5.2 installation without borking things up. The reason I thought it would be useful is that I'm running a Debian-based distribution of Linux called sidux right now, and neither Python 2.6 nor Python 3.0 are in the package repositories. I assume that Ubuntu and other Debian-based distributions might be the same way, and that there are others like me who would like to tinker with these new releases.

Just before attempting to install Python 3.0 on my computer, I started getting ready to write this article. Before I knew it, Python 3.0 was installed on my system, and I had no notes to share with you! That is how stinkin easy it is to install Python 3.0 without interfering with an existing install. I mentioned to a good friend that I wasn't sure the process was worth writing an article because it was so simple, but he encouraged me to carry on. Thanks bro.

So, without any further ado, here is what I did to install Python 3.0 from source alongside my Python 2.5 installation:

Note: If you're using a Mac or Windows, you should be able to simply install the packages for your platform to accomplish this same feat. Once the packages are in the repositories it will be just as easy for Linux.

$ wget http://python.org/ftp/python/3.0/Python-3.0.tar.bz2
  • Unpack the archive:
$ tar jxf Python-3.0.tar.bz2
  • Descend into the newly extracted Python-3.0 directory:
$ cd Python-3.0
  • Install libreadline-dev. This step is necessary for the arrow keys to work, as pointed out by jazevec below in the comments. If you are on a Debian-based system, you can execute a command such as this:
  $ sudo apt-get install libreadline-dev

Other distributions may have different package names, such as ``readline-dev``.  If neither one of those package names work for your distribution, try searching your package manager for ``readline`` and install the development files.  Alternatively, you should be able to manually install what you need by installing `readline itself <http://directory.fsf.org/project/readline/>`_.
  • Configure Python 3.0 for your computer:
$ ./configure
  • Compile Python 3.0:
  $ make


**UPDATE** (9 Dec): Depending on your setup, you may or may not see a message such as this after executing the ``make`` command (thanks again to ``jazevec`` for pointing out that this can happen)::

  Failed to find the necessary bits to build these modules:
  _dbm               _gdbm              _hashlib
  _sqlite3           _ssl               _tkinter
  bz2                zlib
  To find the necessary bits, look in setup.py in detect_modules() for the module's name.

If you need any of those capabilities, you should install the appropriate development files for the missing module(s).  For example, above, we installed the ``libreadline-dev`` package.  To resolve the missing module problem for each one listed above (except ``_dbm`` because it's apparently borked on Debian right now... possibly other distros too), install these packages:

  * ``tk-dev`` to satisfy ``_tkinter``
  * ``libsqlite3-dev`` to satisfy ``_sqlite3``
  * ``libbz2-dev`` to satisfy ``bz2``
  * ``zlib1g-dev`` to satisfy ``zlib``
  * ``libssl-dev`` to satisfy ``_ssl`` and ``_hashlib``
  * ``libgdbm-dev`` to satisfy ``_gdbm``
  • Install Python 3.0:
  $ sudo make install

or:
# make install
  • Test Python 3.0:
$ python3.0
Python 3.0 (r30:67503, Dec  5 2008, 11:05:45)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> print 'Hi'
File "<stdin>", line 1
    print 'Hi'
            ^
SyntaxError: invalid syntax
>>> print('Hi')
Hi
>>>
  • Make sure that your old version of Python is still around:
$ python
Python 2.5.2 (r252:60911, Sep 29 2008, 21:15:13)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> print 'Hi'
Hi
>>>
  • Rejoice

That's about it folks! Extremely simple and fast, compared to what it could have been.

If you find yourself in a situation where you don't have access to sudo or straight root-level access, you can install Python 3.0 locally by doing something like this:

  • Configure Python 3.0 for your computer:
$ ./configure --prefix=$HOME/local/
  • Compile Python 3.0:
$ make
  • Install Python 3.0:
  $ make install

Not that I omitted the ``sudo`` part of the command here.
  • Symlink to Python 3.0, assuming you have a bin/ directory in your home directory (i.e. /home/[yourusername]/bin), and that said bin directory is on your PATH:
$ ln -s ~/local/bin/python3.0 ~/bin
  • Test your locally installed Python 3.0:
  $ python3.0

or, if your local ``bin`` directory isn't on your ``PATH``:
$ ~/bin/python3.0
  • Do the dance.

Please comment with any problems you find with this process, or any additional advice you can offer to newbies!

django-pendulum news

I've made several fun improvements to my Pendulum Django application. Perhaps the most noteworthy for most people is the addition of a default jQuery-powered date picker for adding and updating entries. I was hesitant to make anything like this be required because some people might prefer controls other than the one I chose. However, I tried to make it easy to override this default date picker if you so desire. Hopefully someone will report on how easy/difficult it is.

Also, I've added django-pendulum to the PyPI, which is Python's little package repository. You can think of it as an apt-get repository for those of you familiar with Debian/Ubuntu Linux or derivatives. There is a utility called easy_install, which obviously makes it easy to install packages that are found in the PyPI. The command to install django-pendulum with easy_install is easy_install django-pendulum. Good stuff!

There have been various other changes to the code, and I have some more changes planned. We'll see how long it takes me to get around to making these changes...

Another Notch for Free Software

I tend to use Amarok, one of KDE's most popular audio players, to manage my music. Amarok can usually handle synchronizing music on an iPod just fine, but it turns out that it doesn't play very well with my 3-gen iPod nano. For one reason or another, any time I tried to copy music from my linux box using Amarok, the whole iPod became useless. The device recognized that space was being used, but it wasn't recognizing any music or movies or anything. Because of this, I have been using my wife's macbook for the past while to synchronize my iPod. It's been working fine, but it sure is inconvenient to interrupt her when she needs to be doing homework or something and I want to pop into iTunes for a few minutes.

Today I was listening to my iPod and doing some homework between classes when all of the sudden my music stopped playing. I thought the playlist might have ended, so I went to start the music up again only to find out that the little guy had locked up (it does this from time to time). I couldn't remember the reboot sequence off the top of my head, so I Googled it and stumbled across a short and sweet article with the goods.

This article also referenced the use of a program called Floola in place of iTunes. I decided to investigate it briefly because my homework was boring me. Come to find out that Floola is free, works on Windows, Linux, and MacOS, and it actually does support my 3-gen nano! It also has some cool features like being able to download music videos straight from YouTube and converting them to work on the iPod. I'm really enjoying the program! One less reason to keep Windows around!

Dexpot

I often find myself in need of more screen space when I'm using my computer. I have so many different things up on my screen at all times. Since Linux is my default operating system, I take advantage of the virtual desktop functionality that is built into most every popular window manager to make it seem like I have more screen space.

This is all fine and dandy, but it makes using Windows all the more frustrating to me. I've tried some utilities in the past that offered virtual desktop capabilities, but they always seemed weak and crashed a lot. For years I had given up on virtual desktops in Windows.

Today I decided to look into the current options for such functionality in Windows. In my searching, I came across Dexpot. I installed it, and it seems to be doing exactly what I want and more! It seems to be stable, and it only takes up 8,776K of my memory. That part could be better, but I'm not complaining :)

Anyway, I thought some of you might benefit from this utility as well.

Installing Django on Shared Hosting (Site5)

This article is a related to my previously posted article about installing Django, an advanced Web framework for perfectionists, on your own computer. Now we will learn how to install Django on a shared hosting account, using Site5 and fastcgi as an example. Depending on your host, you may or may not have to request additional privileges from the support team in order to execute some of these commands.

Note: Django requires at least Python 2.3. Newer versions of Python are preferred.

Note: This HOWTO assumes familiarity with the UNIX/Linux command line.

Note: If the wget command doesn't work for you (as in you don't have permission to run it), you might try curl [url] -O instead. That's a -O as in upper-case o.

Install Python

Site5 (and many other shared hosting providers that offer SSH access) already has Python installed, but you will want to have your own copy so you can install various tools without affecting other users. So go ahead and download virtual python:

mkdir ~/downloads
cd ~/downloads
wget http://peak.telecommunity.com/dist/virtual-python.py

Virtual Python will make a local copy of the installed Python in your home directory. Now you want to make sure you execute this next command with the newest version of Python available on your host. For example, Site5 offers both Python 2.3.4 and Python 2.4.3. We want to use Python 2.4.3. To verify the version of your Python, execute the following command:

python -V

If that displays Python 2.3.x or anything earlier, try using python2.4 -V or python2.5 -V instead. Whichever command renders the most recent version of Python is the one you should use in place of python in the next command. Since python -V currently displays Python 2.4.3 on my Site5 sandbox, I will execute the following command:

python ~/downloads/virtual-python.py

Again, this is just making a local copy of the Python installation that you used to run the virtual-python.py script. Your local installation is likely in ~/lib/python2.4/ (version could vary).

Make Your Local Python Be Default

To reduce confusion and hassle, let's give our new local installation of Python precedence over the system-wide Python. To do that, open up your ~/.bashrc and make sure it contains a line similar to this:

export PATH=$HOME/bin:$PATH

If you're unfamiliar with UNIX-based text editors such as vi, here is what you would type to use vi to make the appropriate changes:

  • vi ~/.bashrc to edit the file
  • go to the end of the file by using the down arrow key or the j key
  • hit o (the letter) to tell vi you want to start typing stuff on the next line
  • type export PATH=$HOME/bin:$PATH
  • hit the escape key
  • type :x to save the changes and quit. Don't forget the : at the beginning. Alternatively, you can type :wq, which works exactly the same as :x.

Once you've made the appropriate changes to ~/.bashrc, you need to make those changes take effect in your current SSH session:

source ~/.bashrc

Now we should verify that our changes actually took place. Type the following command:

which python

If they output of that command is not something like ~/bin/python or /home/[your username]/bin/python, something probably didn't work. If that's the case, you can try again, or simply remember to use ~/bin/python instead of python throughout the rest of this HOWTO.

Install Python's setuptools

Now we should install Python's setuptools to make our lives easier down the road.

cd ~/downloads
wget http://peak.telecommunity.com/dist/ez_setup.py
python ez_setup.py

This gives us access to a script called easy_install, which makes it easy to install many useful Python tools. We will use this a bit later.

Download Django

Let's now download the most recent development version of Django. SSH into your account and execute the following commands (all commands shall be executed on your host).

svn co http://code.djangoproject.com/svn/django/trunk ~/downloads/django-trunk

Now we should make a symlink (or shortcut) to Django and put it somewhere on the Python Path. A sure-fire place is your ~/lib/python2.4/site-packages/ directory (again, that location could vary from host to host):

ln -s ~/downloads/django-trunk/django ~/lib/python2.4/site-packages
ln -s ~/downloads/django-trunk/django/bin/django-admin.py ~/bin

Now verify that Django is installed and working by executing the following command:

python -c "import django; print django.get_version()"

That command should return something like 1.0-final-SVN-8964. If you got something like that, you're good to move onto the next section. If, however, you get something more along the lines of...

Traceback (most recent call last):
    File "<string>", line 1, in ?
ImportError: No module named django

...then your Django installation didn't work. If this is the case, make sure that you have a ~/downloads/django-trunk/django directory, and also verify that ~/lib/python2.4/site-packages actually exists.

Installing Dependencies

In order for your Django projects to become useful, we need to install some other packages: PIL (Python Imaging Library, required if you want to use Django's ImageField), MySQL-python (a MySQL database driver for Python), and flup (a utility for fastcgi-powered sites).

easy_install -f http://www.pythonware.com/products/pil/ Imaging
easy_install mysql-python
easy_install flup

Sometimes, using easy_install to install PIL doesn't go over too well because of your (lack of) permissions. To circumvent this situation, you can always download the actual PIL source code and install it manually.

cd ~/downloads
wget http://effbot.org/downloads/Imaging-1.1.6.tar.gz
tar zxf Imaging-1.1.6.tar.gz
cd Imaging-1.1.6
ln -s ~/downloads/Imaging-1.1.6/PIL ~/lib/python2.4/site-packages

And to verify, you can try this command:

python -c "import PIL"

If that doesn't return anything, you're good to go. If it says something about "ImportError: No module named PIL", it didn't work. In that case, you have to come up with some other way of installing PIL.

Setting Up A Django Project

Let's attempt to setup a sample Django project.

mkdir -p ~/projects/django
cd ~/projects/django
django-admin.py startproject mysite
cd mysite
mkdir media templates

If that works, then you should be good to do the rest of your Django development on your server. If not, make sure that ~/downloads/django-trunk/django/bin/django-admin.py exists and that it has a functioning symlink (shortcut) in ~/bin. If not, you'll have to make adjustments according to your setup. Your directory structure should look something like:

  • projects
    • django
      • mysite
        • media
        • templates
        • __init__.py
        • manage.py
        • settings.py
        • urls.py

Making A Django Project Live

Now we need to make your Django project accessible from the Web. On Site5, I generally use either a subdomain or a brand new domain when setting up a Django project. If you plan on having other projects accessible on the same hosting account, I recommend you do the same. Let's assume you setup a subdomain such as mysite.mydomain.com. On Site5, you would go to ~/public_html/mysite for the next few commands. This could differ from host to host, so I won't go into much more detail than that.

Once you're in the proper place, you need to setup a few things: two symlinks, a django.fcgi, and a custom .htaccess file. Let's begin with the symlinks.

ln -s ~/projects/django/mysite/media ~/public_html/mysite/static
ln -s ~/lib/python2.4/site-packages/django/contrib/admin/media ~/public_html/mysite/media

This just makes it so you can have your media files (CSS, images, javascripts, etc) in a different location than in your public_html.

Now for the django.fcgi. This file is what tells the webserver to execute your Django project.

#!/home/[your username]/bin/python
import sys, os

# Add a custom Python path.
sys.path.insert(0, "/home/[your username]/projects/django")

# Switch to the directory of your project. (Optional.)
os.chdir("/home/[your username]/projects/django/mysite")

# Set the DJANGO_SETTINGS_MODULE environment variable.
os.environ['DJANGO_SETTINGS_MODULE'] = "mysite.settings"

from django.core.servers.fastcgi import runfastcgi
runfastcgi(method="threaded", daemonize="false")

And finally, the .htaccess file:

1
2
3
4
5
6
RewriteEngine On
RewriteBase /
RewriteRule ^(media/.*)$ - [L]
RewriteRule ^(static/.*)$ - [L]
RewriteCond %{REQUEST_URI} !(django.fcgi)
RewriteRule ^(.*)$ django.fcgi/$1 [L]

The .htaccess file makes it so that requests to http://mysite.mydomain.com/ are properly directed to your Django project. So, now you should have a directory structure that something that looks like this:

  • public_html
    • mysite
      • media
      • static
      • .htaccess
      • django.fcgi

If that looks good, go ahead and make the django.fcgi executable and non-writable by others:

chmod 755 ~/public_html/mysite/django.fcgi

After that, head over to http://mysite.mydomain.com/ (obviously, replace the mydomain accordingly). If you see a page that says you've successfully setup your Django site, you're good to go!

Afterthoughts

I've noticed that I need to "restart" my Django sites on Site5 any time I change the .py files. There are a couple methods of doing this. One includes killing off all of your python processes (killall ~/bin/python) and the other simply updates the timestamp on your django.fcgi (touch ~/public_html/mysite/django.fcgi). I find the former to be more destructive and unreliable than the latter. So, my advice is to use the touch method unless it doesn't work, in which case you can try the killall method.

Good luck!

Using Django to Design Your Database Schema

Last night I had a buddy of mine ask me how I would approach a particular database design problem. I get similar questions quite often from my peers--suggests there is something important lacking from the database classes out there. Instead of answering him directly, I decided to come up with this tutorial for using Django to design your database schema.

For those of you new to Django, this article might seem a bit advanced. In time I will have more introductory-level Django tutorials, but I hope this one is easy enough.

Create a Django Project

The first step is to create a Django project. If you already have a project that you can play with, you can skip this step. To create a project, go to a place where you want to keep your code (like C:\projects or /home/me/projects) in a command prompt/terminal and run the following command:

django-admin.py startproject myproject

This will create a new directory in your current location called myproject (you can replace myproject with whatever you'd like so long as you're consistent). This new directory will contain a few files:

  • __init__.py
  • manage.py
  • settings.py
  • urls.py

If you get an error message when running the above command, you might not have Django installed properly. See Step-by-Step: Installing Django for details on installing Django.

Create An Application

Once you have a Django project setup, you should create a new application.

Note: If you're using Windows, you will probably need to omit the ./ on the ./manage.py commands. I will include them here for everyone else who's using Linux or a Mac.

cd myproject
./manage.py startapp specialapp

This will create a new directory in your myproject directory. This new directory will contain three files: __init__.py, models.py, and views.py. We are only concerned with the models.py file in this article.

Create Your Models

Models are usually a direct representation of what your database will be. Django makes creating these models extremely easy, and Python's syntax makes them quite readable. The Django framework asks for models to be defined in the models.py file that was created in the last step. Here's an example (for my buddy who prompted the creation of this article):

from django.db import models

class Component(models.Model):
    item_number = models.CharField(max_length=20)
    name = models.CharField(max_length=50)
    size = models.CharField(max_length=10)
    quantity = models.IntegerField(default=1)
    price = models.DecimalField(max_digits=8, decimal_places=2)

class Project(models.Model):
    name = models.CharField(max_length=50)
    components = models.ManyToManyField(Component)
    instructions = models.TextField()

(for more information about models, see the Django Model API Reference)

I don't know about you, but that code seems pretty straightforward to me. I'll spare you all the details about what's going on (that can be a future article).

Install Your New Application

Once you have your models setup, we need to add our specialapp to our list of INSTALLED_APPS in order for Django to register these models. To do that, open up settings.py in your myproject directory, go to the bottom of the file, until you see something like

1
2
3
4
5
6
INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
)

When you find that, add your specialapp to the list

1
2
3
4
5
6
7
INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'specialapp'
)

Setup Your Database

Now you need to let Django know what kind of database you're using. Django currently supports MySQL, SQLite3, PostgreSQL, and Oracle natively, but you can get third-party tools that allow you to use other database (like SQL Server).

Still in your settings.py, go to the top until you see DATABASE_ENGINE and DATABASE_NAME. Set that to whatever type of database you are using:

DATABASE_ENGINE = 'sqlite3'
DATABASE_NAME = 'myproject.db'

Save your settings.py and go back to your command prompt/terminal.

Get Django's Opinion For Your Schema

Make sure you're in your myproject directory and run the following command:

./manage.py sqlall specialapp

This command will examine the models that we created previously and will generate the appropriate SQL to create the tables for your particular database. For SQLite, we get something like this for output:

BEGIN;
CREATE TABLE "specialapp_component" (
      "id" integer NOT NULL PRIMARY KEY,
      "item_number" varchar(20) NOT NULL,
      "name" varchar(50) NOT NULL,
      "size" varchar(10) NOT NULL,
      "quantity" integer NOT NULL,
      "price" decimal NOT NULL
)
;
CREATE TABLE "specialapp_project" (
      "id" integer NOT NULL PRIMARY KEY,
      "name" varchar(50) NOT NULL,
      "instructions" text NOT NULL
)
;
CREATE TABLE "specialapp_project_components" (
      "id" integer NOT NULL PRIMARY KEY,
      "project_id" integer NOT NULL REFERENCES "specialapp_project" ("id"),
      "component_id" integer NOT NULL REFERENCES "specialapp_component" ("id"),
      UNIQUE ("project_id", "component_id")
)
;
COMMIT;

Notice how Django does all sorts of nifty things, like wrapping the table creation queries in a transaction, setting up indexes, unique keys, and defining relationships between tables. The output also offers a solution to the original problem my buddy had: an intermediate table that just keeps track of relationships between projects and components (the specialapp_project_components table).

Notice that the SQL above may not work with database servers other than SQLite.

Enhancing The Intermediate Table

After my buddy reviewed this article, he asked a very interesting and valid question: What if a project needs 3 of one component? In response, I offer the following models (this requires a modern version of Django--it doesn't work on Django 0.96.1 or earlier):

from django.db import models

class Component(models.Model):
    item_number = models.CharField(max_length=20)
    name = models.CharField(max_length=50)
    size = models.CharField(max_length=10)
    quantity = models.IntegerField(default=1)
    price = models.DecimalField(max_digits=8, decimal_places=2)

class Project(models.Model):
    name = models.CharField(max_length=50)
    components = models.ManyToManyField(Component, through='ProjectComponent')
    instructions = models.TextField()

class ProjectComponent(models.Model):
    project = models.ForeignKey(Project)
    component = models.ForeignKey(Component)
    quantity = models.PositiveIntegerField()

    class Meta:
        unique_together = ['project', 'component']

Running ./manage.py sqlall specialapp now generates the following SQL:

BEGIN;
CREATE TABLE "specialapp_component" (
    "id" integer NOT NULL PRIMARY KEY,
    "item_number" varchar(20) NOT NULL,
    "name" varchar(50) NOT NULL,
    "size" varchar(10) NOT NULL,
    "quantity" integer NOT NULL,
    "price" decimal NOT NULL
)
;
CREATE TABLE "specialapp_project" (
    "id" integer NOT NULL PRIMARY KEY,
    "name" varchar(50) NOT NULL,
    "instructions" text NOT NULL
)
;
CREATE TABLE "specialapp_projectcomponent" (
    "id" integer NOT NULL PRIMARY KEY,
    "project_id" integer NOT NULL REFERENCES "specialapp_project" ("id"),
    "component_id" integer NOT NULL REFERENCES "specialapp_component" ("id"),
    "quantity" integer unsigned NOT NULL,
    UNIQUE ("project_id", "component_id")
)
;
CREATE INDEX "specialapp_projectcomponent_project_id" ON "specialapp_projectcomponent" ("project_id");
CREATE INDEX "specialapp_projectcomponent_component_id" ON "specialapp_projectcomponent" ("component_id");
COMMIT;

As you can see, most of the SQL is the same. The main difference is that the specialapp_project_components table has become specialapp_projectcomponent and it now has a quantity column. This can be used to keep track of the quantity of each component that a project requires. You can add however many fields you want to this new intermediate table's model.

Using This SQL

There are several ways you can use the SQL generated by Django. If you want to make your life really easy, you can have Django create the tables for you directly. Assuming that you have specified all of the appropriate database information in your settings.py file, you can simply run the following command:

./manage.py syncdb

This will execute the queries generated earlier directly on your database, creating the tables (if they don't already exist). Please note that this command currently will not update your schema if the table exists but is missing a column or two. You must either do that manually or drop the table in question and then execute the syncdb command.

Another option, if you want to keep your DDL(Data Definition Language) in a separate script (maybe if you want to keep it in some sort of version control) is something like:

./manage.py sqlall specialapp > specialapp-ddl-080813.sql

This just puts the output of the sqlall command into a file called specialapp-ddl-080813.sql for later use.

Benefits of Using Django To Create Your Schema

  • Simple: I personally find the syntax of Django models to be very simple and direct. There is a comprehensive API that explains and demonstrates what Django models are capable of.
  • Fast: Being that the syntax is so simple, I find that it makes designing and defining your schema much faster than trying to do it with raw SQL or using a database administration GUI.
  • Understandable: Looking at the model code in Django is not nearly as intimidating as similar solutions in other frameworks (think about Java Persistence API models).
  • Intelligent: Using the same model code, Django can generate proper Data Definition Language SQL for several popular database servers. It handles indexes, keys, relationships, transactions, etc. and can tell the difference between server types.

Downfalls of Using Django To Create Your Schema

  • The Table Prefix: Notice how all of the tables in the SQL above were prefixed with specialapp_. That's Django's safe way of making sure models from different applications in the same Django project do not interfere with each other. However, if you don't plan on using Django for your end project, the prefix could be a major annoyance. There are a couple solutions:
    • A simple "search and replace" before executing the SQL in your database
    • Define the db_table option in your models
  • Another Technology: Django (or even Python) may or may not be in your organization's current development stack. If it's not, using the methods described in this article would just become one more thing to support.

Other Thoughts

I first thought about doing the things mentioned in this application when I was working on a personal Java application. I like to use JPA when developing database-backed applications in Java because it abstracts away a lot of the database operations. However, I don't like coming up with the model classes directly, so I usually reverse engineer them from existing database tables.

Before thinking about the things discussed in this article, I created the tables by hand, making several modifications to the schema before I was satisfied with my JPA models. This proved to be quite bothersome and time-consuming.

After using Django to develop my tables, the JPA models turned out to be a lot more reliable, and they were usually designed properly from the get-go. I haven't created tables manually ever since.

If you find yourself designing database schemas often, and you find that you have to make several changes to your tables before you/the project requirements are satisfied, you might consider using Django to do the grunt work. It's worked for me, and I'm sure it will work for you too.

Good luck!

Step-by-Step: Installing Django

Being the Django and Python zealot that I am, I often find myself trying to convert those around me to this awesome development paradigm. Once I break them, these people often ask me a lot of questions about Django. Over the past few months I've noticed that one of the biggest sticking points for people who are new to Django is actually getting it up and running to begin with. In response, this is the first in a series of articles dedicated to getting Django up and running.

What is Django?

The Django Web site describes Django as "a high-level Python Web framework that encourages rapid development and clean, pragmatic design." Basically, Django is just about the most amazing thing for Web development. I have tinkered with several different Web technologies, but nothing seems to even come close to what Django can do for me.

What is Python?

Python is a programming language used in numerous aspects of computing these days. It has a very simple yet powerful syntax. It's an easy language for beginners to pick up, but it provides adequate levels of power for the more experienced developers out there. If you have never programmed anything before, or you have dabbled with something like BASIC, Python should be fairly straightforward. If you are a programming veteran, but have only worked with languages like C, C++, Java, etc, you might struggle a bit with the syntax of the language. It's not difficult to overcome the differences in a couple hours of hands-on development.

Let's get started.

Installing Python...

Having Python installed is critical--Django does not work without Python. I'm guessing that you're relatively familiar with the procedures for installing software packages on your particular operating system. However, I will share a few notes to point you in the proper direction if you're lost. If nothing else, just head over to the Python download page to download anything you need to install Python. I whole-heartedly recommend using the latest stable version of Python for Django, but you should be able to get by with as early a version as 2.3.

...On Windows

Simply grab the latest version of the Python installer. It is currently version 2.5.2. Once the installer has downloaded successfully, just run through the installation wizard like any other setup program.

...On Mac OS X

Recent Mac OS X computers come with Python pre-installed. To determine whether or not you actually have it, launch the Terminal (Applications > Utilities > Terminal) and type python -c "import sys; print sys.version". If Python is already installed, you will see the version you have installed. If you have a version that is less than 2.3, you should download the newest version. If you don't have Python installed, you will get a "command not found" error. If you're in this boat, just download the latest version of the Python Universal installer and install it.

...On Linux

Most Linux distributions also have Python pre-installed. Just like with Mac OS X, you can check to see by opening up a terminal/konsole session and running the command python -c "import sys; print sys.version". If you have Python installed, you will see its version. If you get an error message when running that command, or you have a version earlier than 2.3, you need to download and install the latest version of Python.

If you're running a Debian-based distribution (like Ubuntu, sidux, MEPIS, KNOPPIX, etc), you can probably use sudo apt-get install python to get Python. If you're running an RPM-based Distribution, you can probably use something like Yum or YaST to install Python.

A sure-fire way to install Python on any Linux system, however, is to install from source. If you need to do this, you simply:

  1. download the source for the latest version of Python
  2. extract it: tar jxf Python-2.5.2.tar.bz2
  3. go into the newly-extracted directory: cd python-2.5.2
  4. configure it: ./configure
  5. compile it: make
  6. install it: make install

(I've only installed Python from source one time, so I might be wrong)

Setting Up Your PYTHONPATH...

Generally speaking, if you didn't have Python installed before starting this tutorial, you will need to setup your PYTHONPATH environment variable. This is a variable that lets Python know where to find useful things (like Django).

...On Windows

  • Open up your System Properties (Win+Break or right click on "My Computer" on your desktop and select Properties)
  • Go to the "Advanced" tab
  • Click the "Environment Variables" button
  • If you have permission to change system variables, click the "New" button in the bottom pane. Otherwise, create the PYTHONPATH variable for your user account using the "New" button in the top (User variables for [username]) pane.
  • Set the variable name to PYTHONPATH
  • Set the variable value to C:\Python25\Lib\site-packages (replace C:\Python25\ with whatever it is on your system if needed)
  • Save it

You may also need to add the python executable to your PATH. If you can successfully run python from a command prompt window, you don't need to worry about it.

If you can't run python from a command prompt, follow the procedure above, but use the PATH variable instead of PYTHONPATH. PATH most likely already exists, so you just need to append/prepend the existing value with something like C:\Python25\ (again, this might need to change depending on where you installed Python)

...On Mac OS X

Your PYTHONPATH should already be setup for you.

...On Linux

Usually you just need to edit your ~/.bash_rc script to setup your PYTHONPATH environment variable. Go ahead and open that up in your preferred text editor and make sure there's something in it like:

export PYTHONPATH=/usr/lib/python2.5/site-packages:$PYTHONPATH

Save any changes necessary and run the following command:

source ~/.bash_rc

This will take care of updating your current session with any changes you made to your ~/.bash_rc.

Installing Django

Once you have Python and have verified that you have version 2.3 or later, you are ready to install Django. Currently, the latest stable release is 0.96.1, but this is grossly out-dated. Django 1.0 will be released on September 2nd 2008, so the "unstable" copy of Django is pretty close to what 1.0 will have to offer. There are some incredibly useful improvements in the unstable version that I don't think I could do without anymore, so that's what I'll talk about installing here.

First, you need to have a subversion client. On Windows, the most popular one is called TortoiseSVN. On Mac OS X, I have played with a few, but I think Versions is a pretty decent one. Linux also has several to choose from, but if you're using Linux, you're probably going to use the command line anyway (right?).

For brevity, I will just use the subversion commands necessary to accomplish this task (instead of discussing all GUI interfaces to subversion).

The exact location that Django should be installed differs from system to system, but here are some guidelines for typical setups:

  • Windows: C:\Python25\Lib\site-packages
  • Linux: /usr/lib/python2.5/site-packages
  • Mac OS X: /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages

If you want a definite location, run the following command:

python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"

Once you know that location, go there in your command prompt or terminal session. Then execute this command svn co http://code.djangoproject.com/svn/django/trunk/django django. You will see loads of output, showing all of the files that you are downloading as you install Django.

As soon as that process completes, you should run python -c "import django" to make sure everything worked properly. If the command doesn't display an ImportError, you're good. Otherwise, you need to try again.

Getting Access to Django Scripts...

Once you can successfully import django, you might want to make sure you can run the django-admin.py script that comes with Django.

...On Windows

This process is very similar to what we did with the PYTHONPATH environment variable earlier.

  • Open your System Properties again
  • Go to the Advanced tab
  • Click the Environment Variables button
  • Find your PATH environment variable (either for your user or system-wide)
  • Make sure that the variable value contains something like C:\Python25\Lib\site-packages\django\bin
  • Save any changes
  • Open a fresh command prompt
  • Try to run django-admin.py. If you're successful, you're ready to get started with Django. Otherwise, you need to fix your path to django/bin or just call the django-admin.py script using an absolute path when needed.

...On Mac OS X

You can run a command similar to this:

sudo ln -s /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/django/bin/django-admin.py /usr/local/bin

...On Linux

If you have "root" privileges on your Linux system, you can execute a command like:

sudo ln -s /usr/lib/python2.5/site-packages/django/bin/django-admin.py /usr/local/bin

If you don't have "root" privileges, you can setup your own /usr/local/bin:

mkdir ~/bin

Make sure your ~/.bash_rc contains something like:

export PATH=$HOME/bin:$PATH

Then update your current session with any changes you made to ~/.bash_rc by running this command:

source ~/.bash_rc

And that should do it! Now you should be ready to get started with Django.

Feel free to leave a comment if you're having problems installing Django. Good luck!

Check out Installing Django on Shared Hosting.

Make Your Own iPod-Compatible Audio Books Using Linux

I like music a lot. I think I always have, and I probably always will. I like to be able to listen to good music wherever I go whenever I want. Thanks to the wonders of technology, we have a myriad of portable media devices to choose from. I personally chose an iPod nano. It's a wonderful little toy.

Anyway, as much as I like music, sometimes I feel that my time could be better used doing things more productive than just listening to music. Once I realized I felt this way, I began looking into ways to get my audio books onto my iPod. At first I simply transfered over the MP3s that came straight from the CDs. But I soon realized that this wasn't the most effective use of the iPod's audio book capabilities. So the hunt was on for some good Windows software to convert my MP3 audio books into M4B format for the iPod.

Now, I'm a pretty cheap guy when it comes to paying for software (which is probably one of the main reasons I started using Linux way back when). I found a bunch of different "free" tools that claimed to be able to convert my MP3's, but few of them actually worked well enough for me to stand using them. Eventually, I found a (very round-about) routine that allowed me to turn everything into something my iPod could understand as an audio book. I followed this routine to convert several audio books and transfer them to my iPod. I never actually finished listening to any of them completely.

Last night I started fooling around with converting my DVDs into a format my iPod could understand. When I finally got The Bourne Identity converted properly, I tried to throw it onto my iPod from my wife's Mac. It told me that I would have to erase everything (because I used my own PC to transfer my files before), and I said it was ok. I didn't have any of my original .m4b files around anymore, and so I began looking for ways of creating those audio books (in Linux this time).

It wasn't long before I stumbled upon a particularly interesting post on this exact topic. It requires the use of mp3wrap, mplayer, and faac. Pretty simple, really. Here's what you do:

# mp3wrap outputfilename *.mp3
# mplayer -vc null -vo null -ao pcm:nowaveheader:fast:file=outputfilename.pcm outputfilename_MP3WRAP.mp3
# faac -R 44100 -B 16 -C 2 -X -w -q 80 --artist "author" --album "title" --title "title" --track "1" --genre "Spoken Word" --year "year" -o outputfilename.m4b outputfilename.pcm

Nice and easy, huh? Now to decipher it all.

# mp3wrap outputfilename *.mp3

This command will stitch a bunch of MP3 files into a single MP3. This makes it easier to have a "real" audio book on your iPod.

# mplayer -vc null -vo null -ao pcm:nowaveheader:fast:file=outputfilename.pcm outputfilename_MP3WRAP.mp3

This command converts that one big MP3 file to PCM (uncompressed) format. Somewhere in the output of this command, you will see something like AO: [alsa] 44100Hz 2ch s16le (2 bytes per sample) which comes in handy for the next command:

# faac -R 44100 -B 16 -C 2 -X -w -q 80 --artist "author" --album "title" --title "title" --track "1" --genre "Spoken Word" --year "year" -o outputfilename.m4b outputfilename.pcm

Finally, this command turns the PCM file into an audio book (m4b) file. The 44100, 16, and 2 right after faac all come from that special line in the output of the mplayer command.

As much as I like the command line, I don't like having to remember all of those parameters and options. So I decided to create a utility script (written in Python, of course) to wrap all of these commands into one simple one:

# mp3s2m4b.py BookName mp3s_directory [--quality=0..100] [--artist="artist"] [--album="album"] [--title="title"] [--genre="genre"] [--year=year] [--track=number]

While this might still seem too complex for pleasure, it does reduce a lot of the typing involved with the other three commands. All of the thingies in square brackets (like [--quality=0..100]) are optional. My script runs the commands mentioned previously in order, and suppresses all of the scary output.

I've used my script 4 or 5 different times so far, and it seems to work great. You may download it here.

openSUSE 11.0: Round 2

Ok, ok... I decided to give openSUSE 11.0 another shot. Since my last blog post, I have read some reviews posted by some other people who encountered similar problems when attempting to actually use KDE4. Some of these people opted to install the KDE 3.5.9 remix after that and had more promising result. So, instead of letting my bias get the best of me, I am going to try openSUSE 11.0 one more time using KDE3. The following are the steps I took while going through this process:

  1. Booted from DVD (openSUSE 11.0 x86_64)

  2. Chose "Installation" from the boot menu

  3. After the installer is completely loaded, I selected "English (US)" for both the language and the keyboard layout, read the license agreement, checked the "I Agree to the License Terms" box, and clicked Next.

  4. I waited for a few seconds while the installer probed my hardware and updated some package lists, then I chose "New Installation" and clicked Next.

  5. The next step was to choose my timezone. They have a very simple interface for this--much less frustrating than the counterpart in the most recent release of Ubuntu (8.04 LTS). My system clock is set to Mountain time, so I left that stuff alone and clicked Next.

  6. This step is probably where I screwed up the most last time. It's where you choose which desktop environment you want. You can choose from GNOME 2.22, KDE 4.0, KDE 3.5, and XFCE Deskop. Last time I chose KDE 4.0. This time I chose KDE 3.5 and clicked Next.

  7. After choosing the desktop environment, the installer took me to the disk partition section of the installation. This should be pretty easy for most people, but I changed a few things. Namely, I put the root and home partitions together, and I deleted one of my Windows partitions because Windows is stupid and bloated. Once I verified the disk partition settings, I clicked Next.

  8. This part is where you get to enter the primary system user's information. You can specify the user's name, login, and password. You may also specify a few options including whether or not the root user will have the same password, whether that user will receive system mail, and whether or not that user will be logged in automatically. If you need to, you may change the authentication settings here too. I just entered the information and got on with it. (If you uncheck the box for the root user having the same password, you're prompted for the root password after this screen)

  9. Finally, we get to the step where you get to verify all of the installation settings. I think I'll just go with the configuration for now. When you click Install, you're prompted to verify that you really want to install. Use your head on this one.

  10. After all of that, it began to format my partitions. One neat thing that I noticed while it was installing was the fact that they rolled commonly-installed packages into what they call "images." This seems to increase installation speed considerably. In the past, I've had most RPM-based distribution installations take as long as (or longer than) Windows takes to install. Granted, the difference there is that you actually get useful stuff once Linux is installed, whereas with Windows, you're stuck with something barely usable and you still have to install drivers for every piece of hardware except your monitor.

    Anyway... the openSUSE folks seem to have addressed that problem recently (maybe just in this release). This went a LOT faster than I've ever seen (on any computer). Despite the use of those images, though, there were still nearly 500 packages that needed to be installed. It seems to be quite evident that the packages are working faster than ever before. It's refreshing (though it did still take quite a bit longer to install than most Debian-based distributions I've tried).

  11. After all of the packages are installed, the system does some configuration and then reboots itself. When it comes back up, the installer will appear, do some more hardware detection and configuration, and then go straight into your desktop.

openSUSE actually didn't detect my 1680x1050 resolution (I didn't know any modern distribution wouldn't anymore), so I just went into YaST and set the resolution to what it should be. And then it locked up, and I had to do a hard reboot. Let's hope I can stop that from happening again. I suppose so long as I can still see things other than my mouse, I should be good.

Upon rebooting into my desktop, the resolution was still crappy. When I went to change it this time, though, I noticed that dual-head mode was enabled. That's stupid. I never plug a second monitor into my laptop. I disabled that, then tried to change the resolution. After logging out and back in, it seems to have changed the resolution properly. While I realize that I do have an extremely crappy video card, Ubuntu and others have been able to offer me 3D acceleration. This option is currently unavailable with openSUSE. Perhaps a little research will solve that problem.

After a few minutes of configuration and preference setting, my system locked up yet again. And another hard reset did the trick of getting it operational again. One more and it's outta here!! I do have to say... Minus the quirks with the resolution and drivers, the distribution does not seem bad at all. It might be worth trying out on a different computer--maybe I'd have better luck.

Alright, now I'm going to check the software management tool for a real driver for my video card. Looks like I may have found some. I hope they work. I'm using a "1-click installer" that I found from a Google search. The installation went fine, but after logging out and back in (to have the drivers take effect), it locked up again.

So, round two folks. Again, it might just be user error. It might just be my computer. Or openSUSE really might just suck. I don't think I'll be trying it on my computer again for a while. I might try on a different system altogether, but not on my main laptop.

First Impressions: openSUSE 11.0

Those of you who have ever held any sort of conversation with me have probably heard or have personal experience with my bigotry concerning Linux. I absolutely love Linux, and I make all sorts of excuses for the things it doesn't to as well as Windows and Mac OS to convince people to use Linux. It's just the way I am.

I've been using Linux as my main operating system ever since about 2000, though I did dabble with it a few times before that. I started out with RedHat Linux way back when, and then moved on to Mandrake (now Mandriva) Linux. As time passed, I found out about this particular distribution called "SuSE Linux," which claimed to be able to detect hardware even better than the others I had tried. It even looked really pretty. I began to really want to use this distribution. It got to the point where I almost spent $80 on it, just so I could play around.

Eventually, I got my hands on a free copy by downloading all of the packages from their FTP server or something. I managed to get this installed, and I was even more impressed than I had anticipated. SuSE Linux was amazing. But by this time, I had already become addicted to downloading and trying out any distribution I could get my hands on. That meant that SuSE spent a few days or weeks on my computer before I replaced it with something else.

As I tried more and more distributions of Linux, I began to form opinions about them. I observed what certain distributions did well, and made hard mental notes about what each distribution didn't do so well. It wasn't long before I noticed that basically all of the RPM-based distributions I had tried suffered from two major problems: bloated installation packages and severe system slow-downs as time went on. It seemed that RPM-based distributions always slowed down just as bad as Windows machines. Other types of Linux, such as Slackware, Gentoo, and Debian, didn't seem to suffer from this nearly as bad.

With these opinions in mind, I carefully chose which distributions I elected to actually install with plans for keeping around a long time. It seemed like I would always download the RPM-based distributions, but I would do it "just in case" someone else wanted the CD or DVD. Sometimes I would download the distributions and never even bother to burn the CD image to disc. I would just stuff the image away for future reference.

However, despite my opinions of RPM-based distributions, I did end up installing SUSE Enterprise Linux Desktop/Server and openSUSE a few years ago. Part of it was for a class I had; another part was to find a distribution that would suit the needs of one of my buddies. I noticed several improvements in the distributions as the years passed, but those lingering problems with bloated packages and system slow-downs still plagued each distribution.

Last week, openSUSE 11.0 was finally released. Just like always, I downloaded the CD and DVD images with no plans of actually installing openSUSE anywhere. As the downloads were going, I read some reviews posted by other people. It sounded like this particular release of openSUSE actually addressed the issues of bloat and system slow-downs (finally!!), so that made me happy, but I still didn't quite consider installing it on any of my computers. I did use one of the live CD's at work for a day, though, and it treated me well.

This morning I got the itch to change the distribution I had installed on my main computer. I was going through the list of recent downloads that I had, and it occurred to me that the most recent version I had was openSUSE 11.0. It also occurred to me that it had been at least two years since I had seriously considered installing openSUSE or SLED/SLES on my computer. So I decided that maybe everything I had read was worth looking into on my own and possibly revisiting my biased opinion of RPM-based distributions.

I started the installation early this morning while I took notes and worked from another machine. The installation went very smoothly. Everything was logical and clean. It really was a good experience. The packages really did seem to install considerably faster than any release in the past, so I had high hopes for how the system would perform after installation. After everything was said and done, my computer rebooted into the freshly installed KDE 4.0 desktop of openSUSE 11.0. It looked nice, and it was actually functional--which I cannot honestly say about any other distributions that have a KDE 4.0 remix.

Since up to this morning I hadn't been able to use KDE 4 long enough to figure out what's changed, that's where I started. I explored the new menu, which I have to admit is quite funky, but I guess that's how the industry likes things nowadays. I played around with some of the personal settings that it offers. Things seemed logical enough, but it is quite a change from KDE 3.5, which I've been using for quite a while.

After a couple minutes of tinkering, I noticed a little bubble in the corner that said something about installing some system updates. I clicked it and ran through some sort of wizard, but I guess there were no updates to install. Or maybe I just have super-slow Internet and it was taking forever to download the changes. Whatever the case, I kept on tinkering with some settings while the updater did its thing.

Next thing I know, my screen goes black and flashes a few times. Then all I can see is a white mouse on a black background. That's it. Nothing else. I'm really not sure what the problem was. The settings I was playing with seemed fairly innocent, as I modify those sorts of settings all the time on KDE 3.5. After a few minutes of white-mouse-on-black-screen fun, I decided a reboot might solve the problem.

A couple minutes later, I was presented with my loading screen, followed by the black screen and white mouse. That's it. Nothing else.

Needless to say, despite all of the improvements that I did notice in this release of openSUSE, it left a rather bitter taste in my mouth in other areas. openSUSE is no longer on my computer--it's long been replaced with yet another distribution.

Maybe it's user error. Maybe it's my computer's hardware. Or maybe openSUSE really does suck. Whatever the case, it wouldn't surprise me if I wait another year or two to try out another RPM-based distribution.