Recently in Python Category

Create a Simple App with Django and MongoDB: Part 1

In my previous post, I talked about using MongoDB. Once I felt I started to grasp the idea of it, I wanted to try to use it in a more useful application. This is my first attempt on creating an application on Django using MongoDB. If there is something wrong in what I do, please let me know. I'll be more than happy to receive suggestions.

This is a first part of a (supposedly) multiple series, but I am no expert by any means. I'm writing these down as a way to explore more about MongoDB, and if I can finish this multiple series, I will then start evaluating whether MongoDB is suitable for any of my projects. Today, we are only going scratch the surface by focusing on creating documents and inserting some data to those documents. Hopefully, the other parts of this series will be updated in a more predictable timeframe.

Application Background

As an experiment, I was thinking to create a simple application where I can create tickets with multiple statuses (kind of like an issue tracker, but a simple one). I also want to know if there are any updates to its status over the time.

I learned most of this stuff from Django-Mumblr, and MongoEngine documentation. So thanks to them for making it opensource.

Requirements

In order to make Django work with MongoDB, we need to have a Python driver for MongoDB. It's called pyMongo, and you can download it with pip, or easy_install. Another library we need to have is MongoEngine, an ORM-like (they call it Object-Document-Mapper) library that interacts with MongoDB. MongoEngine itself needs pyMongo, so instead of working with whatever API pyMongo provides, we are going to use MongoEngine API instead. To install MongoEngine, you can run:

$ sudo pip install mongoengine

The above command will install mongoengine, as well as pymongo.

Creating Django Environment

We will start by having a project, and one application. We run:

$ django-admin.py startproject mongoticket
$ cd mongoticket
$ python manage.py startapp ticket

Once the project (mongoticket) and the application (ticket) is created, we need to create a connection to tell Django to access which database. In settings.py under the project directory, we need to add these two lines:

from mongoengine import connect
connect('ticket', 'dummy', 'dummy')

This tells that the project will use a database called "ticket" with a user "dummy" and a password "dummy". If you need to learn more on how to create MongoDB database with authentication, you can see my reference from my other post

Creating a Document Schema

I think I mentioned it before that MongoDB itself is a schema-free database engine. However, I think having schema is still better than not having schema at all. Some rows can have more fields than the others, but for the most part, they should have some identical fields. This is what MongoEngine provides. Besides, I think I'm still used to the concept of RDBMS, so not having a schema at all still frightens me.

Anyway, let's create a file called document.py inside ticket directory, and here's the content of the file:

from mongoengine import *

class Status(Document):
    name = StringField()


class TicketStatus(EmbeddedDocument):
    before = ReferenceField(Status)
    after = ReferenceField(Status)
    changed_on = DateTimeField()


class Ticket(Document):
    description = StringField(required=True, unique=True)
    created_on = DateTimeField()
    changes = ListField(EmbeddedDocumentField(TicketStatus))

1. Status Document

   Status will be a holder to define statuses. They will only have a few entries such as 'New', 'Accepted', 'On Progress', 'Closed', 'Rejected' and so on. If we were using a Django ORM, we could've done it with Choices, but I am not sure if MongoEngine supports that, so we use document instead.

2. Ticket Document

   Another document is called Ticket. As you can see here, the fields for Ticket is description, created_on and changes. In changes field, I want to have a list of dictionary that tells me when have the statuses been updated.

   For example, if I create a ticket called "fix this bug", I want to know when this ticket was created, when was it accepted, on progress, and finished. In the end, I want to know how long does it take to finish a bug.

   In ORM, this is similar to ManyToOne relationships. In MongoDB, we don't have to as it is a non-relational database engine.

3. TicketStatus Document

   The third document we have is TicketStatus but it inherits from EmbeddedDocument rather than Document. With EmbeddedDocument, we can embed a particular document to another document. In this case, a changes field in Ticket document will have a list of multiple TicketStatus document. This is similar to Ticket.objects.ticketstatus_set.all() in Django ORM but the difference is TicketStatus collection is never written to the database.

   before and after field is similar to ForeignKey in ORM. It references from Status Document.

Now we are ready to start inserting some data to the database

Inserting Data to the Database

Since we are only going to inserting data in this part, we are not going to create views yet. Rather, we will work on Python shell. We will probably start working on views functions on the next series.

Now, let's create entries to our Status document. Inside your Django project directory (mongoticket), run:

$ python manage.py shell
... snip ...
>>> from ticket.document import Status, TicketStatus, Ticket
>>> from datetime import datetime
>>>
>>> Status(name='New').save()
>>> Status(name='On Progress').save()
>>> Status(name='Rejected').save()
>>> Status(name='Closed').save()
>>> 
>>> new = Status.objects(name='New')[0]
>>> progress = Status.objects(name='On Progress')[0]
>>> rejected = Status.objects(name='Rejected')[0]
>>> closed = Status.objects(name='Closed')[0]

These are the basic statuses we are going to have. In reality, we probably need more, but we will just start with the basics.

Next, let's create a ticket:

>>> new_ticket = Ticket(description='Creating a Document', created_on=datetime.now())
>>> new_ticket.save()

You can now access its fields from new_ticket.description and new_ticket.created_on

Once we have our ticket, let's try to add some changes to it. For example, assuming that the first created ticket has a status of 'New', now let's move it to 'On Progress' as if a ticket has been accepted and someone is working on it.

>>> new_to_progress = TicketStatus(before=new, after=progress, 
...     changed_on=datetime.now())
>>> q = Ticket.objects(id=new_ticket.id)
>>> q.update(push__changes=new_to_progress)

If you notice earlier, changes field in Ticket document contains a list of embedded documents of TicketStatus document. So what we did was, we created a TicketStatus object, we look for the ticket we want to add the its changes to, and we push the object to the changes list.

Let's say that moments later the ticket has been taken care of, and it is now done. We want to add another TicketStatus to be embedded in Ticket document to identify this. For that, we do:

>>> close_ticket = TicketStatus(before=progress, after=closed, 
...     changed_on=datetime.now())
>>> q.update(push__changes=close_ticket)

So now, we should have a ticket called 'Create a document' where it has been created, moved to 'On Progress', and finally 'Closed'. Let's make sure if it works correctly.

>>> q = Ticket.objects(id=new_ticket.id)[0]
>>> for i in q.changes:
...     'From \'%s\' => \'%s\', on %s' % (i.before.name, i.after.name, 
...     i.changed_on.strftime("%B %d, %Y at %I:%M %p"))
... 
u"From 'New' => 'On Progress', on February 21, 2010 at 08:11 AM"
u"From 'On Progress' => 'Closed', on February 21, 2010 at 08:23 AM"
>>>

There you go, it is exactly what we wanted.

That is it for this series. We have created a document schema in document.py, created some objects for each documents and we link them as well as embedding documents to another document. Hopefully, this is helpful for you who are just starting to work with MongoDB on Django. Stay tune for the next series as we will start creating views functions and accessing it from the web.

If you have any comments or suggestions, please let me know.

Setting up PIL with libjpeg on Mac OS X Leopard

Background

For the past few months I've been playing around with Django, and it has come to a situation where I need Imaging Library to fulfill my project's requirements. In Python, it is handled with PIL. So, here's what I did to setup PIL with libjpeg support on Mac OS X Leopard.

Default vanilla Mac OS X install doesn't have libjpeg by default, so I need to install them first.

Methods of Installing PIL on Leopard

There are a few methods you can use to install PIL on Mac OS X, they are:

1. Install libjpeg from source, and then also install PIL from source
2. Use MacPorts / fink to install libjpeg and PIL automatically
3. Or, you can also install libjpeg from MacPorts, and then PIL from source

I decided to use the third approach. First of all, I'm still not confident to install library from source, and also I want to use binary form where as much as possible. Also, I didn't install everything from MacPorts even though they are available. My primary reason is, if I installed PIL from MacPorts, it will also install Python, which I don't want. Apple already included Python in the default install. If you prefer to use fink over MacPorts, you can also do this

Installation

1. Download MacPorts and run the installer
2. Install libjpeg by using these command lines: sudo port install jpeg. This will install libjpeg in "/opt/local/lib"
3. Download PIL from source from pythonware

4. Extract the file:

   $ tar zxvf Imaging-1.1.6.tar.gz
   $ cd Imaging-1.1.6
   

5. Edit setup.py, and change the line JPEG_ROOT to: JPEG_ROOT = "/opt/local/lib/libjpeg.dylib"

6. Compile and build the source: sudo python setup.py install

The above step will install PIL package in "/Library/Python/2.5/site-packages/". This is the default directory where Python packages are located in Mac OS X.

Verify the Package

Now, here's what I did to check whether PIL was successfully installed on my computer.

In Python interpreter, I did the following:

% python
Python 2.5.1 (r251:54863, Nov 12 2008, 17:08:51)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more
>>> import Image
>>> image = Image.open("/Users/rezmuh/test-image.jpg")
>>> image.save("/Users/rezmuh/pil-image.jpg")
>>> ^D

If there is no error and you can see the new image copied from an old image, then congratulations, PIL is successfully installed :)

If you know any simpler ways to install PIL, please let me know