Recently in Database Category

Create a Simple App with Django and MongoDB: Part 1

In my previous post, I talked about using MongoDB. Once I felt I started to grasp the idea of it, I wanted to try to use it in a more useful application. This is my first attempt on creating an application on Django using MongoDB. If there is something wrong in what I do, please let me know. I'll be more than happy to receive suggestions.

This is a first part of a (supposedly) multiple series, but I am no expert by any means. I'm writing these down as a way to explore more about MongoDB, and if I can finish this multiple series, I will then start evaluating whether MongoDB is suitable for any of my projects. Today, we are only going scratch the surface by focusing on creating documents and inserting some data to those documents. Hopefully, the other parts of this series will be updated in a more predictable timeframe.

Application Background

As an experiment, I was thinking to create a simple application where I can create tickets with multiple statuses (kind of like an issue tracker, but a simple one). I also want to know if there are any updates to its status over the time.

I learned most of this stuff from Django-Mumblr, and MongoEngine documentation. So thanks to them for making it opensource.

Requirements

In order to make Django work with MongoDB, we need to have a Python driver for MongoDB. It's called pyMongo, and you can download it with pip, or easy_install. Another library we need to have is MongoEngine, an ORM-like (they call it Object-Document-Mapper) library that interacts with MongoDB. MongoEngine itself needs pyMongo, so instead of working with whatever API pyMongo provides, we are going to use MongoEngine API instead. To install MongoEngine, you can run:

$ sudo pip install mongoengine

The above command will install mongoengine, as well as pymongo.

Creating Django Environment

We will start by having a project, and one application. We run:

$ django-admin.py startproject mongoticket
$ cd mongoticket
$ python manage.py startapp ticket

Once the project (mongoticket) and the application (ticket) is created, we need to create a connection to tell Django to access which database. In settings.py under the project directory, we need to add these two lines:

from mongoengine import connect
connect('ticket', 'dummy', 'dummy')

This tells that the project will use a database called "ticket" with a user "dummy" and a password "dummy". If you need to learn more on how to create MongoDB database with authentication, you can see my reference from my other post

Creating a Document Schema

I think I mentioned it before that MongoDB itself is a schema-free database engine. However, I think having schema is still better than not having schema at all. Some rows can have more fields than the others, but for the most part, they should have some identical fields. This is what MongoEngine provides. Besides, I think I'm still used to the concept of RDBMS, so not having a schema at all still frightens me.

Anyway, let's create a file called document.py inside ticket directory, and here's the content of the file:

from mongoengine import *

class Status(Document):
    name = StringField()


class TicketStatus(EmbeddedDocument):
    before = ReferenceField(Status)
    after = ReferenceField(Status)
    changed_on = DateTimeField()


class Ticket(Document):
    description = StringField(required=True, unique=True)
    created_on = DateTimeField()
    changes = ListField(EmbeddedDocumentField(TicketStatus))

1. Status Document

   Status will be a holder to define statuses. They will only have a few entries such as 'New', 'Accepted', 'On Progress', 'Closed', 'Rejected' and so on. If we were using a Django ORM, we could've done it with Choices, but I am not sure if MongoEngine supports that, so we use document instead.

2. Ticket Document

   Another document is called Ticket. As you can see here, the fields for Ticket is description, created_on and changes. In changes field, I want to have a list of dictionary that tells me when have the statuses been updated.

   For example, if I create a ticket called "fix this bug", I want to know when this ticket was created, when was it accepted, on progress, and finished. In the end, I want to know how long does it take to finish a bug.

   In ORM, this is similar to ManyToOne relationships. In MongoDB, we don't have to as it is a non-relational database engine.

3. TicketStatus Document

   The third document we have is TicketStatus but it inherits from EmbeddedDocument rather than Document. With EmbeddedDocument, we can embed a particular document to another document. In this case, a changes field in Ticket document will have a list of multiple TicketStatus document. This is similar to Ticket.objects.ticketstatus_set.all() in Django ORM but the difference is TicketStatus collection is never written to the database.

   before and after field is similar to ForeignKey in ORM. It references from Status Document.

Now we are ready to start inserting some data to the database

Inserting Data to the Database

Since we are only going to inserting data in this part, we are not going to create views yet. Rather, we will work on Python shell. We will probably start working on views functions on the next series.

Now, let's create entries to our Status document. Inside your Django project directory (mongoticket), run:

$ python manage.py shell
... snip ...
>>> from ticket.document import Status, TicketStatus, Ticket
>>> from datetime import datetime
>>>
>>> Status(name='New').save()
>>> Status(name='On Progress').save()
>>> Status(name='Rejected').save()
>>> Status(name='Closed').save()
>>> 
>>> new = Status.objects(name='New')[0]
>>> progress = Status.objects(name='On Progress')[0]
>>> rejected = Status.objects(name='Rejected')[0]
>>> closed = Status.objects(name='Closed')[0]

These are the basic statuses we are going to have. In reality, we probably need more, but we will just start with the basics.

Next, let's create a ticket:

>>> new_ticket = Ticket(description='Creating a Document', created_on=datetime.now())
>>> new_ticket.save()

You can now access its fields from new_ticket.description and new_ticket.created_on

Once we have our ticket, let's try to add some changes to it. For example, assuming that the first created ticket has a status of 'New', now let's move it to 'On Progress' as if a ticket has been accepted and someone is working on it.

>>> new_to_progress = TicketStatus(before=new, after=progress, 
...     changed_on=datetime.now())
>>> q = Ticket.objects(id=new_ticket.id)
>>> q.update(push__changes=new_to_progress)

If you notice earlier, changes field in Ticket document contains a list of embedded documents of TicketStatus document. So what we did was, we created a TicketStatus object, we look for the ticket we want to add the its changes to, and we push the object to the changes list.

Let's say that moments later the ticket has been taken care of, and it is now done. We want to add another TicketStatus to be embedded in Ticket document to identify this. For that, we do:

>>> close_ticket = TicketStatus(before=progress, after=closed, 
...     changed_on=datetime.now())
>>> q.update(push__changes=close_ticket)

So now, we should have a ticket called 'Create a document' where it has been created, moved to 'On Progress', and finally 'Closed'. Let's make sure if it works correctly.

>>> q = Ticket.objects(id=new_ticket.id)[0]
>>> for i in q.changes:
...     'From \'%s\' => \'%s\', on %s' % (i.before.name, i.after.name, 
...     i.changed_on.strftime("%B %d, %Y at %I:%M %p"))
... 
u"From 'New' => 'On Progress', on February 21, 2010 at 08:11 AM"
u"From 'On Progress' => 'Closed', on February 21, 2010 at 08:23 AM"
>>>

There you go, it is exactly what we wanted.

That is it for this series. We have created a document schema in document.py, created some objects for each documents and we link them as well as embedding documents to another document. Hopefully, this is helpful for you who are just starting to work with MongoDB on Django. Stay tune for the next series as we will start creating views functions and accessing it from the web.

If you have any comments or suggestions, please let me know.

Basic Commands to Get You Started with MongoDB

MongoDB is a schema-free, document-oriented and non-relational database engine. For more information about MongoDB, their website, a Wikipedia Entry or these screencasts will probably explain it better than I do.

I wanted to try MongoDB out of curiousity. While there are other options to document-oriented database engine such as CouchDB, or Tokyo Tyrant, I decided to go with MongoDB because I find their documentation is easy to follow.

After fiddling with MongoDB for a few hours, I thought I'd write down a few basic commands to get you started with MongoDB. Coming from a RDBMS background like MySQL and PostgreSQL, I found some of these commands are quite useful for me to get me going with MongoDB. Hopefully, it can help you too if you are thinking to give MongoDB a shot.

As a quick note, a few symbols are repetitively used, and these are the common conventions (if you will):

1. $ identifies that the command is executed from the command prompt
2. > identifies that the command is executed from MongoDB shell prompt

Installing MongoDB

Download from MongoDB Website. Once downloaded, extract the file and you will be presented with a directories containing MongoDB executables.

Assuming the folder created is called mongodb and it is located in /home/rezmuh/mongodb. You should have the following commands and folders:

$ ls
GNU-AGPL-3.0        bin         mongodb.log
README          include
THIRD-PARTY-NOTICES lib

Run MongoDB as a daemon

By default, MongoDB is running on port 27017, and database files are located in /data/db/ directory. So, make sure there is no application running on that port, and /data/db exists.

$ pwd
/home/rezmuh/mongodb
$ mkdir -p /data/db/
$ bin/mongod --fork --logpath ./mongodb.log --logappend

Create a new user

$ bin/mongo
> use admin
> db.addUser('dummy, 'dummy')
{ "user" : "dummy", "pwd" : "734b8c8ddfce845642c258769bb3e936" }

If you see a similar message to the above, it means that a user 'dummy' with password 'dummy' is successfully created. Now, we want to make sure that MongoDB is running with autentication.

In order to do that, we need to restart MongoDB daemon to use authentication:

$ killall mongod
$ bin/mongod --fork --logpath ./mongodb.log --logappend --auth
forked process: [...]
all output going to: ./mongodb.log

Connecting to a Database with authentication

$ bin/mongo -u [username] -p [password] [db name]

As a reminder, MongoDB seems to handle users within a database. So from the previous commands, we created a user 'dummy' for 'admin' database.

So at this point, we can only run:

$ bin/mongo -u dummy -p dummy admin

This will connect admin database as a user 'dummy'.

Create a new Database

> use new_database

In MongoDB, use new_database will switch our access to the 'new_database' database. If the database does not exist, it will automatically create it for you.

To create a new user to new_database, run:

> db.addUser('dummy', 'differentpassword')

To access your new_database with a specified user, run:

$ bin/mongo -u dummy -p differentpassword new_database
MongoDB shell version: 1.2.2
url: new_database
connecting to: new_database
type "help" for help

The above commands show that a user 'dummy' is created to access 'new_database' but it has a different password than the one who can access database 'admin'

Deleting a database

> use [db name]
> db.dropDatabase()

Working with Collections (Tables)

> use new_database
> db.profile.save({name: 'Reza Muhammad', blog: 'http://rezmuh.sixceedinc.com'})
> db.profile.find()
{ "_id" : ObjectId("4b7d2bdfe003af231920152a"), "name" : "Reza Muhammad", "blog" : "http://rezmuh.sixceedinc.com" }
> show collections
profile
system.indexes
system.users

The above commands will insert 'Reza Muhammad' to a field name, and 'http://rezmuh.sixceedinc.com' to a field blog in a profile table. If the table does not exist, it will create it automatically. The show collections commands will list all the collections exist in the database. In this case, profile collection exists because we already created one (through db.profile.save())

Since MongoDB is a schema-free database engine, you can also create a new record in profile collection that has different fields than previous mentioned.

Some Useful Commands

1. Use bin/mongo to to a MongoDB server without authentication
2. User bin/mongo -u [username] -p [password] [db name] to connect to a [db name] database with a specified username and password
3. show dbs lists all the available databases
4. use [db name] will let you access the database if it exists. Otherwise, it will create a new database
5. show collections will give you a list of collections (tables) in the database
6. To get a list of data in a specific collection, use db.[collection name].find()
7. To get a list of all available methods within a collection, run db.[collection name].help()
8. db.getName() will inform you the database name you're currently accessing
9. db.dropDatabase() will drop the current database you're accessing
10. db.addUser('[username]', '[password]') will create a new user for to access the current database
11. db.help() lists all the available methods you can use within a database

So there you go, those were the commands to get me started with MongoDB.