South migrations with MPTT

We've been using django-MPTT at work for quite a while. It's a great way to manage hierarchical data in a read-efficient way, and we use it heavily in our CMS application. I'll definitely be talking about it further in future posts.

Recently we moved our database migrations from our defunct dmigrations project to Andrew Godwin's wonderful South application. One of South's best features is the ability to 'freeze' the ORM within each migration, so that you can manipulate the db via the familiar Django syntax rather than having to deal with raw SQL.

However, we ran into a problem when trying to use this to add new instances to a model that uses MPTT. We're actually using Ben Frishman's fork of django-mptt, which he wrote while he was working for us this summer. This has a base model class that defines all the MPTT fields and methods, rather than monkey-patching them in as the original version does.

The issue was that the frozen ORM only includes the basic fields that are defined on the actual model. This led to trouble when inserting a new object, especially when it's in the middle of an existing tree. MPTT includes values which identify an item's place in its tree, and when a new object is inserted most of the elements in the tree have to be updated to reflect the new positioning. django-mptt normally deals with all the SQL changes necessary, but this wasn't happening within a migration, because the dynamically-created model wasn't inheriting the correct models and fields.

The answer turned out to be simple, although it is undocumented. The frozen ORM definitions are stored in each migration as a nested dictionary. Each model is an key in the top level dictionary, whose value is a dictionary containing the field name/definitions as keys/values. However, in the sub-dictionaries, along with the field definitions, you can also store Meta defintions, including a South-specific extension: _bases, which defines the model base to inherit from. For example:

{
    'categories.category': {
        'Meta': {'unique_together': "(['slug', 'parent'],)", '_bases': ('mptt.models.Model',)},
        'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
        'name': ('django.db.models.fields.CharField', [], {'max_length': '50'}),
        'parent': ('django.db.models.fields.related.ForeignKey', [], {'blank': 'True', 'related_name': "'children'", 'null': 'True', 'to': "orm['categories.Category']"}),
        'slug': ('django.db.models.fields.CharField', [], {'max_length': '50'}),
    }
}

This ensures that the frozen category model inherits from mptt.models.Model, and gains all the special MPTT magic.