Django Model

Behaviors

Advanced Patterns to Manage Model Complexity

by Kevin Stone

CTO/Founder Subblime

GH: kevinastone | TW: @kevinastone | LI: kevinastone

How do we maintain our Django

Models as our application grows in

complexity?

10s-100s of Models

+ Views, Templates, Tests...

Compositional Model Behaviors

The Compositional Model pattern allows you to manage the complexity of your models through compartmentalization of functionality into manageable components.

The Benefits of Fat Models

Encapsulation
Single Path
Separation of Concerns (MVC)

Without the Maintenance Cost

DRY
Readability
Reusability
Single Responsibility
Testability

Compositional Model Behaviors

Decompose models into core reusable mixins

Traditional


from django.db import models
from django.contrib.auth.models import User


class BlogPost(models.Model):
    title = models.CharField(max_length=255)
    body = models.TextField()
    slug = models.SlugField()
    author = models.ForeignKey(User, related_name='posts')
    create_date = models.DateTimeField(auto_now_add=True)
    modified_date = models.DateTimeField(auto_now=True)
    publish_date = models.DateTimeField(null=True)

Decomposed into Behaviors


from django.db import models
from django.contrib.auth.models import User


class BlogPost(models.Model):
    title = models.CharField(max_length=255)
    body = models.TextField()
    slug = models.SlugField()
    author = models.ForeignKey(User, related_name='posts')
    create_date = models.DateTimeField(auto_now_add=True)
    modified_date = models.DateTimeField(auto_now=True)
    publish_date = models.DateTimeField(null=True)


from django.db import models
from .behaviors import Authorable, Permalinkable, Timestampable, Publishable


class BlogPost(Authorable, Permalinkable, Timestampable, Publishable, models.Model):
    title = models.CharField(max_length=255)
    body = models.TextField()

Reusable Behaviors


from django.contrib.auth.models import User


class Authorable(models.Model):
    author = models.ForeignKey(User)
    
    class Meta:
        abstract = True


class Permalinkable(models.Model):
    slug = models.SlugField()    
    
    class Meta:
        abstract = True

...

Reusable Behaviors (continued)


...

class Publishable(models.Model):
    publish_date = models.DateTimeField(null=True)
    
    class Meta:
        abstract = True


class Timestampable(models.Model):
    create_date = models.DateTimeField(auto_now_add=True)
    modified_date = models.DateTimeField(auto_now=True)
    
    class Meta:
        abstract = True

Models are more than just Fields

That was just common fields, but what about everything else models encapsulate?

Properties
Custom Methods
Method Overloads (save(), etc...)
Validation
Querysets

Traditional Fat Model


class BlogPost(models.Model):
    ...

    @property
    def is_published(self):
        from django.utils import timezone
        return self.publish_date < timezone.now()

    @models.permalink
    def get_absolute_url(self):
        return ('blog-post', (), {
            "slug": self.slug,
        })

    def pre_save(self, instance, add):
        from django.utils.text import slugify
        if not instance.slug:
            instance.slug = slugify(self.title)

Behaviors with Methods

Maintains separation of concerns

class Permalinkable(models.Model):
    slug = models.SlugField()
    
    class Meta:
        abstract = True
    
    def get_url_kwargs(self, **kwargs):
        kwargs.update(getattr(self, 'url_kwargs', {}))
        return kwargs
    
    @models.permalink
    def get_absolute_url(self):
        url_kwargs = self.get_url_kwargs(slug=self.slug)        
        return (self.url_name, (), url_kwargs)
    
    def pre_save(self, instance, add):
        from django.utils.text import slugify
        if not instance.slug:
            instance.slug = slugify(self.slug_source)

Behaviors with Methods (continued)

Maintains separation of concerns


class Publishable(models.Model):
    publish_date = models.DateTimeField(null=True)
    
    class Meta:
        abstract = True
    
    objects = PassThroughManager.for_queryset_class(PublishableQuerySet)()

    def publish_on(self, date=None):
        from django.utils import timezone
        if not date:
            date = timezone.now()
        self.publish_date = date
        self.save()

    @property
    def is_published(self):
        from django.utils import timezone
        return self.publish_date < timezone.now()

Behavior Based Model


from django.db import models
from .behaviors import Authorable, Permalinkable, Timestampable, Publishable


class BlogPost(Authorable, Permalinkable, Timestampable, Publishable, models.Model):
    title = models.CharField(max_length=255)
    body = models.TextField()
    
    url_name = "blog-post"

    @property
    def slug_source(self):
        return self.title

Naming Tips

Use "<verb>-able" naming pattern for behaviors

Readily identifiable as a compositional Mixin and not a standalone Model.
The word Mixin is already overly-generic...
(even though the naming gets weird quickly e.g. OptionallyGenericRelateable)

Custom Queryset Chaining

We all know to chain queryset methods, but what about adding custom manager methods?

Let's Find Posts from a Given Author (username1) that are Published (publish_date in the past)

Typical Tutorial Queries

No Encapsulation

from django.utils import timezone
from .models import BlogPost

>>> BlogPost.objects.filter(author__username='username1') \
.filter(publish_date__lte=timezone.now())

Custom Managers

Let's create methods on a custom Manager to handle the past-publication date and author filters


class BlogPostManager(models.Manager):
    
    def published(self):
        from django.utils import timezone
        return self.filter(publish_date__lte=timezone.now())
    
    def authored_by(self, author):
        return self.filter(author__username=author)


class BlogPost(models.Model):
    ...
    
    objects = BlogPostManager()


>>> published_posts = BlogPost.objects.published()
>>> posts_by_author = BlockPost.objects.authored_by('username1')

Custom Manager

But what about chaining our filters?


>>> BlogPost.objects.authored_by('username1').published()
AttributeError: 'QuerySet' object has no attribute 'published'

>>> type(Blogpost.objects.authored_by('username1'))
<class 'django.db.models.query.QuerySet'>

Solution: Custom Querysets

Combined with PassthroughManager from django-model-utils


from model_utils.managers import PassThroughManager

class PublishableQuerySet(models.query.QuerySet):
    def published(self):
        from django.utils import timezone
        return self.filter(publish_date__lte=timezone.now())


class AuthorableQuerySet(models.query.QuerySet):
    def authored_by(self, author):
        return self.filter(author__username=author)


class BlogPostQuerySet(AuthorableQuerySet, PublishableQuerySet):
    pass


class BlogPost(Authorable, Permalinkable, Timestampable, Publishable, models.Model):
    ...
    
    objects = PassThroughManager.for_queryset_class(BlogPostQuerySet)()

Chainable Custom Querysets

Now you can chain custom methods inherited from multiple behaviors


>>> author_public_posts = BlogPost.objects.authored_by('username1').published()

>>> type(Blogpost.objects.authored_by('username1'))
<class 'example.queryset.BlogPostQuerySet'>

Encapsulate the Business Logic

What's more legible and maintainable?

   BlogPost.objects.filter(author__username='username1').filter(publish_date__lte=timezone.now())

-or-


    BlogPost.objects.authored_by('username1').published()

Testing Behaviors

Create matching Behavior tests to validate our models

Same Benefits as for Models

DRY
Readability
Reusability
Single Responsibility

Existing Unit Test Example


from django.test import TestCase

from .models import BlogPost


class BlogPostTestCase(TestCase):
    def test_published_blogpost(self):
        from django.utils import timezone
        blogpost = BlogPost.objects.create(publish_date=timezone.now())
        self.assertTrue(blogpost.is_published)
        self.assertIn(blogpost, BlogPost.objects.published())

Behavior Test Mixin


class BehaviorTestCaseMixin(object):
    def get_model(self):
            return getattr(self, 'model')
    
    def create_instance(self, **kwargs):
        raise NotImplementedError("Implement me")


class PublishableTests(BehaviorTestCaseMixin):
    def test_published_blogpost(self):
        from django.utils import timezone
        obj = self.create_instance(publish_date=timezone.now())
        self.assertTrue(obj.is_published)
        self.assertIn(obj, self.model.objects.published())

Behavior Based Unit Tests


from django.test import TestCase

from .models import BlogPostfrom .behaviors.tests import PublishableTests


class BlogPostTestCase(PublishableTests, TestCase):
    model = BlogPost
    
    def create_instance(self, **kwargs):
        return BlogPost.objects.create(**kwargs)

Complete Test Case


class BlogPostTestCase(PublishableTests, AuthorableTests, PermalinkableTests, TimestampableTests, TestCase):
    model = BlogPost
    
    def create_instance(self, **kwargs):
        return BlogPost.objects.create(**kwargs)

    def test_blog_specific_functionality(self):
        ...

Additional Model Testing Tips

Use Factory Boy for creating test instances/fixtures

Use Inherited TestCases to validate different scenarios


class StaffBlogPostTestCase(PublishableTests, AuthorableTests, PermalinkableTests, TimestampableTests, BaseBlogPostTestCase):
    det setUp(self):
        self.user = StaffUser()

class AuthorizedUserBlogPostTestCase(PublishableTests, AuthorableTests, PermalinkableTests, TimestampableTests, BaseBlogPostTestCase):
    det setUp(self):
        self.user = AuthorizedUser()

(Same behavior expected for Staff or Authorized User)

Reusability

We eventually build up a Library of Behaviors

Permalinkable
Publishable
Authorable
Timestampable

Re-usable both across our own Apps and shareable through the Community

More Examples

Moderatable - BooleanField('approved')
Scheduleable - (start_date and end_date with range queries)
GenericRelatable (the triplet of content_type, object_id and GenericForeignKey)
Orderable - PositiveSmallIntegerField('position')

Reusability Example

from django.db import models
from .behaviors import Authorable, Permalinkable, Timestampable, Publishable


class BlogPost(Authorable, Permalinkable, Timestampable, Publishable, models.Model):
    title = models.CharField(max_length=255)
    body = models.TextField()
    
    url_name = "blog-post"

    @property
    def slug_source(self):
        return self.title


class BlogComment(Authorable, Permalinkable, Timestampable, models.Model):
    post = models.ForeignKey(BlogPost, related_name='comments')
    subject = models.CharField(max_length=255)
    body = models.TextField()

    url_name = 'blog-comment'
    
    def get_url_kwargs(self, **kwargs):
        return super(BlogComment, self).get_url_kwargs(post_slug=self.post.slug, **kwargs)
    
    @property
    def slug_source(self):
        return self.subject

Reusability Enforces Standards

Common Idioms, esp. in Templates and Template Tags
Permissions and Security
Testability

It's ultimately about Separation of Concerns

Keep the business logic encapsulated in the behavior
Standardize the interface to shared behaviors for consistency
Authorable always means obj.author, not obj.user or obj.owner

Recommended App Layout

querysets.py

behaviors.py (uses querysets)

models.py (composition of querysets and behaviors)

factories.py (uses models)

tests.py (uses all, split this into a module for larger apps)

I usually have a common app that has the shared behaviors, model and behavior test mixins with no dependencies on other apps.

Limitations/Pitfalls

Basically the challenges of Django Model Inheritance

Leaky Abstractions

Meta Options don't implicitly inherit (ordering, etc)
Manager vs Queryset vs Model (some duplication of logic)
ModelField options (toggling default=True vs default=False)

You often need to handle the composition yourself

(such as merging custom QuerySet classes)

(or combining Meta Options)

3rd Party Helpers

Don't Re-invent the Wheel

Django Extensions (UUIDField, AutoSlugField, etc)
Django Model Utils (already mentioned)
Filters (django-filter)

Test Helpers

Factories (factory boy)
Mocking (mock)

Questions?

The End

Example code available at

https://github.com/kevinastone/django-model-behaviors-example

About the Author:

Kevin Stone is the CTO and Founder of Subblime

Interested in working on these challenges? Subblime is hiring