pythondjangomodel-view-controllerdata-access-layerbusiness-logic-layer

Separation of business logic and data access in django


I am writing a project in Django and I see that 80% of the code is in the file models.py. This code is confusing and, after a certain time, I cease to understand what is really happening.

Here is what bothers me:

  1. I find it ugly that my model level (which was supposed to be responsible only for the work with data from a database) is also sending email, walking on API to other services, etc.
  2. Also, I find it unacceptable to place business logic in the view, because this way it becomes difficult to control. For example, in my application there are at least three ways to create new instances of User, but technically it should create them uniformly.
  3. I do not always notice when the methods and properties of my models become non-deterministic and when they develop side effects.

Here is a simple example. At first, the User model was like this:

class User(db.Models):

    def get_present_name(self):
        return self.name or 'Anonymous'

    def activate(self):
        self.status = 'activated'
        self.save()

Over time, it turned into this:

class User(db.Models):

    def get_present_name(self): 
        # property became non-deterministic in terms of database
        # data is taken from another service by api
        return remote_api.request_user_name(self.uid) or 'Anonymous' 

    def activate(self):
        # method now has a side effect (send message to user)
        self.status = 'activated'
        self.save()
        send_mail('Your account is activated!', '…', [self.email])

What I want is to separate entities in my code:

  1. Database level entities, i.e. database level logic: What kind of data does my application store?
  2. application level entities, i.e. business level logic: What does my application do?

What are the good practices to implement such an approach that can be applied in Django?


Solution

  • It seems like you are asking about the difference between the data model and the domain model – the latter is where you can find the business logic and entities as perceived by your end user, the former is where you actually store your data.

    Furthermore, I've interpreted the 3rd part of your question as: how to notice failure to keep these models separate.

    These are two very different concepts and it's always hard to keep them separate. However, there are some common patterns and tools that can be used for this purpose.

    About the Domain Model

    The first thing you need to recognize is that your domain model is not really about data; it is about actions and questions such as "activate this user", "deactivate this user", "which users are currently activated?", and "what is this user's name?". In classical terms: it's about queries and commands.

    Thinking in Commands

    Let's start by looking at the commands in your example: "activate this user" and "deactivate this user". The nice thing about commands is that they can easily be expressed by small given-when-then scenario's:

    given an inactive user
    when the admin activates this user
    then the user becomes active
    and a confirmation e-mail is sent to the user
    and an entry is added to the system log
    (etc. etc.)

    Such scenario's are useful to see how different parts of your infrastructure can be affected by a single command – in this case your database (some kind of 'active' flag), your mail server, your system log, etc.

    Such scenario's also really help you in setting up a Test Driven Development environment.

    And finally, thinking in commands really helps you create a task-oriented application. Your users will appreciate this :-)

    Expressing Commands

    Django provides two easy ways of expressing commands; they are both valid options and it is not unusual to mix the two approaches.

    The service layer

    The service module has already been described by @Hedde. Here you define a separate module and each command is represented as a function.

    services.py

    def activate_user(user_id):
        user = User.objects.get(pk=user_id)
    
        # set active flag
        user.active = True
        user.save()
    
        # mail user
        send_mail(...)
    
        # etc etc
    

    Using forms

    The other way is to use a Django Form for each command. I prefer this approach, because it combines multiple closely related aspects:

    forms.py

    class ActivateUserForm(forms.Form):
    
        user_id = IntegerField(widget = UsernameSelectWidget, verbose_name="Select a user to activate")
        # the username select widget is not a standard Django widget, I just made it up
    
        def clean_user_id(self):
            user_id = self.cleaned_data['user_id']
            if User.objects.get(pk=user_id).active:
                raise ValidationError("This user cannot be activated")
            # you can also check authorizations etc. 
            return user_id
    
        def execute(self):
            """
            This is not a standard method in the forms API; it is intended to replace the 
            'extract-data-from-form-in-view-and-do-stuff' pattern by a more testable pattern. 
            """
            user_id = self.cleaned_data['user_id']
    
            user = User.objects.get(pk=user_id)
    
            # set active flag
            user.active = True
            user.save()
    
            # mail user
            send_mail(...)
    
            # etc etc
    

    Thinking in Queries

    You example did not contain any queries, so I took the liberty of making up a few useful queries. I prefer to use the term "question", but queries is the classical terminology. Interesting queries are: "What is the name of this user?", "Can this user log in?", "Show me a list of deactivated users", and "What is the geographical distribution of deactivated users?"

    Before embarking on answering these queries, you should always ask yourself this question, is this:

    Presentational queries are merely made to improve the user interface. The answers to business logic queries directly affect the execution of your commands. Reporting queries are merely for analytical purposes and have looser time constraints. These categories are not mutually exclusive.

    The other question is: "do I have complete control over the answers?" For example, when querying the user's name (in this context) we do not have any control over the outcome, because we rely on an external API.

    Making Queries

    The most basic query in Django is the use of the Manager object:

    User.objects.filter(active=True)
    

    Of course, this only works if the data is actually represented in your data model. This is not always the case. In those cases, you can consider the options below.

    Custom tags and filters

    The first alternative is useful for queries that are merely presentational: custom tags and template filters.

    template.html

    <h1>Welcome, {{ user|friendly_name }}</h1>
    

    template_tags.py

    @register.filter
    def friendly_name(user):
        return remote_api.get_cached_name(user.id)
    

    Query methods

    If your query is not merely presentational, you could add queries to your services.py (if you are using that), or introduce a queries.py module:

    queries.py

    def inactive_users():
        return User.objects.filter(active=False)
    
    
    def users_called_publysher():
        for user in User.objects.all():
            if remote_api.get_cached_name(user.id) == "publysher":
                yield user 
    

    Proxy models

    Proxy models are very useful in the context of business logic and reporting. You basically define an enhanced subset of your model. You can override a Manager’s base QuerySet by overriding the Manager.get_queryset() method.

    models.py

    class InactiveUserManager(models.Manager):
        def get_queryset(self):
            query_set = super(InactiveUserManager, self).get_queryset()
            return query_set.filter(active=False)
    
    class InactiveUser(User):
        """
        >>> for user in InactiveUser.objects.all():
        …        assert user.active is False 
        """
    
        objects = InactiveUserManager()
        class Meta:
            proxy = True
    

    Query models

    For queries that are inherently complex, but are executed quite often, there is the possibility of query models. A query model is a form of denormalization where relevant data for a single query is stored in a separate model. The trick of course is to keep the denormalized model in sync with the primary model. Query models can only be used if changes are entirely under your control.

    models.py

    class InactiveUserDistribution(models.Model):
        country = CharField(max_length=200)
        inactive_user_count = IntegerField(default=0)
    

    The first option is to update these models in your commands. This is very useful if these models are only changed by one or two commands.

    forms.py

    class ActivateUserForm(forms.Form):
        # see above
       
        def execute(self):
            # see above
            query_model = InactiveUserDistribution.objects.get_or_create(country=user.country)
            query_model.inactive_user_count -= 1
            query_model.save()
    

    A better option would be to use custom signals. These signals are of course emitted by your commands. Signals have the advantage that you can keep multiple query models in sync with your original model. Furthermore, signal processing can be offloaded to background tasks, using Celery or similar frameworks.

    signals.py

    user_activated = Signal(providing_args = ['user'])
    user_deactivated = Signal(providing_args = ['user'])
    

    forms.py

    class ActivateUserForm(forms.Form):
        # see above
       
        def execute(self):
            # see above
            user_activated.send_robust(sender=self, user=user)
    

    models.py

    class InactiveUserDistribution(models.Model):
        # see above
    
    @receiver(user_activated)
    def on_user_activated(sender, **kwargs):
            user = kwargs['user']
            query_model = InactiveUserDistribution.objects.get_or_create(country=user.country)
            query_model.inactive_user_count -= 1
            query_model.save()
        
    

    Keeping it clean

    When using this approach, it becomes ridiculously easy to determine if your code stays clean. Just follow these guidelines:

    The same goes for views (because views often suffer from the same problem).

    Some References

    Django documentation: proxy models

    Django documentation: signals

    Architecture: Domain Driven Design