Why do custom managers break related object queries?

I know this is a long one, but I promise its worth a read: useful solutions, and learning about the inner workings of Django!

The setup

I came across this issue while working with a logged model with a ForeignKey relationship to another logged model. When deserializing an instance, I want to include its related instances.

Note that I use the django_rest_framework serializers and specify the related fields in the Meta.fields option. This isn't very relevant so I won't include the code but can if requested.

Tiny Example:

models.py

class A(models.Model):
    ...
class B(models.Model):
    a = models.models.ForeignKey(A, on_delete=models.CASCADE)
    ...

Deserializing an instance of A should return something such as: {..., 'b_set': [3, 6, 7], ...}. We get a possibly empty array of related IDs.

The issue

The issue arises when adding custom managers:

• First, we define a LoggedModelManager, all it does is filter out the logs from the current instances.
• Second, we define the models A and its logs ALog. A gets the custom manager -> to ensure A.objects.all() only returns instances of A, not ALog, and then we ensure ALog has the default manager (django takes care of filtering away the non-log instances).
• Third we do the same for B. Notice that B has a ForeignKey to A.

# Defining the Manager
class LoggedModelManager(models.Manager):

     # Save the name of the logging model, which are `ALog` and `BLog` in our example.
    def __init__(self, *args, **kwargs):
        logging_model = kwargs.pop('logging_model')
        self.logging_model_name = logging_model.lower()
        super().__init__(*args, **kwargs)

    # Here we filter out the logs from the logged instances.
    def get_queryset(self) -> models.QuerySet:
        return super().get_queryset().filter(**{self.logging_model_name+"__isnull": True})

# Define A and its log
class A(models.Model):
    objects = LoggedModelManager(logging_model='ALog')
    ...
class ALog(A):
    objects = models.Manager()
    instance = models.ForeignKey(A, on_delete=models.CASCADE)

# Define B and its log
class B(models.Model):
    objects = LoggedModelManager(logging_model='BLog')
    ...
class BLog(B):
    objects = models.Manager()
    instance = models.ForeignKey(B, on_delete=models.CASCADE)

The outcome: A will deserialize to: {..., 'alog_set': [...], ...}. But notice that b_set is missing!

My (uninformed) solution.

After many hours of frustration and reading the docs, the following rewrite of B fixed this issue:

Note that BLog has to overwrite these modifications too but I've omitted this.

class B(models.Model):
    objects = LoggedModelManager(logging_model='BLog')
    objects_as_related = models.Manager()
    ...
    class Meta:
        base_manager_name = 'objects'
        default_manager_name = 'objects_as_related'

This works: A now deserializes to {..., 'alog_set': [...], 'b_set': [...] ...}. Note that the default_manager_name manager cannot be my custom manager.

My Question

1. Why does adding a custom manager break this? It returns a QuerySet, so why not return an empty 'b_set': [] instead of it not existing?

2. The documentation on Default Managers is super confusing, it says that:

"By default, Django uses an instance of the Model._base_manager manager class when accessing related objects, not the _default_manager on the related object."

But my example appears to show the contrary? Am I miss-reading the documentation?

3. I would appreciate any insights into this phenomenon or corrections into my statements!

Thank you for your time!

After a lot of debugging, I found the issue!

The Issue

This code did not work as intended because LoggedModelManager.__init__ fails when provided no parameters.

I made the assumption that mycustom manager is only instantiated when my model B is defined:

class B(models.Model):
    objects = LoggedModelManager(logging_model='BLog') # I thought this was the only place the constructor was called.

However, this is not the case. When resolving related model sets, the ModelSerializer class from the Django Rest Framework library will instantiate a manager for the related model. This instantiation is done with none of the parameters I expected, therefore an exception was raised and the related set ignored. Sadly, this error is not reported by the serializer, making it very hard to debug.

The Solution

Make sure your custom manager can be instantiated at a later time with no parameters. I did this by returning the default QuerySet if this was the case:

# Defining the Manager
class LoggedModelManager(models.Manager):

     # Save the name of the logging model, which are `ALog` and `BLog` in our example.
    def __init__(self, *args, **kwargs):
        if 'logging_model' in kwargs:
            logging_model = kwargs.pop('logging_model')
            self.logging_model_name = logging_model.lower()
        else:
            self.logging_model_name = None
        super().__init__(*args, **kwargs)

    # Here we filter out the logs from the logged instances.
    def get_queryset(self) -> models.QuerySet:
        if self.logging_model_name is None:
            return super().get_queryset()
        return super().get_queryset().filter(**{self.logging_model_name+"__isnull": True})

Hope this was helpful! I wonder if this is worth a ticket to ensure the Django Rest Framework serializers actually raise an error instead of silencing it.