pythondjangodjango-rest-frameworkmodelmulti-table-inheritance

Add Django model parent class to an existing "child" model for multi-table inheritance


I would like to add a new parent class to an existing model, which inherits from the same "super" parent class. Example of initial condition:

from django.db import models

class Ticket(PolymorphicModel):
    name = models.CharField(max_length=50)
    company = models.CharField(max_length=80)
    price = models.CharField(max_length=10)

class MovieTicket(Ticket):
    # functions and logic

For my implementation, I would like to add an "intermediary" EntertainmentTicket model, which inherits from Ticket and is inherited by MovieTicket for logic grouping purposes. Desired final condition:

class Ticket(PolymorphicModel):
    name = models.CharField(max_length=50)
    address = models.CharField(max_length=80)
    price = models.CharField(max_length=10)

class EntertainmentTicket(Ticket):
    # some functions and common logic extracted

class MovieTicket(EntertainmentTicket):
    # logic unique to MovieTicket

Note that the child classes have the same fields as Ticket, they only contain functions and logic. I cannot make Ticket into abstract, because I have other models pointing to it in a foreign key relationship. I use django-polymorphic to return the appropriate ticket type.

I have made migrations for EntertainmentTicket, I presume the next step is to create instances of EntertainmentTicket that point to the same Ticket instances that the current MovieTickets are pointing to. What is the best way to do this?


Solution

  • As long as you aren't adding extra fields to these two models, you can make them into proxy ones.

    class EntertainmentTicket(Ticket):
        class Meta:
          proxy = True
    
        # some functions and common logic extracted
    
    class MovieTicket(EntertainmentTicket):
        class Meta:
            proxy = True
    
        # logic unique to MovieTicket
    
    

    This has no effect on the database and it's just how you interact with them in Django itself


    Edit

    I completely missed that PolymorphicModel and wasn't aware of it actually making database tables..
    Proxy model's wouldn't really fit your existing scheme, so this is what I would do:

    1. Rename MovieTicket to EntertainmentTicket

    2. Create a new Model MovieTicket

    This could be done in two migrations, but I did makemigrations after each step and then copied them into 1 & deleted #2

    Example End Migration:

    # Generated by Django 3.2.4 on 2023-03-27 15:29
    
    from django.db import migrations, models
    import django.db.models.deletion
    
    class Migration(migrations.Migration):
    
        dependencies = [
            ('contenttypes', '0002_remove_content_type_name'),
            ('child', '0001_initial'),
        ]
    
        operations = [
            migrations.RenameModel(
                old_name='MovieTicket',
                new_name='EntertainmentTicket',
            ),
            migrations.CreateModel(
                name='MovieTicket',
                fields=[
                    ('entertainmentticket_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='child.entertainmentticket')),
                ],
                options={
                    'abstract': False,
                    'base_manager_name': 'objects',
                },
                bases=('child.entertainmentticket',),
            ),
        ]
    
    

    3. Handle old MovieTicket objects (now EntertainmentTicket)

    # Create Movie Ticket objects pointing to the new model
    for i in EntertainmentTicket.objects.all():
      MovieTicket.objects.create(entertainmentticket_ptr_id=i.pk)
    

    I did all this in a test project and it all worked, plus it keeps your PolymorphicModel thing


    Edit 2

    ya just had to make it more complicated, didn't you?!- this was horrible! but i did learn alot.

    I recommend making a temp project and playing around with it before trying anything in Production.

    Starting Models:

    from django.db import models
    from polymorphic.models import PolymorphicModel
    
    class Ticket(PolymorphicModel):
        name = models.CharField(max_length=50)
        company = models.CharField(max_length=80)
        price = models.CharField(max_length=10)
    
    class MovieTicket(Ticket):
        pass
    
    class MarvelTicket(MovieTicket):
        pass
    
    class TheatreTicket(Ticket):
        pass
    

    Starting Data:

    # Note my app was named child
    from child.models import *
    MoveTicket.objects.create(name='test movie', company='test company', price='$5')
    MarvelTicket.objects.create(name='Ant Man', company='Marvel', price='$250')
    TheatreTicket.objects.create(name='Singing in the Rain', company='Gene Kelly ', price='$2')
    

    Updated Models:

    from django.db import models
    from polymorphic.models import PolymorphicModel
    
    class Ticket(PolymorphicModel):
        name = models.CharField(max_length=50)
        company = models.CharField(max_length=80)
        price = models.CharField(max_length=10)
    
    class EntertainmentTicket(Ticket):
        pass
    
    class MovieTicket(EntertainmentTicket):
        pass
    
    class MarvelTicket(MovieTicket):
        pass
    
    class TheatreTicket(EntertainmentTicket):
        pass
    

    Migrations

    See: Moving a Django Foreign Key to another model while preserving data? for the blueprint of how

    You'll need to change the child in all of the apps.get_model('child', 'Ticket') calls to match your app

    Migration #1. Temp Model, Deleting.

    Steps:

    # Generated by Django 3.2.4 on 2023-03-27 23:17
    
    from django.db import migrations, models
    import django.db.models.deletion
    
    def create_transfer_objects(apps, schema_editor):
        """
        Pack Movie, Theatre and Marvel into the temp model
    
        Structure of temp:
            pk          = auto-generated number
            ticket_type = two char string
            ticket_pk   = PK for parent Ticket
    
        Explications:
            pk, needs a pk
            ticket_type, so it knows what model to create with later
    
            ticket_pk, what parent ticket it's associated with
    
                Note Marvel still stores parent ticket and fetch Movie from ticket
                    I just found this easier instead of adding more temp fields
        """
    
        transfer_model = apps.get_model('child', 'TransferModel')
    
        # create direct -> ticket items
        entertainment_models = [
            ['00', apps.get_model('child', 'MovieTicket')],
            ['01', apps.get_model('child', 'TheatreTicket')],
        ]
        for t, m in entertainment_models:
            for m_obj in m.objects.all():
                transfer_model.objects.create(
                    ticket_type=t,
                    ticket_pk=m_obj.ticket_ptr.pk,
                )
    
        # create passthrough ticket items
        # pass through item's pk is still a ticket, just another name
        entertainment_models = [
            ['02', apps.get_model('child', 'MarvelTicket')],
        ]
        for t, m in entertainment_models:
            for m_obj in m.objects.all():
                if '02':
                    # use movie ticket
                    ticket_pk = m_obj.movieticket_ptr.pk
                # elif ...
                transfer_model.objects.create(
                    ticket_type=t,
                    ticket_pk=ticket_pk,
                )
    
    def reverse_transfer_objects(apps, schema_editor):
        """
        Reverse the process of creating the transfer object,
        This will actually create the Movie, Theatre and Marvel objects
    
        """
    
        transfer_model = apps.get_model('child', 'TransferModel')
        ticket_model = apps.get_model('child', 'Ticket')
    
        # reverse direct -> ticket items
        target_dict = {
            '00': apps.get_model('child', 'MovieTicket'),
            '01': apps.get_model('child', 'TheatreTicket'),
        }
        for obj in transfer_model.objects.filter(ticket_type__in=target_dict.keys()):
            target_dict[obj.ticket_type].objects.create(
                ticket_ptr=ticket.objects.get(pk=obj.ticket_pk),
            )
    
        # reverse passthrough ticket items
        # Note: This only does "1 level" below
        target_dict = {
            '02': {
                'obj': apps.get_model('child', 'MarvelTicket'),
                'target': apps.get_model('child', 'MovieTicket'),
                'field': 'movieticket_ptr',
            },
        }
        for obj in transfer_model.objects.filter(ticket_type__in=target_dict.keys()):
            target_dict[obj.ticket_type]['obj'].objects.create(**{
                target_dict[obj.ticket_type]['field']: target_dict[obj.ticket_type]['target'].objects.get(pk=obj.ticket_pk),
            })
    
    def migrate_to_entertainment(apps, schema_editor):
        """
        Create Entertainment objects from tickets
        """
        ticket_model = apps.get_model('child', 'Ticket')
        entertainmentticket_model = apps.get_model('child', 'EntertainmentTicket')
        for ticket_obj in ticket_model.objects.all():
            entertainmentticket_model.objects.create(
                ticket_ptr=ticket_obj
            )
    
    def reverse_entertainment(apps, schema_editor):
        # Removing Model should do the trick
        pass
    
    class Migration(migrations.Migration):
    
        dependencies = [
            ('child', '0001_initial'),
        ]
    
        operations = [
            # create a temp model to old values
            migrations.CreateModel(
                name='TransferModel',
                fields=[
                    ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                    ('ticket_type', models.CharField(choices=[('00', 'Movie'), ('01', 'Theatre'), ('02', 'Marvel')], default='00', max_length=2)),
                    ('ticket_pk', models.PositiveIntegerField(default=0)),
                ],
            ),
    
            # Create & Pack TransferModels
            migrations.RunPython(
                # This is called in the forward migration
                create_transfer_objects,
                # This is called in the backward migration
                reverse_code=reverse_transfer_objects
            ),
    
            # Delete
            migrations.DeleteModel(
                name='MovieTicket',
            ),
            migrations.DeleteModel(
                name='TheatreTicket',
            ),
            migrations.DeleteModel(
                name='MarvelTicket',
            ),
    
            # Create New Model
            migrations.CreateModel(
                name='EntertainmentTicket',
                fields=[
                    ('ticket_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='child.ticket')),
                ],
                options={
                    'abstract': False,
                    'base_manager_name': 'objects',
                },
                bases=('child.ticket',),
            ),
    
            # Connect Entertainment to Ticket
            migrations.RunPython(
                # This is called in the forward migration
                migrate_to_entertainment,
                # This is called in the backward migration
                reverse_code=reverse_entertainment
            ),
        ]
    

    Migration #2. Reforming

    Steps:

    # Generated by Django 3.2.4 on 2023-03-28 01:01
    
    from django.db import migrations, models
    import django.db.models.deletion
    
    def connect_to_entertainment(apps, schema_editor):
        """
        Recreate all the lost models using Temp Model,
        Connect through Entertainment instead of Ticket directly.
        """
        transfer_model = apps.get_model('child', 'TransferModel')
        entertainmentticket_model = apps.get_model('child', 'EntertainmentTicket')
    
        target_dict = {
            '00': apps.get_model('child', 'MovieTicket'),
            '01': apps.get_model('child', 'TheatreTicket'),
        }
        for obj in transfer_model.objects.filter(ticket_type__in=target_dict.keys()):
            target = entertainmentticket_model.objects.get(
                ticket_ptr__pk=obj.ticket_pk,
            )
            target_dict[obj.ticket_type].objects.create(
                entertainmentticket_ptr=target,
            )
    
        # Create passthrough ticket items
        target_dict = {
            '02': {
                'obj': apps.get_model('child', 'MarvelTicket'),
                'target': apps.get_model('child', 'MovieTicket'),
                'field': 'movieticket_ptr',
            },
        }
        for obj in transfer_model.objects.filter(ticket_type__in=target_dict.keys()):
            target_dict[obj.ticket_type]['obj'].objects.create(**{
                target_dict[obj.ticket_type]['field']: target_dict[obj.ticket_type]['target'].objects.get(
                    pk=obj.ticket_pk
                ),
            })
    
    def reverse_entertainment_connect(apps, schema_editor):
        """
        Recreate Temp Model from Movie, Theatre and Marvel.
        """
        transfer_model = apps.get_model('child', 'TransferModel')
    
        entertainment_models = [
            ['00', apps.get_model('child', 'MovieTicket')],
            ['01', apps.get_model('child', 'TheatreTicket')],
        ]
        for t, m in entertainment_models:
            for m_obj in m.objects.all():
                transfer_model.objects.create(
                    ticket_type=t,
                    ticket_obj=m_obj.entertainmentticket_ptr.ticket_ptr,
                )
    
        # Create passthrough ticket items
        entertainment_models = [
            ['02', apps.get_model('child', 'MarvelTicket')],
        ]
        for t, m in entertainment_models:
            for m_obj in m.objects.all():
                if '02':
                    # use movie ticket
                    ticket_pk = m_obj.movieticket_ptr.pk
                # elif ...
                transfer_model.objects.create(
                    ticket_type=t,
                    ticket_pk=ticket_pk,
                )
    
    class Migration(migrations.Migration):
    
        dependencies = [
            ('child', '0002_transfer'),
        ]
    
        operations = [
            # Create the Previously Removed Models
            migrations.CreateModel(
                name='MovieTicket',
                fields=[
                    ('entertainmentticket_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='child.entertainmentticket')),
                ],
                options={
                    'abstract': False,
                    'base_manager_name': 'objects',
                },
                bases=('child.entertainmentticket',),
            ),
            migrations.CreateModel(
                name='TheatreTicket',
                fields=[
                    ('entertainmentticket_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='child.entertainmentticket')),
                ],
                options={
                    'abstract': False,
                    'base_manager_name': 'objects',
                },
                bases=('child.entertainmentticket',),
            ),
            migrations.CreateModel(
                name='MarvelTicket',
                fields=[
                    ('movieticket_ptr', models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to='child.movieticket')),
                ],
                options={
                    'abstract': False,
                    'base_manager_name': 'objects',
                },
                bases=('child.movieticket',),
            ),
    
            #  Recreate
            migrations.RunPython(
                # This is called in the forward migration
                connect_to_entertainment,
                # This is called in the backward migration
                reverse_code=reverse_entertainment_connect
            ),
    
            # Delete Temp Model
            migrations.DeleteModel(
                name='TransferModel',
            ),
    
        ]
    

    Run python manage migrate

    Side Notes:

    I know this is a drastic change and a lot of stuff, but holy it gets complicated quick!

    I tried for the longest time to keep the original models and just repoint or rename, but because they were One-to-One and Primary Keys it was doomed from the start.

    I also tried doing it in a single migration, but Django didn't like the immediate recreation of exact models. it was almost as if they weren't forgotten yet.

    If you had another middle like model like:

    class Ticket(PolymorphicModel):
        pass
    
    class SpeakerTicket(Ticket):  # <- like this
        pass
    
    class MovieTicket(Ticket):
        pass
    
    # etc..
    

    All you'd have to do is filter out those items when creating the EntertainmentTicket objects in the migrations

    # Migrations #1
    def migrate_to_entertainment(apps, schema_editor):
        """
        Create Entertainment objects from tickets
        """
        ticket_model = apps.get_model('child', 'Ticket')
        entertainmentticket_model = apps.get_model('child', 'EntertainmentTicket')
    
    
        ticket_object_list = ticket_model.objects.all()
    
        speaker_model = apps.get_model('child', 'SpeakerTicket')
        ticket_object_list = ticket_object_list.exclude(
            pk__in=speaker_model.objects.all().values_list('ticket_ptr')
        )
    
        # .. exclude another, etc
    
        for ticket_obj in ticket_object_list:
            entertainmentticket_model.objects.create(
                ticket_ptr=ticket_obj
            )