djangomigrationdatabase-migrationdjango-migrations

How to make your custom django migrations reversible?


My original problem was, given a db table with 60M rows I need to convert a field type from boolean to integer field. I thought of creating a custom django migration for this (please do let me know if you have a better approach than this) which looks like this-

def make_changes(apps, schema_editor):
    vcs_model = apps.get_model('iot_app', 'AbstractVCSCompartmentData')
    vcs_model.objects.select_related('vcsdata').all().update(charging_status_v2=F('charging_status'))
    vcs_model.objects.select_related('vcsdata').all().update(charging_status_backup=F('charging_status'))  # backup




class Migration(migrations.Migration):
    dependencies = [
        ('iot_app', '0030_auto_20220225_1027.py'),
    ]

    operations = [

        migrations.AddField(  # backup field
            model_name='AbstractVCSCompartmentData',
            name='charging_status_backup',
            field=models.PositiveIntegerField(blank=True, null=True),
        ),


        migrations.AddField(
            model_name='AbstractVCSCompartmentData',
            name='charging_status_v2',
            field=models.PositiveIntegerField(blank=True, null=True),
        ),

        migrations.RunPython(make_changes),

        migrations.RemoveField(
            model_name='AbstractVCSCompartmentData',
            name='charging_status',
        ),
        migrations.RenameField(
            model_name='AbstractVCSCompartmentData',
            old_name='charging_status_v2',
            new_name='charging_status',
        ),
    ]


I want to unroll all the changes i.e., making my custom migration reversible. I have gone through RunPython doc. But I am confused as in how can i perform addition of a new field in my reverse_code() function.

The idea of creating a backup field is to reinstate the db to its previous state.

Is this is the correct approach, given I have 60M rows in that table? wouldn't it lock the table for the time being? I want to do this as efficient as possible


Solution

  • i edited my answer after your comments - at the moment when RunPython is executed in reversion step, charging_status has been renamed to charging_status_v2 and the old charging_status field is present. So you can copy directly from backup to charging_status

    def make_changes(apps, schema_editor):
        vcs_model = apps.get_model('iot_app', 'AbstractVCSCompartmentData')
        vcs_model.objects.select_related('vcsdata').all().update(charging_status_v2=F('charging_status'))
        vcs_model.objects.select_related('vcsdata').all().update(charging_status_backup=F('charging_status'))  # backup
    
    
    def backwards(apps, schema_editor):
        vcs_model = apps.get_model('iot_app', 'AbstractVCSCompartmentData')
        vcs_model.objects.select_related('vcsdata').all().update(charging_status=F('charging_status_backup'))
    
    
    
    class Migration(migrations.Migration):
        dependencies = [
            ('iot_app', '0030_auto_20220225_1027.py'),
        ]
    
        operations = [
    
            migrations.AddField(  # backup field
                model_name='AbstractVCSCompartmentData',
                name='charging_status_backup',
                field=models.PositiveIntegerField(blank=True, null=True),
            ),
    
    
            migrations.AddField(
                model_name='AbstractVCSCompartmentData',
                name='charging_status_v2',
                field=models.PositiveIntegerField(blank=True, null=True),
            ),
    
            migrations.RunPython(make_changes, backwards),
    
            migrations.RemoveField(
                model_name='AbstractVCSCompartmentData',
                name='charging_status',
            ),
            migrations.RenameField(
                model_name='AbstractVCSCompartmentData',
                old_name='charging_status_v2',
                new_name='charging_status',
            ),
        ]