How do you preserve createdon, createdby, modifiedon, and modifiedby in a Dataverse migration?

Having tried SSIS and KingswaySoft, I opted to write a console app to migrate my Dataverse tables instead. Code is more familiar and has proved easier for me than the no-code approach. I ran into an issue preserving the user/datestamps from the source records. When migrating these values are overwritten by the system.

I found an OOTB way to override createdon using overriddencreatedon but that approach is not comprehensive and doesn't work for all 4 fields.

How can the meta facts from the source record be preserved when migrated to the target table?

I authored a RecordStamps plugin and connected it to the Create/Update operations for each of the tables being migrated, in particular the Pre Operation stage.

I created an intermediary table for migration purposes called Migration Meta(s). In it I created Meta Created On, Meta Created By, Meta Modified On, Meta Modified By, all to ensure different column names from the system updated ones. I also named the primary name field Logical Name.

In the migration app, just before migrating any record, I perform this bit of logic:

//cache meta for use in RecordStamps Plugin.
try {
  var mm = new Entity("crab9_migrationmeta", rec.Id);
  mm["crab9_logicalname"] = rec.LogicalName;
  foreach (var key in new string[] { "modifiedby", "modifiedon", "createdby", "createdon" }) {
    mm[$"crab9_meta{key}"] = rec.Contains(key) ? rec[key] : null;
  }
  var found = false;
  try {
    this.targetDb.Retrieve(mm.LogicalName, mm.Id, new ColumnSet(false));
    found = true;
  } catch {
  }
  if (found) {
    this.targetDb.Update(mm);
  } else {
    this.targetDb.Create(mm);
  }
} catch (Exception ex) {
  throw new Exception("Unable to set meta.", ex);
}

I save the source record metadata values to this one table and so track every record which gets migrated. Recall that GUIDs are globally unique. After the meta is saved, I then migrate the record from the source to the target Dataverse environment.

The RecordStamps Plugin intercepts the Create/Update, uses the Id and Logical Name to Retrieve the bullseye hit from the Migration Meta table and then replaces those facts just before they're written. Since it adds some overhead to the environment, it's only enabled during migrations.

using System;
using Microsoft.Xrm.Sdk;
using Microsoft.Xrm.Sdk.Query;

namespace RecordStamps {
  public class RecordStampsPlugin : IPlugin {
    public static readonly string[] fields  = new string[] { "createdon", "createdby", "modifiedon", "modifiedby" };

    public void Execute(IServiceProvider serviceProvider) {
      var tracingService = (ITracingService)serviceProvider.GetService(typeof(ITracingService));
      var context = (IPluginExecutionContext)serviceProvider.GetService(typeof(IPluginExecutionContext));
      var serviceFactory = (IOrganizationServiceFactory)serviceProvider.GetService(typeof(IOrganizationServiceFactory));
      var service = serviceFactory.CreateOrganizationService(context.UserId);
      var target = (Entity)context.InputParameters["Target"];

      if (context.Stage == 20) {
        try {
          tracingService.Trace("Checking record stamps at {1} for {0}", Tracing.Show(target), context.MessageName);

          var mm = service.Retrieve("crab9_migrationmeta", target.Id, new ColumnSet(true));
          if (mm != null) {
            if (mm["crab9_logicalname"] != target.LogicalName) {
              throw new Exception("Meta logical name does not match target.");
            }
            tracingService.Trace("Applying record stamps {0}", Tracing.ShowAll(mm));
            foreach (var key in fields) {
              var mkey = $"crab9_meta{key}";
              if (mm[mkey] != null) {
                target[key] = mm[mkey];
                tracingService.Trace("Applied {0} => {1}", key, Tracing.Show(target[key]));
              }
            }
          }
        } catch (Exception ex) {
          tracingService.Trace("Error {0}", Tracing.Show(ex));
        }
      }
    }
  }
}

There may be some iteration with this plugin logic, but so far it's worked well. The Tracing.Show is custom logic for pretty printing.

I mention Create/Update. I found I could not just create inactive records in the target environment as taken from the source. I had to initially migrate them (Create) as active and then a moment later (Update) set them inactive.

The additional benefit of the Migration Meta table is it is easy to undo a migration as everything you migrated is tracked. You can add other fields to it if you want to capture anything else relevant to your migration. My console app supports "migrate" and "purge" commands. This makes it well suited to iterative development in a test environment.

As systemuser records are key dependencies, I migrate them first and I don't bother fixing their meta facts. You can scan your source tables to determine which ones are needed.

Also, for some tables, there are records which already exist in the target environment under a different id. I load some dictionaries to track and handle these mappings.