Migration and Data Seeding

Important

Sorry, this section is outdated. Kindly check the Data Seeder module and Service documentation.

We have our equipment microservice up and running, now we need a way to automatically seed some testing data for the local and staging environments. The Suite has a Migrations microservice which is in charge of doing so. We need to configure the Migrations microservice to produce seed instances of our Equipment entity. It also provides a migrations template with basic scaffolding for seeding new entities, which we can run as follows:

dotnet new migrations -E Equipment

This command will generate the following files inside Migrations BoundedContext

1. EquipmentMigratableEntity
2. EquipmentMigrationProfile
3. MigrateEquipmentsModule
4. EquipmentsInMemoryMigrationModule
5. EquipmentsInMemoryMigrationModuleOptions
6. MigrateEquipmentsInMemoryProvider
7. EquipmentsTestDataMigrationModule
8. EquipmentsToMigrate

Lets see how to configure each class.

EquipmentMigratableEntity

Locate EquipmentMigratableEntity inside ITsynch.Suite.Migrations.Entities/MigrationModules/Equipments and adapt it with the properties from our entity.

namespace ITsynch.Suite.Migrations.Entities.Equipments
{
    public class EquipmentMigratableEntity : IMigratableEntity
    {
        public Guid? CorrelationId { get; set; }

        public string? LegacyId { get; set; }

        public string DisplayName { get; set; } = null!;

        public string Code { get; set; } = null!;
    }
}

The class must implement the IMigratableEntity interface and have the same fields the entity has, in our case, DisplayName and Code.

Configuring AutoMapper profile

The migration module needs to be able to map our EquipmentMigratableEntity into CreateOrUpdateEquipment message. First let's add a reference to the ITsynch.Suite.Equipments.Application.Contracts project to the ITsynch.Suite.Migrations.Entities project so we can reference the creation message. To do so add the following:

<ProjectReference
Include="$(ServicesPath)Equipments/ITsynch.Suite.Equipments.Application.Contracts/ITsynch.Suite.Equipments.Application.Contracts.csproj" />

Then on the same Equipments folder locate EquipmentsMigrationProfile file and add the corresponding mappings.

namespace ITsynch.Suite.Migrations.Entities.Equipments;

internal class EquipmentMigrationProfile : Profile
{
    public EquipmentMigrationProfile()
    {
        this.CreateMap<EquipmentMigratableEntity, CreateOrUpdateEquipment>()
            .ForMember(m => m.CorrelationId, b => b.MapFrom(info => info.CorrelationId))
            .ForMember(m => m.DisplayName, b => b.MapFrom(info => info.DisplayName))
            .ForMember(m => m.Code, b => b.MapFrom(info => info.Code));
    }
}

MigrateEquipmentsModule

Template configures a MigrateEquipmentsModule where we register our migratable entity and its corresponding automapper profile. We still need to add a reference to contracts project inside this file.

using ITsynch.Suite.Equipments.Application;

namespace ITsynch.Suite.Migrations.Entities;

public class MigrateEquipmentsModule : SuiteModule
{
    public override void SetupModule(IModuleBuilder builder)
    {
        base.SetupModule(builder);

        builder.DependsOn<MigrationModule, MigrationModuleOptions>(opts =>
        {
            opts.AddFlatEntity<MigrateEquipmentsModule, EquipmentMigratableEntity, CreateOrUpdateEquipment, EquipmentUpdated>();
        });
    }
}

As seen above, we need to depend from the MigrationModule and register our entity using the AddFlatEntity method on the MigrationModuleOptions. This method has 4 generic parameters, first the module itself, second the IMigratableEntity, third the message that creates our entity, and fourth the response message, because the Migrations microservice uses Request/Response when it creates the entities.

Provider

We want to be able to seed equipments on the LocalDev environment, for that have an in-memory provider for EquipmentMigratableEntity, alongside an Options class and a Module on the ITsynch.Suite.Migrations.InMemoryProviders project, insideEnvironments/Shared directory.

EquipmentsInMemoryMigrationModuleOptions class will look like this.

namespace ITsynch.Suite.Migrations;

public class EquipmentsInMemoryMigrationModuleOptions
{
    public ICollection<EquipmentMigratableEntity> EquipmentsToMigrate { get; } = new List<EquipmentMigratableEntity>();
}

In this option we define a collection where we are later going to deposit the instances of equipments to be created.

Then, on the same folder we have EquipmentsInMemoryMigrationModule which should look as follows:

namespace ITsynch.Suite.Migrations;

public class EquipmentsInMemoryMigrationModule : SuiteModule
{
    public override void SetupModule(IModuleBuilder builder)
    {
        base.SetupModule(builder);
        builder.DependsOn<MigrateEquipmentsModule>();
        builder.AddOptions<EquipmentsInMemoryMigrationModuleOptions>();
    }
}

We need to make the module to depend on the MigrateEquipmentModule module that we created early, where we defined the EquipmentMigratableEntity class. Then we register the options class we created before using the AddOptions method.

Finally we have the MigrateEquipmentsInMemoryProvider:

namespace ITsynch.Suite.Migrations;

[ScopedDependency(typeof(IFlatEntityMigrationProvider<EquipmentMigratableEntity>))]
internal class MigrateEquipmentsInMemoryProvider : BaseFlatInMemoryProvider<EquipmentMigratableEntity>
{
    private readonly EquipmentsInMemoryMigrationModuleOptions equipmentsInMemoryMigrationModuleOptions;

    public MigrateEquipmentsInMemoryProvider(IOptions<EquipmentsInMemoryMigrationModuleOptions> equipmentsInMemoryMigrationModuleOptions)
    {
        this.equipmentsInMemoryMigrationModuleOptions = equipmentsInMemoryMigrationModuleOptions.Value;
    }

    protected override IEnumerable<EquipmentMigratableEntity> GetAllEntities() =>
        this.equipmentsInMemoryMigrationModuleOptions.EquipmentsToMigrate;
}

We need to inherit from the BaseFlatInMemoryProvider class and implement its GetAllEntities method. To do so we inject the options we created before and we return the EquipmentsToMigrate collection. We are also registering the Provider on DI using the ScopedDependency attribute.

Now we have an in-memory equipments provider that will seed any equipments that we insert on the EquipmentsToMigrate collections. This allows us to fill that collection with different data depending on the environment.

Important

In most cases, no further setup of these classes is needed. Yet, it is critical that you ensure everything makes sense and this scaffolding is compliant with your own requirements.

The Test Data project

As we mentioned on the previous section, we have an implementation of the in-memory provider that we can use to provide data just by depending from the EquipmentsInMemoryMigrationModule and using its option class to fill the collection. At this point we could import that module on the LocalDev environment and seed data, and then do the same on the Staging environment. That would be ok, but we would have some duplicated code as we want the same equipments to be seeded on both the LocalDev and the Staging environments. To avoid that we have the TestData project that acts as an intermediate environment where we can seed data that is going to be used in both environments, then each environment only has to import the data for the desired entities.

Creating the Test Data

Find the ITsynch.Suite.Migrations.TestData project on the Environments/Shared directory. Here we have an Equipments folder with a static class where we can seed our test data.

namespace ITsynch.Suite.Migrations;

internal static class EquipmentsToMigrate
{
    public static EquipmentMigratableEntity Engine { get; } = new EquipmentMigratableEntity()
    {
        CorrelationId = new Guid("90df0000-5438-00ff-826f-08d9ab912871"),
        Code = "1",
        DisplayName = "Engine",
    };

    public static EquipmentMigratableEntity Fridge { get; } = new EquipmentMigratableEntity()
    {
        CorrelationId = new Guid("90df0000-5438-00ff-b95c-08d9ab912871"),
        Code = "2",
        DisplayName = "Fridge",
    };

    public static IEnumerable<EquipmentMigratableEntity> All { get; } = new[]
    {
            Engine,
            Fridge,
    };
}

Then we have Suite Module called EquipmentsTestDataMigrationModule which looks as follows:

namespace ITsynch.Suite.Migrations;

public class EquipmentsTestDataMigrationModule : SuiteModule
{
    public override void SetupModule(IModuleBuilder builder)
    {
        base.SetupModule(builder);

        builder.DependsOn<EquipmentsInMemoryMigrationModule, EquipmentsInMemoryMigrationModuleOptions>(opts =>
        {
            opts.EquipmentsToMigrate.AddRange(EquipmentsToMigrate.All);
        });

In it we add a dependency to our in-memory provider module, and we use its options to seed the hardcoded data we created above.

Adding Equipments Test Data to the LocalDev environment

Finally we can add our test data to the LocalDev environment. To do so, locate the LocalDevMigrationModule on the ITsynch.Suite.Migrations.LocalDev project. On it we will add a dependency to our EquipmentsTestDataMigrationModule module below the already existing dependencies:

   public override void SetupModule(IModuleBuilder builder)
    {
        base.SetupModule(builder);

        builder.DependsOn<AdminCenterTestDataMigrationModule>();
        builder.DependsOn<ApplicationsTestDataMigrationModule, ApplicationsTestDataMigrationModuleOptions>(opts =>
        {
            opts.SuiteEnvironment = "localdev";
        });
        builder.DependsOn<ComponentsTestDataMigrationModule>();
        builder.DependsOn<DepartmentsTestDataMigrationModule>();
        // Coded omitted for brevity.
        // ...
        builder.DependsOn<EquipmentsTestDataMigrationModule>();
    }