fidzu : web

I've always been a big advocate of Drupal's Migrate API. It's incredibly flexible, allowing developers to pull data from just about anywhere. With contributed modules like Migrate Plus and Migrate Source CSV, that includes CSV files, SQL databases, JSON APIs, and other content management systems.

But migrations can be time-consuming. The process usually looks something like this:

1. Discover the existing data and define the new content structures.
2. Map source data to destination fields.
3. Write migration configurations and transformations.
4. Run migrations, QA the results, fix problems, and repeat.

That final step is often the most frustrating. Complex relationships, inconsistent legacy data, and unexpected edge cases can turn migration QA into a lengthy process of trial and error.

Drupal migrations are a particularly good fit for AI-assisted development. There's a lot of structured data to analyze, custom code to write, and repetitive testing to work through. I've found it especially valuable for data discovery, transformations, and QA.

Understanding the source data

Before writing migration configurations, we often need to spend considerable time understanding the source data, especially when working with unfamiliar databases or legacy content structures.

With a local, container-based development environment, an AI coding assistant can be given controlled access to a copy of the source database. This allows it to inspect tables, write SQL queries, identify patterns, and produce reports without manually investigating every relationship.

Consider rich-text fields in Drupal. They might contain embedded images, media references, links to PDFs, or other files that aren't immediately obvious from the content structure.

On a recent project, we wanted to migrate only media assets that were actually referenced in content. Rather than migrating every file from the legacy system, I used AI to analyze rich-text fields and generate temporary database tables containing unique references to images and documents.

This gave us a clearer picture of which files needed to be migrated and where they were being used.

A prompt might be as straightforward as:

Analyze the rich-text fields in the source database. Identify unique references to images, embedded media, and linked documents. Create a temporary table containing each referenced asset and the source content that uses it. Report any references that cannot be resolved.

Tasks like this previously required writing exploratory SQL, custom scripts, and repeatedly inspecting results. AI can significantly reduce that effort.

Custom transformations

Drupal's migration process plugins are powerful. They allow us to transform incoming data, manipulate values, look up related entities, and handle project-specific requirements.

Core and contributed modules cover many common scenarios, but some migrations require custom process plugins.

Recently, I used AI to help write a custom process plugin that validated existing addresses using Google's Address Validation API.

The site contained approximately 4,500 address entities, many referenced by other content. We suspected there were duplicates and inconsistencies, but determining which addresses could be safely removed wasn't straightforward.

The validation results helped us identify potential duplicates and flag existing address entities for review. This gave us a path toward reducing approximately 4,500 address entities to around 3,000, while accounting for their existing references.

Google's API doesn't perform deduplication. Instead, it gave us more consistent information to inform decisions about which entities should be retained or removed.

This is the kind of improvement I'd considered on previous projects but often couldn't justify because of the development effort involved.

AI makes it much more practical to explore these opportunities. A process plugin that might have required hours of research and development can often be drafted quickly, then reviewed, tested, and refined.

Not every transformation needs to be permanent, reusable code. Some are highly specific to a single migration. That's perfectly fine.

A migration is also an excellent opportunity to improve legacy data rather than simply reproduce its problems in a new system.

Making migration QA more effective

This is where I've found some of the biggest benefits.

Traditional migration QA involves reviewing imported content, identifying issues, adjusting migration scripts, and running everything again.

AI can help automate much of that investigative cycle.

For example, after running a migration, I might ask:

Select 10 migrated entities that collectively represent the most important edge cases, including complex field values, media references, and entity relationships. Compare the source and destination data, identify discrepancies, and explain why each example was selected.

An AI coding assistant with access to both databases can inspect records, compare values, and identify inconsistencies that might otherwise take considerable time to uncover.

It can also help investigate failures, propose fixes to migration configurations or process plugins, and rerun targeted migrations for verification.

Of course, reviewing 10 records doesn't prove an entire migration is correct. Automated checks for record counts, missing references, failed imports, and unexpected field values remain important.

But AI makes it easier to develop those checks and identify meaningful samples.

There's a client-facing benefit here, too. Instead of manually searching for representative content to include in a QA handoff, we can generate a curated set of examples covering different content types, relationships, and transformations.

That gives clients a more useful starting point for reviewing migrated content.

Keep developers in control

AI-assisted migrations still require an understanding of Drupal's data structures, the Migrate API, and the intended destination architecture.

Generated migration code needs review. Data transformations should be deterministic and testable. Complex relationships and potentially destructive cleanup operations require particular care.

It's also important to be deliberate about data access. A local container provides a controlled database environment, but that doesn't automatically mean data shared with an AI service stays local. Use approved tools, minimize access, and avoid exposing sensitive production data. The same considerations apply to third-party APIs, including restrictions on storing or reusing their response data.

The goal isn't to let AI independently decide how data should be migrated. It's to give experienced developers a faster way to investigate problems, test solutions, and verify results.

More time for the work that matters

Drupal's Migrate API was already one of my favorite parts of the platform. AI-assisted development makes it even more useful.

What excites me most isn't necessarily generating migration configurations faster. It's the ability to explore source data more thoroughly, tackle transformations that previously seemed too expensive, and make the QA process considerably more efficient.

Migrations will always require careful planning and validation. But spending less time tracking down obscure data issues means more time improving the quality of the migration itself.