5 Lessons Learned for Content Migration

This past fall I undertook my first content migration effort. This was to migrate an existing site from Expression Engine to Drupal 7. I immediately began looking at the Migrate module to learn its use and here are some lessons I learned along the way.

First of all, be aware that Migrate is a module for developers. It does have a user interface, but this is merely to run the import process. All the real action happens in the code, so you will need to be familiar with object oriented programming in PHP.

Lesson One: Don't Force a 1-1 Relationship

Do not try to force a 1-1 relationship with your migration classes and your destination content types. If you are migrating a site, you very likely are taking the equivalent of Drupal content type in the legacy system, creating similar types in Drupal, and probably combining a few of these because you realized in discovery sessions that some of the legacy system's content types are redundant. Although I did not use this method myself, given another migration project, I would create a 1-1 relationship between the migration classes and the legacy system's content types. This will avoid some awkward logic in your migration code as you try to route multiple possible source fields into a single field in Drupal. In other words, don't do this:

Source type
Source type ---> Migration ---> Drupal content type
Source type /

Do this:

Source type ---> Migration ---> Drupal content type
Source type ---> Migration ---> Drupal content type
Source type ---> Migration ---> Drupal content type

Lesson Two: Document

Be obsessive about in-code documentation/comments. In code comments are always a good idea, and you can never comment too much, but this is especially true with migrations. I did this from the start and it helped greatly when I had to revisit the migration code. Migrate is handy in that it provides tools to integrate in a project management system by linking to available tickets and places to add descriptions for mappings. Use these, your client's product owner will greatly appreciate it as they are reviewing the state of the migration. Which brings us to Lesson 3.

Lesson Three: Break out your migration into tasks

Break out the migration task into it's own project/ticketing system and use it! Migration tasks seem to live in a bubble within the larger site build project. Avoid this by separating the migration effort into it's own sub-project with a dedicated ticketing system. This way you won't clutter up the build's ticket queue with migration work and the whole team can observe the progress of the migration.

Lesson Four: A Gotchya!

A small gotchya on Migrate: If your source and legacy databases are owned by separate users, remember to set the "join" option to false in each of your migration classes with this bit of code:

$options = array('map_joinable' => FALSE); 
$this->source = new MigrateSourceSQL($query, array(), $count_query, $options);

This may be the cause of some permissions errors. You may wish to make these databases joinable however, since the performance hit of not joining these is huge, (close to doubling the import in my own casual observation).

Lesson Five: Entity Reference Field Handlers

Initially I thought that Entity references with more than a single value weren't supported, because I tried to use it in a similar way as the taxonomy handler, by passing a comma delimited list of values. But it turns out, you can indeed just pass an array of source ids:

$row->entity_reference_source[] = $entry_id;

During this effort, I took it upon myself to develop a small Drupal module that would allow users to explore the Channel data (Expression Engine's version of a content type) of an Expression Engine site. I call it Expression Engine Analyzer and have posted it on as a sandbox project. It's been a great tool to help self understand the source data better and proven to be an excellent visual aid with clients while we discuss the the migration work. The whole team wishes we had such a tool at the beginning of a project, and going forward, I may consider making similar tools at the beginning of other migration efforts.