Getting to Know Your Migration

Series Overview & ToC | Next Article

SERIES INTRODUCTION

Today, we are excited to launch a detailed blog series that serves as an in-depth guide on utilizing the Drupal 10 Migrate API. This series is designed to equip you with thorough explanations and step-by-step guidance for migrating all your data from a Drupal 7 site to Drupal 10, facilitating a smooth upgrade to the most current version of the platform.

As we progress, we will introduce a DDEV-based development environment, complete with instructions to set it up. This environment is prepared with a Drupal 7 site already installed and a Drupal 10 site set as the migration target. Through the course of this series, we will walk you through the entire data migration process, covering a wide range of topics along the way, empowering you to migrate your own sites confidently.

The Migrate API, created and co-maintained by Tag1 team members, is a flexible and powerful system that can be used to collect data from multiple sources and import it into Drupal. While migrating data from Drupal 7 to modern versions of Drupal is a common use case, it is just one of many supported scenarios. In fact, the Migrate API is a complete extract, transform, and load (ETL) framework that can be used to import data from CSV files into your site, among many other scenarios.

Many years ago, I worked on a Drupal 6 to 9 data migration that used three different sources: the Drupal 6 site, a Drupal 9 site used for content staging, and multiple CSV files, which contained multilingual content. That data migration is an example of how flexible the Migrate API is. While the focus of this series is the migration of data from a Drupal 7 site, any general knowledge you have about the API and ETL frameworks will prove useful. For more information, I suggest you read another series of articles I wrote on the topic: 31 Days of Drupal Migrations.

Before a deep dive into the technicalities of performing a data migration, it is crucial to understand your current site. In this blog post, we will take a step back and discuss the different components of a Drupal site. This will help you evaluate different approaches and tools that can be utilized for the migration. To keep things simple, we will not touch on additional services like search, distributed content storage, caching layers, or proxies. Instead, we will focus on the Drupal application layer:

• Configuration: this determines the structure of the site. It includes all the different content types and the fields attached to them, taxonomy vocabularies, menus, user roles and permissions, views, media types, commerce product types, field collection and paragraph types, and webforms. General site settings are also part of the configuration. For example, site name, slogan, active default and administrative themes, enabled modules, date and time formats, text formats and WYSIWYG profiles, image styles, and pathauto patterns for url aliases.

• Content: this is user-submitted data. Nodes, taxonomy terms, files (addressed in the next point), menu items, users, media items, products, and webform submissions are all considered content. A single page on the website can contain a variety of content pieces. For instance, you can have an article (node) that includes a featured image (file), tags (taxonomy terms) related to the content presented, and a reference to the author (user).

• Files: these are a special type of content in Drupal. They are assets uploaded by users to be managed and served by Drupal. For example, images, text documents, and spreadsheets. While videos can be uploaded directly to the site, they are usually uploaded to dedicated video hosting providers such as YouTube and Vimeo. After they are uploaded, the videos can be embedded within Drupal.

• Modules: they provide functionality to the website. Drupal core itself comes with dozens of modules. You can also provide more functionality by adding modules from the contributed repository or by writing custom ones. Modules mostly consist of PHP code.

• Themes: they control the appearance of the site. Fonts, colors, and responsive behavior are all responsibilities of the theme. Similar to modules, Drupal core ships with a handful of themes, and more available in the contributed repository. Most sites use custom themes that align with the branding of the organization. Themes mostly consist of PHP code, CSS, JavaScript, and assets, such as images.

Configuration is usually created by site builders and developers. Content and files are generally added by content editors and, when allowed, by site visitors. Modules and themes are mostly the responsibility of backend and frontend developers, respectively.

Another aspect that is important to understand is where each of these site components live. Configuration and content are stored in the database. File references are stored in the database, but the assets themselves live in the file system. Modules and themes live in the file system.

Now, we can consider three different approaches for migrating to Drupal 10:

• Manually create configuration and content in the new site. This is a valid approach for small sites. If the site structure is simple and there is not much content, installing a fresh Drupal 10 site and recreating everything from scratch is achievable.

• Manually create the configuration and automatically migrate the content. This is the most common approach I have seen over the years. Organizations usually take the opportunity of migrating to Drupal 10 to make improvements and remove things that are no longer needed on their site. This is often reflected in changes to the content model and general site structure. For this approach, site builders and developers manually create the new site configuration per the new requirements. Then, content is moved into the new site structure.

• Automatically migrate configuration and content. With Drupal 7’s end of life approaching, I have seen more organizations take this approach. If little or no configuration changes are required on the new site, you can automate the migration of the current configuration. If minor tweaks are needed, those can be handled manually on the new site. Once the content model is in place, the content is moved over.

Notice that in the three scenarios, the focus is on configuration and content. That is because the Migrate API is designed to help automate the migration of those two components only. Technically speaking, files are also considered content in Drupal. As such, they are also supported, but neither modules nor themes are part of the scope of the Migrate API.

If your site uses contributed modules and themes, you must verify that they have Drupal 10 compatible versions. If not, you can look for alternative projects that serve as replacements or offer similar functionality. For instance, the Drupal 10 entityqueue module is the successor of the Drupal 7 nodequeue module. If no replacement or alternative is found, consider help porting the module or theme to Drupal 10. As a last resort, you can write custom code for any required feature that is otherwise not readily available.

The code architecture in Drupal 10 is significantly different than that of Drupal 7. Because of this, any custom module or theme needs to be recreated. In the case of modules, you can use the Module Upgrader to get started in the porting process and finish with manual changes. As for themes, there are no tools that assist with the porting. Before writing any custom code, consider your alternatives. In many situations, that which required custom code before could be covered by new functionality added to Drupal core or contributed modules/themes. Alternatively, some features may no longer be needed and can be dropped from the migration.

Note: Matt Glaman has released a project named Retrofit. It aims to run Drupal 7 code on a Drupal 10 site without having to make modifications to it. While I have not tested it myself, Matt has an impressive track record of impactful contributions to Drupal. He also contributes to PHPStan, Drupal Rector, Upgrade Status, and more.

Before closing, it is appropriate to acknowledge that while the Migrate API is the most commonly used method, it is one of many ways to move data into Drupal. Another popular approach is the Feeds module and its ecosystem. There is also a module that uses the Feeds user interface on top of the Migrate API. Another option is to write custom code to get data into Drupal. No matter the approach, I highly recommend that when importing data, Drupal APIs are used instead of direct writes to database tables. Drupal is a flexible and complex system. When a piece of content is created it will likely end up in multiple tables. When using Drupal APIs, access controls, validation, caching, and other systems get a chance to act upon the data that is being imported. Writing to database tables directly bypasses a lot of the safety checks that Drupal implements.

In our next post, we will explain how to audit a Drupal 7 site in preparation for a Drupal 10 upgrade. We will also explain some assumptions and limitations in the Migrate API and how to work around them. And, right before diving into the hands-on exercises, we will explain a key concept in data migration projects: Drupal entities. Stay tuned!

Series Overview & ToC | Next Article

Image by Gerd Altmann from Pixabay