Migrate Plus
Provides extensions to Drupal core migration framework functionality with configuration entities, additional plugins, and examples.
migrate_plus
Install
composer require 'drupal/migrate_plus:^6.0'
Overview
Migrate Plus is a comprehensive extension module for Drupal's core Migrate framework. It provides essential enhancements including the ability to store migration configurations as configuration entities, allowing them to be flexibly loaded, modified, and exported through Drupal's configuration management system.
The module introduces migration groups for organizing related migrations and sharing common configuration. It provides three new plugin types: authentication plugins for securing HTTP requests, data fetcher plugins for retrieving content from various sources, and data parser plugins for processing JSON, XML, and SOAP data formats.
Additionally, Migrate Plus includes over 20 process plugins for transforming data during migration, source plugins for URL-based and SQL table data sources, and a destination plugin for writing directly to database tables. The module also provides an event system for object-oriented manipulation of source data before processing.
Features
- Configuration entity storage for migrations allowing YAML-based migration definitions that can be exported/imported via Drupal's config system
- Migration groups for organizing migrations and sharing common configuration across multiple migrations
- URL source plugin supporting file- and HTTP-based data retrieval with pluggable fetchers and parsers
- JSON, XML, and SOAP data parser plugins for parsing various data formats from remote sources
- HTTP authentication plugins including Basic, Digest, NTLM, and OAuth2 authentication methods
- Entity lookup and generate process plugins for matching and creating entities during migration
- Over 20 process plugins including string replacement, DOM manipulation, array operations, and conditional processing
- Table source and destination plugins for migrating data to/from arbitrary database tables
- PREPARE_ROW event for object-oriented manipulation of source data before processing begins
- Support for pagination in JSON sources with multiple pager types (urls, cursor, page, paginator)
Use Cases
Importing content from a JSON API
Use the 'url' source plugin with 'http' data fetcher and 'json' data parser to import content from REST APIs. Configure authentication if needed, specify the item_selector to locate data within the JSON structure, and map fields using the process pipeline. Supports pagination through various pager types including cursor-based and page-number-based pagination.
Migrating data from external XML feeds
Use the 'url' source with 'xml' or 'simple_xml' parser. The 'xml' parser uses XMLReader for memory-efficient streaming of large files, while 'simple_xml' provides full XPath support for smaller files. Configure item_selector using XPath-like syntax to identify elements to import.
Looking up and creating taxonomy terms during migration
Use the 'entity_lookup' process plugin to find existing terms by name, or 'entity_generate' to create terms that don't exist. Configure bundle_key and bundle for vocabulary matching. This is ideal for populating entity reference fields during node migration.
Migrating data between database tables
Use the 'table' source plugin to read from any SQL table and the 'table' destination plugin to write to tables not registered with Drupal's Schema API. Configure id_fields for tracking and fields for column mapping. Supports batch inserts for performance.
Processing HTML content during migration
Use the 'dom' process plugin to convert HTML strings to DOMDocument objects, then apply transformations using dom_remove, dom_select, dom_str_replace, and dom_apply_styles plugins. Export back to string using 'dom' with export method.
Sharing configuration across multiple migrations
Create a migration_group configuration entity with shared_configuration containing common settings like source database connection, authentication credentials, or default field mappings. All migrations in the group inherit this configuration.
Migrating files from blob data
Use the 'file_blob' process plugin when source data contains binary file content (like base64-encoded images). Decode the data first using callback plugin with base64_decode, then use file_blob to create files at specified URIs.
Conditional field migration
Use 'skip_on_value' to skip rows or fields based on source values. Use 'gate' for more complex conditional logic where a different field controls whether the current value passes through.
Reusable process pipelines
Create YAML files in module/migrations/process/ containing common process plugin chains. Reference them using the 'snippet' process plugin with module and path configuration. This promotes code reuse and maintainability.
Tips
- Use migration groups to organize related migrations and share common configuration like database connections or source URLs
- The 'entity_generate' plugin extends 'entity_lookup' - if you only need to find existing entities, use 'entity_lookup' for better performance
- For large XML files, use the 'xml' parser (XMLReader) instead of 'simple_xml' to avoid memory issues
- Subscribe to the MigrateEvents::PREPARE_ROW event for complex row manipulation instead of writing custom process plugins
- When debugging migrations, use 'drush migrate:import --idlist' to test specific rows
- The 'snippet' plugin helps maintain DRY principles by allowing process pipeline reuse across migrations
- Use 'access_check: false' on entity_lookup when migrating as a trusted user to avoid permission-related lookup failures
- Configure batch_size in table destination for improved performance when importing large datasets
- Use the transpose plugin to combine parallel arrays before sub_process when source data has related arrays
Technical Details
Hooks 2
hook_migration_plugins_alter
Implemented by migrate_plus to integrate shared group configuration into migrations, set default migration class, convert null idMap to empty array, and strip deriver prefix for config entity-based migrations.
hook_migrate_prepare_row
Implemented by migrate_plus to dispatch the PREPARE_ROW event, allowing object-oriented responses to row preparation.