Drupal 8 Data Migration Planning

Background

When we move the Drupal 7 workflow information from the old site to the new site we need to keep the information intact and accurate. To do this, we need a plan which will account for:

Consideration

Challenges

Ideas

Consideration

Challenges

Ideas

User Accounts Should Be The Same

 

Date / Times Should Be The Same

  • We can’t hand enter this data, as it will have different dates/times.

  • Is there a migration tool for this?

  • Can we do this with direct SQL?

Workflow Action Notes Should Be The Same

  • We should ensure these get brought across to the new website.

  • If we can figure out the Date / Times, we should be able to figure out this, right?

If we import the data into the new site, but continue to use the old site for a bit longer, we should also have a plan to keep the audit/decision information synced between the two.

  • How can we sync these?

 

If direct drupal Migration is not an option

  • This has proven to be the case, as the Workflow module in D8 is in alpha. The alpha release has all functionality of the original module, however it does not have the migration functionality implemented

  • We need to determine possible ways of migrating:

    • The workflows

    • And the data (all the things listed above)

 

Comparison of Migration Options

 

Option 1: Find Script on Internet

Option 2: Write a script ourselves using Drupal Migration Code

Option 3: Write a script ourselves using direct database calls

Option 4: Hand migrate

 

Option 1: Find Script on Internet

Option 2: Write a script ourselves using Drupal Migration Code

Option 3: Write a script ourselves using direct database calls

Option 4: Hand migrate

Description

In this option, we have tried a related option that is available on the internet and takes the idea if someone already did migration in order to investigate.

In this option, We have to study internal architecture how the workflow, rule modules module functional in the Drupal 8 sites, and based on those inputs we need to create a custom module that will migrate the data from drupal 7 to drupal 8.

Similar to the option 2, but using direct database calls instead of writing php code to perform the migrations.

In this option, We have to migrate workflow rules from Drupal 8 administrator by taking the reference which we had on the Drupal 7 site.

We will also need to hand migrate the historical Access and Publishing Requests that are contained within the current system. This would be done through the UI interface by replicating the original not by copy the data through the database.

Workflow Rules

Workflow included 12 states and rule is having 14 configurational settings

Workflow included 12 states and rule is having 14 configurational settings

Workflow included 12 states and rule is having 14 configurational settings

Workflow included 12 states and rule is having 14 configurational settings

Pros

  • Time-Saving approach

  • Easy to implement

  • A custom module can be used on multiple sites.

  • Already have some familiarity with the APIs and hooks provided by Drupal 8, which lowers the overhead of research costs

  • The IDE will also provide IntelliSense when working with objects provided by the modular creator (lowers research costs)

 

  • Time-Saving approach

  • Easy to implement

Cons

  • Very less information available and haven't found any successful workaround regarding the same.

  • Time-consuming approach

  • The Workflow modules specific APIs and hooks need to be researched and tested

  • Time-consuming approach

  • We will have to also understand if there were any value/data definition changes within the new database to correctly apply the transformations.

  • Workflow states and rules condition should be accurate. otherwise, functionality will not work as excepted.

Data

  • Drupal site is having 2 workflows i.e “API Publish” and “Application Approval Request” which will have their certain state respectively.

  • Workflow Rule is having 14 rules based on a certain event condition.

  • Drupal site is having 2 workflows i.e “API Publish” and “Application Approval Request” which will have their certain state respectively.

  • Workflow Rule is having 14 rules based on a certain event condition.

  • D7 Workflow data will be residing under the below-mentioned tables.

workflow_node workflow_node_history workflow_scheduled_transition workflow_states workflow_transitions workflow_type_map workflows
  • D8 Workflow data will be residing under the below-mentioned tables.

config config_snapshot workflow_transition_schedule workflow_transition_history
  • Drupal site is having 2 workflows i.e “API Publish” and “Application Approval Request” which will have their certain state respectively.

  • Workflow Rule is having 14 rules based on a certain event condition.

Pros

 

  • Already have some familiarity with the APIs and hooks provided by Drupal 8, which lowers the overhead of research costs

  • The data storage logic has already been written by the module owner, so we won’t make mistakes by going on this approach (lower risk)

  • The IDE will also provide IntelliSense when working with objects provided by the modular creator (lowers research costs)

 

  • Already having configuration settings (state & transition) in Drupal 8 site that easy to port into the Drupal 8.

Cons

 

 

  • We will need to research the difference between the D7 and D8 database schemas.

  • We will have to also understand if there were any value/data definition changes within the new database to correctly apply the transformations.

  • Reproducing the accurate Dates and Times of the original approvals (who they were done by and when) could not be replicated in this approach. (There would be data loss, or data distortion.)

Notes

On the basis of research on the internet, there is no direct way available to migrate workflow and rules directly from Drupal 7 to Drupal 8.
Here is the list of the POC:-
1: I have tried migration through the Drush commands. These steps are given in the doc Migration Process from Drupal 7 to 8

Result: Failed, I have tried to migrate workflow and rules data through the brush but the process is not supported to migrate the data from D7 to D8.

Useful Link: https://www.drupal.org/docs/upgrading-drupal/upgrade-using-drush

2: Tried to import workflow state by adding code and followed steps mentioned here in this link https://www.drupal.org/docs/upgrading-drupal/upgrade-using-drush But there is not success found with that.

Result: Failed, As node was not imported to the D8.

3: In order to migrate rule from D7 to D8, I have used a few modules with providing missing functionality in Drupal 8. Here is the list of those modules.

  • D8 Rule Essentials

  • Typed Data

Result: Failed

4: I have tried to apply the patch which is having on Drupal community https://www.drupal.org/files/issues/2020-09-24/workflow-add_migrate_support_for_the_workflow_system_on_d8-3015900-3-D8.patch

Result: Failed

 

On the basis of the POC on this approach, the D7 and D8 database scheme has been changed significantly.

In drupal 7, workflow module with a separate table w.r.t their functionality module. whereas in Drupal 8, it will use a common table except for workflow history and transition.

 

 

On the basis of the research that we can make the workflow settings manually which included state and transition and we will import the history by using a script that will get the data from the Drupal 7 site.

 

 

 

 

Outstanding Questions

Oct 22, 2020

  • (Not a question) Currently, we have a feeling that a possible approach would be to move the Workflow Rules by hand between the D7 and D8 systems and move the Data using a sql script or php code; this would be a split approach.

  • What research needs to be done to determine how to import existing API Request and API Publishing documents from D7 to D8 such that they will be associated with the workflow system?

  • What research needs to be done to determine how to use a sql script or php code to transfer the audit history (ie. workflow_transition_history/schedule tables) from D7 to D8?

  • When the research is complete, will we be able to do a final “fresh” migration of the data? Can we repeat that process later as well?

Steps to Perform the Migration

Below mentioned steps need to be followed in order to work on workflow migration from D7 to D8.

Step 1: Firstly installed the Workflow Rule module on the Drupal 8 website. Apart from the workflow rule module we also need to install few more modules listed below to make that working with all mandatory functionality.

  • Content Moderation

  • Workflows Field 

  • D8 Rules Essentials

  • Typed Data

  • Content Access

Step 2: Check workflow states and state transitions from the D7 site and follow the step mentioned on this link https://www.drupal.org/docs/8/core/modules/workflows/overview to add the states and transition on Drupal 8 website.

 

Step 3: Role permission for transition state,