On a recent Thursday afternoon, Gabor asked if I was going to pick back up on me efforts of checking Drupal 9 module readiness. That got me a mention in the Driesnote last year and to be honest is my favorite claim to fame in any project. I had not thought about reviving it though. I mean people are tracking this stuff I assumed, but independent validation of notions is a really good thing to have. So, I said to myself, “that would be a good idea.”
Gabor, if you don’t know, has been diligently working on getting us all ready for that June 3rd launch date of the next major version of Drupal 9, which has a lot of anticipation around it. It will be the first major release to not require a rebuild for site owners, just an update. I think that is pretty awesome given the technical sophistication that is Drupal at this time in history. He has been the driving contributor for the upgrade_status module development. TL;DR, this module lets you know what to fix in your modules to be D9 ready!
Basically, the idea is to run this checking tool against all the modules that I can install with the command line (more on that later) and spit out consumable reports we can do analysis on.
So, I did what anyone about to embark on a quest for computer science knowledge would do, I did a fresh install of Ubuntu and started setting up my environment to make Drupal work. Why a fresh install? When was the last time you installed a minimal operating system and just pulled down only what you needed from the interwebs, as you need it? So snappy, so clean. If you have not taken Ubuntu 19 out for a spin, well, you are missing out on a great OS experience. Also, doing as much tinkering and sampling as I do of various repos and settings, a fresh Ubuntu install would make it easier to debug, and boy howdy did I need to debug as I went.
Getting Drupal working was super fast and easy because I had the perfect pairing of tools to get me going: Composer and the, completely new and amazing to me, Drupal 8 Quick Start Command . Literally 5 minutes from the time I installed Composer, I had a working copy of Drupal 8.8.5 installed and running. This is a demo setup but you can get to hacking on it immediately. Served my purposes as I ran it more or less nonstop for a few days and it didn’t fall over on its own, though you can push it over with a wrong move or two.
A quick note about Drupal always assuming you want to immediately pop it into production. I get it, this saves a lot of devops hassles to make the options assume you don’t want people messing with things, but we are on a local box explicitly messing with things, so we need to run chmod u+w web/sites/default in order to give the files permission to actually let us Composer install stuff.
One of the very, very recent updates (as in hours before I agreed to do this) Gabor has been working on and what he asked me to investigate was the modification of the Drush command upgrade_status:analyze to the command upgrade_status:checkstyle. It runs the same check, just outputs the results to XML format. So I needed Drush and Composer let me require drush/drush which gets me Drush 10 automagically. If you have not seen the generate feature of it yet, here is a video I did about it in Drush 9, not much has changed.
The script itself is pretty short and simple without all my inline commenting, basically
1) load in a CSV of module names,
2) loop through it, composer require modules one at a time,
3) runupgrade_status:checkstyle and redirect the output into a file with the module name and
4) uninstall the module before beginning the loop again.
When I did the ‘check all’ project last time, I used “drupal-check”, a PHPStan based project for static code analysis. I didn’t need a working install, just a code repo. Using a running Drupal site makes things a little stranger.
Before, I was able to leverage Git to just flip between branches as I went, configuration was not part of the mix. Also upgrade_status needs the module to be enabled. Drush lets me do those things easily. git reset --hard should have let me just flush all the code changes, but something interesting happened, and I think it was due in part to Drupal caching, the code stayed put. At least parts of it. I eventually went so far as to composer remove each module, then rm -rf each module folder. Seems like overkill but it worked.
It take approx 45 seconds per cycle, but it worked. 45 seconds x ~6000 modules = 75 hours of script run time to do them all. So naturally I reached out for help to the internet for help scaling the process. Very fortunate for me Arrow a.k.a Aaron Feledy, master of all things Lando and ccjjmartin a.k.a Christopher Martin immediately jumped on board this train.
Aaron built the awesomeness that is the Lando recipe for this setup, which you can find in the repo. Chris dedicated some time and machine resources to make multiple Lando instances run as well as made tweaks and suggestions to everything along the way. This would have taken me a full week of futzing with things on a single, underpowered laptop. Instead I got to hang out with some amazing Drupal folks for a couple days online and produce the final results.
And the results are….not finished running as of the time of this writing. About another 2000 or so modules to check, but soon this will be public. I just wanted to write this up while it was fresh in my brain. I love me some Bash and Drupal. I am so lucky to have found this community. Looking forward to a grand future with Drupal 9.
If you want to run this yourself or just see what we are working with, check it here: https://github.com/mcdwayne/d9-upgrade_status-project