Future-proofing your module (and dependencies) with GitLab CI

Since Drupal 8, there has been a mostly smooth migration path between major versions, so you can upgrade from 8 to 9 to 10 relatively easily.

One bump in the road can be the availability of modules for the new version. Drupal 8 and 9 were only supported for a year after the release of their successors, which didn’t help. While quite a lot of modules had a release supporting the next version quickly, there seemed to be quite a few that were very slow to update, and quite a few sites were waiting for one or two modules to release compatible versions before being able to upgrade. I think there are still many sites running on Drupal 9, which is out of support (over 150,000 of them, actually).

Drupal 10 will remain under support for the full life of Drupal 11, which may take some of the pressure to upgrade away. However, it could also give module maintainers an excuse to delay their release.

Wouldn’t it be nice if modules could automatically be ready for the next release?

Well, with the move to GitLab CI, they (almost) can!

The Drupal community is in the process of replacing the bespoke Drupal CI system with GitLab CI. This provides many improvements to test running. I’m quite pleased by how easy it is to add code quality checks to your module - though getting it to pass them may be a bigger challenge!

But the feature I want to talk about now is the ability to test against multiple versions of Drupal. By default, GitLab CI will run against the current Drupal release, but it makes it a simple matter to select different versions to run your tests against.

If your module still supports Drupal 9, you can have the tests run against that. You can also run against the previous and next minor releases of Drupal 10. Another useful option is to run against the current release, but using the highest available version of PHP.

But the main aim of this article is running against the next major release, currently Drupal 11.

For me, this is exciting because if modules start using it, they can assume they are ready for Drupal 11, and if anything gets deprecated that affects the module or the developer tries to introduce something that is not compatible with the next version, it will get flagged early.

Adding GitLab CI to your module

The CI is controlled by a file, .gitlab-ci.yml, which should be placed in the root directory of your module.

The Drupal Association has provided a standard template for GitLab CI, and in the vast majority of cases you should use it. It gives you a lot, and in most cases all you need to do is use the template.

To get started, go to your module’s GitLab page (scroll down your project page and pick “Source Code” from the sidebar).

From the root of your project, click the “+” icon and select “New file”:

On the “New file” page, enter .gitlab-ci.yml in the filename field, and suddenly a drop-down will appear, with the label “Apply a template”:

Select template.gitlab-ci. The new file will be shown:

This file contains mostly comments. You can just save it and and select “Build → Pipelines” from the sidebar, and watch it run.

You may get errors and your pipeline will fail, or you may get warnings, such as in the above example, which is warning of a phpcs issue. Drupal has quite strict coding standards rules, so you may need to do a bit of tidying up in your project to get all the green ticks.

The default pipeline works, but it only tests against the current Drupal version.

Every time you push to a merge request, your change will be tested. And when the merge request is merged, it will get run again to make sure the tests still pass.

What’s going on here?

If you look the .gitlab-ci.yml file, it’s mostly made up of comments.

When you strip those out, all that’s left is:

include:
  - project: $_GITLAB_TEMPLATES_REPO
    ref: $_GITLAB_TEMPLATES_REF
    file:
      - '/includes/include.drupalci.main.yml'
      - '/includes/include.drupalci.variables.yml'
      - '/includes/include.drupalci.workflows.yml'

The main thing left is the three includes, with two variables telling GitLab where to find them.

This means that everything is taken from the includes. If you want to know what’s going on there, it’s no secret, and you can find them and inspect them in the “gitlab_templates” repository.

The big advantage of doing it this way is that future improvements to the templates will automatically get used by your module, so it will always be using the latest CI version.

Make sure you check out the official documentation, as it almost certainly explains things I’ve forgotten.

Testing multiple versions

But we want more!

Edit your .gitlab-ci.yml file and add the following (there is a commented out variables section you can overwrite):

variables:
  # Broaden test coverage.
  OPT_IN_TEST_PREVIOUS_MAJOR: 1
  OPT_IN_TEST_MAX_PHP: 1
  OPT_IN_TEST_PREVIOUS_MINOR: 1
  OPT_IN_TEST_NEXT_MINOR: 1
  OPT_IN_TEST_NEXT_MAJOR: 1

Here’s an example of a merge test file like this one running:

As you can see, the tests are passing for past and future versions.

In this case, we’ve fixed our coding standards, but still have some static analysis issues from PhpStan.

And, again, because we are only overriding variables, your CI will always stay up to date. When Drupal 11 comes out, it will become the “phpunit” test, Drupal will still be tested as “previous major”, and tests will begin against Drupal 12 as “next major”.

For many modules, that’s the end of the story. You can enable the tests, and let GitLab do the testing for you.

The Dependencies Strike Back

But what if your module has dependencies?

In that case, things get a little bit more complicated!

This happened to me with the Smart Trim module, which has Token as a dependency, and “Token Filter” as a development dependency.

I enabled “next major” and the build step of the pipeline failed because the dependency modules weren’t ready for Drupal 11.

To make it work, I needed to use Matt Glaman’s lenient Composer extension.

Normally, you’d install it before fetching the modules in question with Composer. Unfortunately, that didn’t work here, because the dependencies are already in the composer.json file for the project.

After an awful lot of trial and error, this is what I came up with. It needs to be added to the .gitlab-ci.yml file. I put it below the variables section:

composer (next major):
  before_script:
    - composer global config --no-plugins allow-plugins.mglaman/composer-drupal-lenient true
    - composer global require mglaman/composer-drupal-lenient
    - composer config --merge --json extra.drupal-lenient.allowed-list '["drupal/token", "drupal/token_filter"]'

This adds a script that runs before the build step for next major only. Three Composer statements get run. The first is just to allow the composer extension to be installed. The second actually installs it, but it installs globally, to get around the problem of installing before the dependencies get installed. The final one adds config lines to allow the dependency modules to use the lenient extension, but note it adds it to the local configuration (that threw me for a while).

This got the build step to run, but it then failed on the test step. This is hardly surprising, since the modules specify ^10 in the core_version_requirement line of the .info.yml file, but don’t specify ^11.

So, we need to add something to the test step as well. This is what I came up:

phpunit (next major):
  before_script:
    - grep -q "\^11" web/modules/contrib/token/token.info.yml || (grep -q "\^10" web/modules/contrib/token/token.info.yml && sed -i "s/\^10/\^10 \|\| ^11/" web/modules/contrib/token/token.info.yml)
    - grep -q "\^11" web/modules/contrib/token_filter/token_filter.info.yml || (grep -q "\^10" web/modules/contrib/token_filter/token_filter.info.yml && sed -i "s/\^10/\^10 \|\| ^11/" web/modules/contrib/token_filter/token_filter.info.yml)

This was a bit of a handful, so I refined it slightly by moving the path to the info.yml files into variables to make it easier to follow:

phpunit (next major):
  before_script:
    - export TOKEN_INFO=web/modules/contrib/token/token.info.yml
    - grep -q "\^11" $TOKEN_INFO || (grep -q "\^10" $TOKEN_INFO && sed -i "s/\^10/\^10 \|\| ^11/" $TOKEN_INFO)
    - export TOKEN_FILTER_INFO=web/modules/contrib/token_filter/token_filter.info.yml
    - grep -q "\^11" $TOKEN_FILTER_INFO || (grep -q "\^10" $TOKEN_FILTER_INFO && sed -i "s/\^10/\^10 \|\| ^11/" $TOKEN_FILTER_INFO)

As before, this adds a script, this time running before the phpunit step for next major. For both dependant modules, it checks if the info.yml file contains ^11, and if it does, it leaves it alone, but if not it replaces the ^10 with ^10 || ^11. I was able to steal this from include.drupalci.main.yml, which does the same for the project’s own info.yml file.

With this added, the phpunit step is passing.

Adding these sections allows “next major” tests to run for modules with dependencies, though hopefully, future additions to the CI scripts will make this easier. It would be nice if you could override a variable with a list of modules, and let the CI do the rest.

Note: As I write this, there is an issue open in the GitLab Templates project that may simplify this process. If/when that proposal becomes part of the template, I’ll update here.

Is that it?

I think it would be naive to assume that your module is ready for Drupal 11 just because your tests aren’t failing.

If your test coverage is good, it should give you a good indication, but it’s always wise to double check with other tools.

Drupal Rector is useful for finding and fixing deprecations in your module. Personally, I find the easiest way to use it is through the Upgrade Status module, which gives you a friendly report listing any changes you need to make.

If this finds any deprecations, it’s a good idea to write a test that fails with the deprecated code before fixing. At least that way, you’re adding to your test coverage.

I don’t think you can ever completely eliminate manual testing (this is why you can only almost be ready automatically), so I would recommend installing the module under an actual Drupal 11 pre-release. I’d probably wait till it’s at least in beta to do this, so you’re testing against something fairly close to the final release. That’s expected to be in late March.

What about .info.yml?

One question to consider is when should you add || ^11 to your .info.yml file?

The CI will patch it in as part of the “next major” test, so it’s not strictly necessary to add it in yet.

However, personally I would add it early as a statement of intent.

It does mean you need to test regularly, but if you keep on top of anything raised, your module will be ready for Drupal 11 when it launches, with minimal effort from you.

Summing up

If you haven’t started using GitLab CI yet, I’d recommend enabling it as soon as possible.

The basic configuration is a great start, but when you start turning on the optional tests, its real power comes to light.

Testing against Drupal 11 early could be a real game changer if a lot of modules start using it. I think it will pay dividends when we see a lot of modules ready for the new Drupal version on launch day.

I hope your module will be one of them!

Thanks to AmyJune Hineline and Mile Anello for their help proofreading this article.