Most Drupal 7 sites keep the crossing to modern Drupal parked for some day, and some day has a habit of never arriving, because the crossing as we all learned it wants a second machine, a dump shipped across, credentials pasted into one more form and a rehearsal budget which runs out after the first try. On a BOA server the two ends sit on one account: the Drupal 7 site keeps serving while a fresh Drupal CMS site goes up beside it, one Ægir task lends the new site read-only eyes on the old database (enforced by the database server itself), and Drupal's own migration engine pulls content, files and users across in place. Rehearsals are throwaway site tasks, the wizard says what will cross before it starts, and the cutover is a move onto your own build and two renames, the old site kept intact. The front end is still a rebuild, your theme, custom code and views; only the logistics go away. Capabilities rather than an enforced workflow, and nothing here picks your exit for you.
Drupal's configuration system has at its heart a key concept: the site owns the configuration. What that means is that after a module has provided install configuration, it no longer has any control over it. You, the site admin, can change that config or even delete it. The module is no longer involved.
This kept the configuration system simple (up to a point); after all, the development of this system was one of the harder parts of the lengthy and difficult Drupal 8 cycle ("The ConfigImporter class was written with a get this done and working attitude" wrote @alexpott on a core issue, and I think the same could be said of the configuration system as a whole).
Since then, the config transformation API was added, and various modules have been created that take advantage of this, such as Configuration Split and Config Merge. These still work in the paradigm of the site owning the configuration. But what if we could turn the clock back, just partially, to when modules could own config too?
The (different) old days
If you were around in the days of Drupal 7, you may remember the various default hooks. Modules such as Views and Flag exposed a hook which allowed modules to define configuration items. (These were not configuration in the modern sense, because there wasn't that concept at the time, but referring to them as such makes things simpler.)
These were thus a code-defined instance of a type of thing that could also be defined in the UI by site admins. A module could supply a default view, and rely on it always existing. A new release of a module could come with enhancements to a that view, and they would exist on the site as soon as the module code was updated to the new version.
Because these hooks were PHP code, it was also possible for configuration to dynamically depend on other aspects of the site. You could do something such as define a default flag for every node type.
The idea
For a long time, I've wondered if something similar could be possible with Drupal's configuration system: put simply, defining configuration in code. Or, as I'm calling it, fixed config.
Many modules, such as Drupal Commerce and LocalGov Drupal rely on configuration items which are essential to their functionality (what I call machinery). Drupal's paradigm of configuration being owned by the site means that a site admin can break or outright delete key parts of the module's functionality: a commerce system missing its cart view, or a directory system missing its node types and vocabularies.
There's a case for admins being able to enhance and tweak a module's machinery, but that leads to the problem of how to reconcile a site's changes with changes that come in a new version of a module. There are modules that aim to help with this, but configuration entities are complex data structures, and without in-depth knowledge of that structure, this is a difficult and painstaking task. Some modules take care of this in update hooks each time they want to change their machinery, but that also requires complicated work each time.
The final push to get me to work on this was the use case of Entity Pager: the maintainer of the Flippy module suggested merging the two modules. But while Entity Pager provides the means for site admins to create any pager for any entity type, Flippy's functionality is to automatically provide a pager for each node type on the site. It seemed to me that the way to achieve this feature on top of Entity Pager was to have a way for the Flippy module to define an entity pager view automatically for each node type. The same way that Drupal 7 default hooks could.
Could this be done within the modern Drupal config system? The module would be in charge, not the site. If the module's code updated and changed the definition of the config, then the site would pick up on that.
Developing the module
The basic requirement is to add config entity definitions that are coming from code.
I could see two ways of doing this: intercept the entity storage handlers, or intercept the config transformation API. I opted for the latter. Partly because working at the entity storage level would require decorating every config entity type storage handler, which seemed heavy-handed, and because working within the config system has its advantages, and finally because one general principle I have found over the years is that complexity should generally be pushed down in a system. The config system is beneath the entity system, so working there would mean that this would be completely invisible to the entity system which would just see some additional entities.
So, I experimented with the config storage transformation API. Could I make it see, for example, a hardcoded node type? The answer was yes: it's simple to add an additional config entity in the STORAGE_TRANSFORM_IMPORT event, and the site sees it just as if it were some other piece of config.
Then the real development work started: making the config system read the fixed config, but also ignore it when necessary. I decided on this simple rule:
- Fixed config is not exported to config sync. Instead, it is always re-created from files on config import.
On Drupal 7, some (but not all) default hooks had a concept of overriding. The site admin could choose to edit a default object, such as a view, and deviate from the version in code. From that point on, the site owned that object, not the module.
I've decided, for now at least, not to support overriding. I think a better, and simpler, pattern is to allow selection of config: the module provides machinery, but also provides a setting to select it as being in use. For our example of the commerce cart view, there would be the fixed config view, and also a config setting where you select which view is used for the cart. This means that the default view remains under the control of the commerce module, but a site admin can duplicate it, change the duplicate, and then use that one instead. The site then owns the duplicate view, but the default view remains available and functional at all times.
The Alpha-1 API
Having got a proof-of-concept, the next step was to design an API too. Rather than use hooks or an event, I decided fixed config should look the same as install config: YAML files. The big advantage here being that while developing it, you can move YAML files from sync or install folders to become fixed config. And it provides a system that is already familiar to developers: fixed config is simply a YAML file in a module's config/fixed folder instead of config/install. (I made a small change and a new release of Config Devel so that its module config export feature works with this folder too.)
But static YAML files doesn't cover all the use cases that the Drupal 7 era hooks used to satisfy. So I added the concept of derivers. We already have plugin derivers in Drupal (and maybe I should have tried to think of a different name); but these are config deriver plugins: they create multiple config items from a single template.
So while a static fixed config file looks identical to an install config file, a derived fixed config file has these additions:
third_party_settings:
fixed_config:
deriver: deriver_plugin_id
dependee_config_patterns:
- node.type.*
What this means is that when config whose name matches node.type.* is created or updated, the fixed config deriver plugin deriver_plugin_id reacts, and maintains the fixed config based on the YAML file that holds these properties.
The node type config entity is the dependee config; the fixed config that gets defined in response to that is the dependent fixed config.
This is how the Flippy module could then work: it defines a single fixed config YAML file for a view, and a deriver plugin which alters these config values:
- Sets the value of the node type filter.
- Sets the path for the page display.
Eating my own dogfood
I find that you never fully realise what an API needs to do or how it needs to work and how to best make it usable until you actually try to use it, and that therefore it's best to do that early on.
I started making the Entity Pager view, and I was soon struck with several things about what I'd made:
- Code that sets a view's entity bundle filters to an incoming bundle entity should be reusable.
- Code that adds the incoming entity's ID as a suffix to the view's path should be reusable, but not necessarily by the same things as the first.
- As well as deriving a single view for each node type, it might be useful to have a single view which adds a new display for each node type.
That reusability feature could be solved by using PHP traits, and requiring developers to composite them into their plugin classes. Or it could be solved by making the plugins themselves smaller, and allowing the deriving process to specify more than one. A pipeline, basically.
And the many-to-one feature could be solved by letting code alter the fixed config YAML file values, rather than deriving from them: the same config values get processed by the PHP code for every config entity it listens for, rather than making a new copy each time.
To me, this suggests a new type of plugin, fixed config alterers, which can do the work in both of these scenarios.
So what I am pondering now would look like this:
- For derived fixed config:
- The fixed config deriver plugin does the bare minimum to the config values from the YAML file.
- The config values are passed through multiple alterer plugins.
- For static fixed config:
- If config patterns are specified, the config values are passed through alterer plugins for each config matching the pattern.
- A better name is needed as it would no longer be completely static!
Am I over-engineering, I wonder? Also, the pipeline of alterers seems very much like the Migrate API's pipeline of processors, but I don't see how to reuse those as they operate on different sorts of values: entire config entity values for my case, and single content entity field values for Migrate.
I'd be interested in hearing ideas and use cases for the fixed config module. It would help me with the process of refining my initial ideas for the API.
I've made an alpha release, try it out and let me know what you think, either in the issue queue or on mastodon.
Do you have potential uses for fixed configuration? I'd love to work on this module with some real-world use cases, and I'm available for hire - contact me!