Module Masquerade
Last week during Flock to Fedora, we had a discussion about what is needed to build a module outside of the Fedora infrastructure (such as through COPR or OBS). I had some thoughts on this and so I decided to perform a few experiments to see if I could write up a set of instructions for building standalone modules.
To be clear, the following is not a supported way to build modules, but it does work and covers most of the bases.
Step 1: Creating module-compatible RPMs
RPMs built as part of a module within Fedora’s Module Build Service are slightly different than RPMs built traditionally. In MBS, all RPMs built have an extra header injected into them: ModularityLabel. This header contains information about what module the RPM belongs to and is intended to help DNF avoid situations where an update transaction would attempt to replace a modular RPM with a non-modular one (due to a transient unavailability of the module metadata). This step may not be absolutely necessary in many cases. If you are trying to create a module from RPMs that you didn’t build, you can probably get away with skipping this step, provided that you don’t care if there might be unpredictable behavior if you encounter a broken repo mirror.
To create a module-compatible RPM, add the following line to your spec file for each binary RPM you are producing:
ModularityLabel: <arbitrary string>
Other than that new RPM label, you don’t need to do anything else. Just build your RPMs and then create a yum repository using the createrepo_c tool. The ModularityLabel can be any string at all. In Fedora, we have a convention to use name:stream:version:context to indicate from which build the RPM originally came from, but this is not to be relied upon. It may change at any time and it also may not be accurately reflective of the module in which it currently resides, due to component-reuse in the Module Build System.
Step 2: Converting the repo into a module
Now comes the complicated part: we need to construct the module metadata that matches the content you want in your module and then inject it into the yum repo you created above. This means that we need to generate the appropriate module metadata YAML for this repository first.
Fortunately, for this simple approach, we really only need to focus on a few bits of the module metadata specification. First, of course, we need to specify all of the required attributes: name, stream, version, context, summary, description and licenses. Then we need to look at what we want need for the artifacts, profiles and api sections.
Artifacts are fairly straightforward: you need to include the NEVRA of every package in the repository that you want to be exposed as part of the module stream. The NEVRA format is of the form examplepackage-0:0.1-5.x86_64.
Once the artifacts are all listed, you can decide if you want to create one or more profiles and if you want to identify the public API of the module.
It is always recommended to check your work with the modulemd-validator binary included in the libmodulemd package. It will let you know if you have missed anything that will break the format.
Shortcut
While drafting this walkthrough, I ended up writing a fairly simple python3 tool called repo2module. You can run this tool against a repository created as in Step 1 and it will output most of what you need for module metadata. It defaults to including everything in the api section and also creating a default profile called everything that includes all of the RPMs in the module.
Step 3: Injecting the module metadata into the repository
Once the module metadata is ready for inclusion, it can be copied into the repository from Step 1 using the following command:
modifyrepo_c --mdtype=modules modules.yaml /path/to/repodata
With that done, add your repository to your DNF/Yum configuration (or merge it into a bigger repository with mergerepo_c, provided you have version 0.13.2 or later) and run dnf module list and you should see your new module there!
Edit 2019-08-16: Modified the section on ModularityLabel to recognize that there is no defined syntax and that any string may be used.
Stephen Gallagher's Open-Source Blog 