calamares/src/modules/README.md

Calamares modules

Calamares modules are plugins that provide features like installer pages, batch jobs, etc. An installer page (visible to the user) is called a "view", while other modules are "jobs".

Each Calamares module lives in its own directory.

All modules are installed in $DESTDIR/lib/calamares/modules.

There are two types of Calamares module:

• viewmodule, for user-visible modules. These use C++ and QWidgets or QML
• jobmodule, for not-user-visible modules. These may be done in C++, Python, or as external processes.

A viewmodule exposes a UI to the user.

There are three interfaces for Calamares modules:

• qtplugin (viewmodules, jobmodules),
• python (jobmodules only),
• process (jobmodules only, not recommended).

Module directory

Each Calamares module lives in its own directory. The contents of the directory depend on the interface and type of the module.

Module descriptor

A Calamares module must have a module descriptor file, named module.desc. For C++ (qtplugin) modules using CMake as a build- system and using the calamares_add_plugin() function -- this is the recommended way to create such modules -- the module descriptor file is optional, since it can be generated by the build system. For other module interfaces, the module descriptor file is required.

The module descriptor file must be placed in the module's directory. The module descriptor file is a YAML 1.2 document which defines the module's name, type, interface and possibly other properties. The name of the module as defined in module.desc must be the same as the name of the module's directory.

Module descriptors must have the following keys:

• name (an identifier; must be the same as the directory name)
• type ("job" or "view")
• interface (see below for the different interfaces; generally we refer to the kinds of modules by their interface)

Module descriptors for C++ modules may have the following key:

• load (the name of the shared library to load; if empty, uses a standard library name derived from the module name)

Module descriptors for Python modules must have the following key:

• script (the name of the Python script to load, nearly always main.py)

Module descriptors for process modules must have the following key:

• command (the command to run) Module descriptors for process modules may have the following keys:
• timeout (how long, in seconds, to wait for the command to run)
• chroot (if true, run the command in the target system rather than the host)

Module descriptors may have the following keys:

• emergency (a boolean value, set to true to mark the module as an emergency module)
• noconfig (a boolean value, set to true to state that the module has no configuration file; defaults to false)
• requiredModules (a list of modules which are required for this module to operate properly)
• weight (a relative module weight, used to scale progress reporting)

Required Modules

A module may list zero (if it has no requirements) or more modules by name. As modules are loaded from the global sequence in settings.conf, each module is checked that all of the modules it requires are already loaded before it. This ensures that if a module needs another one to fill in globalstorage keys, that happens before it needs those keys.

Emergency Modules

Only C++ modules and job modules may be emergency modules. If, during an exec step in the sequence, a module fails, installation as a whole fails and the install is aborted. If there are emergency modules in the same exec block, those will be executed before the installation is aborted. Non-emergency modules are not executed.

If an emergency-module fails while processing emergency-modules for another failed module, that failure is ignored and emergency-module processing continues.

Use the EMERGENCY keyword in the CMake description of a C++ module to generate a suitable module.desc.

A module that is marked as an emergency module in its module.desc must also set the emergency key to true in its configuration file (see below). If it does not, the module is not considered to be an emergency module after all (this is so that you can have modules that have several instances, only some of which are actually needed for emergencies).

Module-specific configuration

A Calamares module may read a module configuration file, named <modulename>.conf. If such a file is present in the module's directory, it can be shipped as a default configuration file. This only happens if the CMake-time option INSTALL_CONFIG is on.

Modules that have noconfig set to true will not attempt to read a configuration file, and will not warn that one is missing; conversely if noconfig is set to false (or is missing, since the default value is false) if there is no configuration file, a warning is printed during Calamares start-up.

The sample configuration files may work and may be suitable for your distribution, but no guarantee is given about their stability beyond syntactic correctness.

The module configuration file, if it exists, is a YAML 1.2 document which contains a YAML map of anything.

All sample module configuration files are installed in $DESTDIR/share/calamares/modules but can be overridden by files with the same name placed manually (or by the packager) in /etc/calamares/modules.

Module Weights

During the exec phase of an installation, where jobs are run and things happen to the target system, there is a running progress bar. It goes from 0% to 100% while all of the jobs for that exec phase are run. Generally, one module creates on job, but this varies a little (e.g. the partition module can spawn a whole bunch of jobs to deal with each disk, and the users module has separate jobs for the regular user and the root user).

By default, modules all "weigh" the same, and each job is equal. A typical installation has about 30 modules in the exec phase, so there may be 40 jobs or so: each job represents 2.5% of the overall progress of the installation.

The consequence is that the unpackfs module, which needs to write a few hundred MB to disk, gets 2.5% of the progress, and the machineid module, which is essentially instantaneous, also gets 2.5% of the progress. This makes progress reporting seem weird and uneven, and suggests to users that Calamares may be "hanging" during the unpackfs stage.

A module may be assigned a different "weight" in the module.desc file (or via the CMake macros for adding plugins). This gives the module more space in the overall progress: for instance, the unpackfs module now has a weight of 12, so (assuming there are 38 modules in the exec phase with a weight of 1, and unpackfs with a weight of 12) regular modules get 2% (1 in 50 total weight) of the overall progress bar, and the unpackfs module gets 24% (12 in 50). While this doesn't speed anything up, it does make the progress in the unpackfs module more visible.

It is also possible to set a weight on a specific module instance, which can be done in settings.conf. This overrides any weight set in the module descriptor.

C++ modules

Type: viewmodule, jobmodule Interface: qtplugin

Currently the recommended way to write a module which exposes one or more installer pages (viewmodule) is through a C++ and Qt plugin. Viewmodules must implement Calamares::ViewStep. They can also implement Calamares::Job to provide jobs.

To add a Qt plugin module, put it in a subdirectory and make sure it has a CMakeLists.txt with a calamares_add_plugin call. It will be picked up automatically by our CMake magic. The module.desc file is not recommended: nearly all cases can be described in CMake.

Modules can be tested with the loadmodule testing executable in the build directory. See the section on testing modules for more details.

C++ Jobmodule

TODO: this needs documentation

C++ Widgets Viewmodule

TODO: this needs documentation

C++ QML Viewmodule

A QML Viewmodule (or view step) puts much of the UI work in one or more QML files; the files may be loaded from the branding directory or compiled into the module. Which QML is used depends on the deployment and the configuration files for Calamares.

Explicit properties

The QML can access data from the C++ framework though properties exposed to QML. There are two libraries that need to be imported explicitly:

import io.calamares.core 1.0
import io.calamares.ui 1.0

The ui library contains the Branding object, which corresponds to the branding information set through branding.desc. The Branding class (in src/libcalamaresui/Branding.h offers a QObject-property based API, where the most important functions are string() and the convenience functions versionedName() and similar.

The core library contains both ViewManager, which handles overall progress through the application, and Global, which holds global storage information. Both objects have an extensive API. The ViewManager can behave as a model for list views and the like.

These explicit properties from libraries are shared across all the QML modules (for global storage that goes without saying: it is the mechanism to share information with other modules).

Implicit properties

Each module also has an implicit context property available to it. No import is needed. The context property config (note lower case) holds the Config object for the module.

The Config object is the bridge between C++ and QML.

A Config object must inherit QObject and should expose, as Q_PROPERTY, all of the relevant configuration information for the module instance. The general description how to do that is available in the Qt documentation.

Python modules

Modules may use one of the python interfaces, which may be present in a Calamares installation (but also may not be). These modules must have a module.desc file. The Python script must implement the Python jobmodule interface.

To add a Python or process jobmodule, put it in a subdirectory and make sure it has a module.desc. It will be picked up automatically by our CMake magic. For all kinds of Python jobs, the key script must be set to the name of the main python file for the job. This is almost universally main.py.

CMakeLists.txt is not used for Python and process jobmodules.

Calamares offers a Python API for module developers, the core Calamares functionality is exposed as libcalamares.job for job data, libcalamares.globalstorage for shared data and libcalamares.utils for generic utility functions. Documentation is inline.

All code in Python job modules must obey PEP8, the only exception are libcalamares.globalstorage keys, which should always be camelCaseWithLowerCaseInitial to match the C++ identifier convention.

Modules can be tested with the loadmodule testing executable in the build directory. See the section on testing modules for more details.

Python Jobmodule

Type: jobmodule Interface: python

A Python jobmodule is a Python program which imports libcalamares and has a function run() as entry point. The function run() must return None if everything went well, or a tuple (str,str) with an error message and description if something went wrong.

Python API

TODO: this needs documentation

PythonQt modules (deprecated)

Type: viewmodule, jobmodule Interface: pythonqt

The PythonQt modules are deprecated and will be removed in Calamares 3.3. Their documentation is also almost completely lacking.

Process modules

Use of this kind of module is not recommended.

Type: jobmodule Interface: process

A process jobmodule runs a (single) command. The interface is process, while the module type must be job or jobmodule.

The module-descriptor key command should have a string as value, which is passed to the shell -- remember to quote it properly. It is generally recommended to use a shellprocess job module instead (less configuration, easier to have multiple instances).

Testing Modules

For testing purposes there is an executable loadmodule which is built, but not installed. It can be found in the build directory. The loadmodule executable behaves like single-module Calamares: it loads global configuration, job configuration, and then runs a single module which may be a C++ module or a Python module, a Job or a ViewModule.

The same application can also be used to test translations, branding, and slideshows, without starting up a whole Calamares each time. It is possible to run multiple loadmodule executables at the same time (Calamares tries to enforce that it runs only once).

The following arguments can be used with loadmodule (there are more; run loadmodule --help for a complete list):

• --global takes a filename and reads the file to provide data in global storage. The file must be YAML-formatted.
• --job takes a filename and reads that to provide the job configuration (e.g. the .conf file for the module).
• --ui runs a view module with a UI. Without this option, view modules are run as jobs, and most of them are not prepared for that, and will crash.