- Using the assignment-operator just generates blank lines.
- Using QLog with a log-level avoids the cDebug()-style special
handling of warnings and errors (useless here, but may as well
fix code style).
When using the `rawfs` module for copying data, it may be useful to
save the source device used for later checks or actions. This commit
therefore adds a `source` field to each corresponding partition entry in
global storage, so that this information can be retrieved later during
the installation process.
Another small improvement is that global storage is now modified only
once (it was previously modified as many times as there were entries
processed by the `rawfs` module).
Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>
When choosing `systemd-boot` as the bootloader, numerous problems
occurred:
- the kernel and initrd were not copied to the EFI System Partition,
and therefore could not be reached by the bootloader
- the fallback entry used the default initramfs image instead of the
fallback image
`systemd-boot` provides the `kernel-install` utility, which
automatically copies the kernel + initramfs to the EFI partition, and
creates the corresponding bootloader entry.
Unfortunately, `kernel-install` cannot be used here as the module is not
executed in a chroot. As setting up one only for running a single
command would be overkill, this patch re-creates what `kernel-install`
usually does:
- copy the kernel and initramfs to their own subdirectory at the root of
the EFI partition
- create the corresponding entry configuration file
To this end, the `systemd-boot` installation code in the `bootloader`
module has been largely refactored, including removing a few duplicate
LOCs.
Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>
In its current state, the 'rawfs' module requires the source partition
to be identified either by its mount point or device name, but using a
symlink to either one (e.g '/dev/disk/by-uuid/...') would fail.
This patch fetches the real path of source partition, allowing the use
of symlinks.
Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>
When setting the size of a partition without indicating the unit, two
problems occur:
- the size is parsed as an integer, not as a string, hence the
configuration parsing fails
- the size parser doesn't recognize the fact that the size has no units
and defaults to 100%
This patch fixes the configuration parsing as well as the size string
parsing.
Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>
When a partition doesn't have a minimum size in the partition layout
configuration, it defaults to using 100% of the available space.
This patch fixes this error by setting the minimum partition size to 0
when the attribute has been omitted.
Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>
- Write out Int, Double
- Special-case empty lists
- Do objects (not lists of objects) correctly
Now passes the tests for all the example config files.
- The code in loadYaml was refactored out of the module-descriptor
loading code, but the variable names in the implementation were
not changed and still strangely specific to the prior task.
- Add global- and job-configurations for test runs.
- Add a driver script that sets up some assumptions on the host
system so that the tests can complete.
- The idea is that these tests together get a decent code-coverage
for the module.
- This is a driver script for running testmodule.py multiple times
with different global- and job-configurations.
- Usage: testpythonrun.sh <modulename>
- Run the script from the build-directory. It uses files from the
tests/ (source) subdirectory to drive the test runs.
- Only need to get the list of supported filesystems *once*,
not for each and every filesystem that is going to be unpacked.
- Be more Python-idiomatic.
In some cases, we might want to copy a filesystem as if we were using a
simple 'dd' command, in order to create an exact copy, down to the block
level.
This can be useful in particular when working with dm-verity for
checking the rootfs integrity: that way, we can make a direct copy of
the rootfs and its verity partition and keep the system usable.
This patch adds a new 'rawfs' module to calamares, making possible to
block-copy a filesystem to a block device.
Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>
In some cases, e.g. when calamares is used as an "initial setup" tool,
we may want to user to go through all the configuration steps in order
to end up with a usable system.
Therefore, disabling the "Cancel" button can be useful in this case.
This commit adds an option to settings.conf which disables this button
when set to "true". If the option is not present in the settings file,
the default behavior ("Cancel" button enabled & visible) is enforced.
Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>
- A size of 64em has a value less than 1024, which is the minimum
size **in pixels**. The check doesn't make sense as-is and would
have to take the unit into account. Leave that to clients of
branding (e.g. CalamaresWindow, which already does this).
Keeping std::initializer_list around is fraught. Causes segfaults
because I'm not keeping the underlying temporary array around
properly. Switch to vectors because those initialize from the
underlying array.
TODO: look into making this sufficiently constexpr -- perhaps
just use std::array and make find() work on that.
For (all?) those cases where we have configuration with
a value followed by a unit, introduce a class that
uses the NamedEnum properties to make parsing and split-up easier.
- Use locale "C" for checking filesystem names
- Also check other possibilities and case-insensitive, to
be more forgiving of weird configurations (and localizations)
- The language and BCP need to be in-sync
- Existing code was inconsistent in setting things, which is why
you could get through the locale page without setting a locale
(at all) or it would keep English in spite of picking Germand on
the welcome page.
- Patch tests to use that API.
Due to a computation error when calculating the total drive space and
each partition's last sector, the last partition's last sector was out
of boundaries, leading to an error creating this partition.
This patch fixes the computation algorithm to get rid of this error.
Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>
- Allow split-setting of the language and formats
- Test new constructors
- Since fromLanguageAndLocation can handle empty localeGen
lists just fine, skip all the weird checks that return
invalid guessed locale configurations.
- Replace createDefault() with a constructor that takes a
locale name; use it with en_US.UTF-8 in those places where
createDefault was previously used.
Some code was copied and adapted from PartitionActions.cpp. For full
compliance, it is best to copy the copyright holders list from this file
to PartitionLayout.cpp.
Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>
When chosing the "Replace partition" option, free space is not handled
like any partition. In order to apply the custom partition layout in
that case too, we have to modify the code where "replace free space" is
handled.
Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>
In some cases where a custom partition layout is used, use of this
layout is mandatory (this can be the case when using a read-only rootfs
which is updated by block-cpying an image file to it).
For these cases, the user must not be able to change the partition
layout, therefore we have to disable manual partitioning.
In order to stay consistent with current behaviour, manual partitioning
is still enabled by default. It will only be disabled if the partition
module's config file contains the corresponding option set to "false".
Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>
When choosing "Install alongside another system", the custom partition
layout is applied to the space freed by resizing the selected partition.
Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>
This patches add new methods to both PartitionLayout and
PartitionCoreModule classes which apply the partition layout to the
available drive space.
In addition, the partition creation code from PartitioinActions is
removed to call the newly created methods instead, thus applying the
custom partition layout when the "Erase whole disk" and "Replace
partition" choices are selected.
Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>
As we move some of the partition creation code away from
PartitionActions, we will need the bytesToSectors function. Rather than
copying it, we export it in the PartitionActions namespace, so that
other classes can use it.
Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>
In order to keep the partition layout during calamares' execution, we
add a PartitionLayout object instance to PartitionCoreModule. This class
will therefore be used to initialize the PartitionLayout object and
interact with it thoughout the program's execution.
When no partition layout is present in the config file, we initialize
the layout with a single ext4 partition mounted on '/', as it was
previously done.
Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>
In order to use a custom partition layout in the partition module, we
need to write this layout in the module's config file, and store it into
a dedicated object.
As it doesn't look appropriate to extend an existing class with layout
information, we create a new PartitionLayout class, which will be used
to parse the layout from the config file and keep it in memory.
Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>
This basically means we talk about localization in the respective
localized variant. e.g. "German (Germany)" ➡ "Deutsch (Deutschland)".
If geoip lookup failed or isn't configured for whatever reason it's a
stretch to expect the user to know english enough to find their own
language. Preferring the localized strings resolves this issue.
Additionally this happens to bypass #712 respectively
https://bugreports.qt.io/browse/QTBUG-34287
as the native names are properly spelled. So, as long as Qt has localized
names the names will also be properly spelled.
- If a key is missing from mount.conf, don't raise KeyError
- If both keys are missing, suggest that mount.conf might
be missing instead (a consequence of INSTALL_CONFIG=OFF, for
instance).
- Simplify code a bit.
- Don't bother returning None explicitly.
- for remove and localInstall, add support for pre- and post-
scripts like there already was for install.
This feels like there's code duplication going on, but I haven't
thought of an elegant way to distinguish the available operations
so that I can pass around functions instead.
- The noload option prevents journal re-play (so it's an extra-
strong read-only) but is only applicable to ext3 and ext4.
Check the FS type before mounting; other FS types don't
accept -o noload and will fail to mount.
- This only applies to legacy (non-EFI) BIOS systems, and
adds the FlagBoot to whatever is already set for the root
filesystem, and only when autopartitioning the device.
Submitted by aliveafter1000.
FIXES: #1046CLOSES: #1049
Suggested by aliveafter1000: having a default value, and then
filling in the default in one place it is used and not others,
is weird. Instead of dropping the one use, remove the default
value: partition flags are important enough to be explicit.
- Handle legacy and modern config, mixed-configs,
- Translate strings to enum values,
- Default and warn as appropriate.
- Doesn't **do** anything with the config, though.
- Use DATA for the qml and branding directories (looks for qml/
and branding/<name>/ in those directories).
- Use CONFIG for the global settings and module settings (looks
for settings.conf and module/<name>/ in those directories).
FIXES#941
- Running lsblk and mount for debugging purposes can be
skipped when the debugging is going to be suppressed anyway.
This will speed things up just a little for regular users.
- While winnowing devices, the zram and nullptr cases
were mixed together; split them, for the sake of
logging more accurately.
- While here, fix up some coding-style issues.
- Q_ASSERT doesn't work in constexpr functions because it's not
- May as well calculate bytes at compile-time, no need to give
the runaround via number-of-MiB
- Distinguish size and atleast; in percentages they mean different
things (atleast is a bit weird as a percentage, but hey).
- Fix bug in percentage calculation.
- Avoid percentage above 100.
- Add documentation in config-file.
- Be more verbose when things go wrong while loading module-
configuration files.
- Allow more forms of paths to be specified. In debug mode, this
is mostyle helpful for the module test-loader.
- Test the parsing of configuration values, that it gets the right
values when given correct(-ish) configurations. Doesn't test
any situations with bad config.
- Make RelativeSize public so we can use it in non-member functions
- Make a template out of matching the string suffixes; this is
safer because the length of the suffix can be computed at compile-time
(+1 for the trailing NUL) rather than writing it out in boilerplate.
- Non-functional as yet, WIP
- Intended for use on SBC images and similar, which need to resize
the root partition to the size of the SD card they're on.
FIXES#921
- The choice of swap needs to be handled in more places,
so make the enum available in the partition module core instead
of just inside the choice page.
- Q_ASSERT doesn't work in constexpr functions because it's not
- May as well calculate bytes at compile-time, no need to give
the runaround via number-of-MiB
This fixes the crash by calling the model-reset first, then
refreshing. Previously, the destructors that do the work
were still being called in the wrong order.
FIXES#1019
- The ResetHelper only finalized changes to the module on
destruction, but calls to refresh() assumed it was already
done. This leads to crashes when refresh() uses an intermediate
state of the model.
Introduce extra helpers, and rename refresh() to avoid calling the
old implementation from any code. The new helper just creates and
destroys a ResetHelper, before creating and destroying an object
that calls the new refreshAfterModelChange().
FIXES#1019
- document new variable from the CMake module
- use it in libcalamaresui to simplify #include'ing the
header for the "all" extension.
Suggested by Denis Proskurin.
ESP == boot. at best this is duplicated information, at worst kpmcore may
implode if you try to set a boot flag since that is technically an MBR
type flag and means nothing within the context of GPT where ESP is the flag
to set.
having ESP as active flag AND then trying to set ESP means nothing is
set since kpmcore will think ESP is already set (it is listed as active
after all). this ultimately meant that nothing was set since there was
no delta between the requested flags and the already active flags.
- The parameter list was wrong, and could never have worked
(apparently noone mixes LightDM with basic setup, even though
it's one of the few with basic_setup()).
- Shorten some lines.
- Move variable initializations closer to where they are first used
- Also warn if no implementations are available
- Handle sysconfig as any other DM; there's no real reason
to treat it specially.
- Add (empty) implementations of all the abstract methods that
are not needed (or supported) by various DMs.
- Order the abstract methods by calling order
- Fix up have_dm
- Make root_mount_point a parameter of the DM classes
- Create instances as needed, then check if they're available
- Keep instances that actually need to be configured
- Clean up remaining cruft from removing old setup_autologin()
with all the string-comparison magic.
- Some DMs allow setting up the default DE. Factor that
out into a method like basic_setup() and setup_autologin()
and add it to the configuration chain.
- For all the DMs that have some kind of basic setup,
introduce a method in the DM class for doing just that.
- The Python code now doesn't call basic setup anywhere,
so this specific revision isn't going to work properly.
- Split the entire body of setup_autologin() to a method
per implementation class.
- Make the check for presence of a DM a class-method,
since if it fails, instantiation is going to be useless.
If displaymanagers is not set in the job config or globally,
it wasn't set at all, leading to a runtime error. Set to
None, so that the regular error message will be triggered.
FIXES#1002
- Reported by Bill Auger (I think), a 15GiB disk wouldn't hold
a 8.9GiB root plus 4GiB swap -- due to 10% overprovisioning
of swap, plus the 2.1GiB fudge factor.
- Calculating first free sector had an off-by-one so that
partitioning would start at 2049.
- EFI boot partition grew 1 sector larger than desired.
- While here, align everything to 1MiB boundaries as well.
FIXES#1008
- Similar to the refactorings in Calamares proper, just split out
collecting the search paths into a static function. This makes
it a little easier to find places that will need expansion for
more-than-one-config-directory.
- Make *mandatory* optional (and default to false); this
allows shorter lists of entries
- Allow degenerate entries which are just a name
(which have *mandatory* set to false as well).
SEE #992
- Allow just a name entry, instead of requiring an object
entry; this makes "foo" equal to { name: "foo", runlevel: "default" }
and simplifies more for the straightfoward case of #974.
- Based on comments from #974, follow the configuration
scheme from services-systemd, so with separate lists
"services" and "disable". This ties it **slightly**
less closely to the commands passed to rc-config.
- If runlevel isn't set (at all) then use "default". For
most systems that do not use multiple runlevels, this
simplifies the configuration to just a list of service names
to add or delete.
- Document the functions some more
- Only "state" (i.e. action) "add" and "del" make sense,
avoid calling rc-update for other keys (e.g. typo's).
This matches the documentation, although there might be
other actions that make sense (see also services-systemd,
with its enable, disable and mask actions).
- With refactored code, introducing new kinds of actions
is very few lines of code. Allow disabling targets
(services was already possible). Allow masking units,
but as a special case require the complete name.
FIXES#975
- The three steps of modifying services in the target
system do basically the same thing, so factor out
the loops and logging into a systemctl() function.
- Log to warning() instead of just debugging, on failure.
- There is more to failing out of loadModules() than just
emitting modulesFailed, so instead share the failure
code with the code after loading modules -- but don't load any.
- Module dependency-checking is done in two phases:
first, catch any unknown modules that are listed
in *requiredModules* and bail out before loading
anything. Second, check that the modules required
by X occur before X in the sequence.
- The value set in module.desc was never stored for use,
but isn't an attribute of the instance, either. It belongs
with the descriptor, in ModuleManager.
If USE_<foo> is given a value that doesn't match **anything**,
then bail out. Since USE_* is an explicit distro choice for a
specific implementation, it's an error if that implementation
is not there.
When there are multiple modules doing a thing and it really only
makes sense to have one of them in a given Calamares compilation,
the USE_<foo> variables allow you to select one, while ignoring
all the other implementations. If USE_<foo> is not set, all
implementations are included (as usual).
- Use YAML-CPP API for finding out if a node has a value at all.
- Asking for Type() of an undefined or NULL node throws an
exception, so the existing code didn't **actually** catch
cases where a required setting wasn't set at all.
- A valid line (as explained in the comments at the top of
the locale.gen file) is <locale> <encoding> (two fields),
so lines with more than two fields can't be valid locale-
listing lines. For them, pretend they name locale "",
which won't be matched.
- Improved debug-logging
- Fix the actual problem of listing locales more than once,
by listing them all, uniqified, at the end, with an explanitory
comment in the generated file.
- Be more accepting of what constitutes a locale-line; this allows
spaces before and after the `#` comment sign, but because we're
uniquifying, this doesn't cause duplicates.
- Because we write the enabled locales at the end, the full file
comment-header is retained un-mangled (instead of accidentally
enabling a locale mentioned as an example there).
Testing for existence of a file in the live system, and then
copying it in the target system, is not a recipe for success.
- Fix the restore-from-backup part.
- Document that your live and target system must both have
/etc/locale.gen if you want this to work at all.
Also make install for yum and dnf follow the documented syntax: options
(-y) before the command (install), even though yum and dnf also accept
the other order. This also makes it consistent with remove.
Untangle the shortcuts; Create and Cancel had an overlap.
Skip 'r' (Revert all changes) and 'e' (Edit) and settle on
'a' (which might also mean "Add").
FIXES#977
Introduce the notion of emergency modules and emergency jobs.
Initial use will probably center around the preservefiles module,
and possibly umount.
FIXES#928
- After a failure, skip non-emergency jobs.
- After running all emergency jobs, then emit failure message.
- In log, distinguish emergency and non-emergency jobs.
Any job can be an emergency job; emergency modules spawn
emergency jobs (but conversely, a non-emergency module
can spawn an emergency job explicitly).
A potentially emergency module is one that has EMERGENCY
(in CMake) or emergency: true (in module.desc) set.
Any such module must also set emergency: true in the
configuration of the module. This is to allow for
instances of a module that **don't** run as emergency
modules, alongside actual emergency ones.
- In many cases, using QLatin1String is a de-optimization, when
applied to a C string literal. Kevin Kofler pointed out that
those should basically all be QStringLiteral, instead. (Compile
tests with -O3 show that in the optimized object file, the
code size difference is negligible).
- Drop the explicit constructor entirely in cases where we're calling
QProcess::execute(), for consistency.
- Do a little less messing around in the mapping of keyboard locales
to keyboard map names.
As Kevin pointed out, there's an extra conversion involved here --
although with -O3 the difference boils away leaving only a call
to a from-ASCII helper or a from-Latin1 helper.
While here, coding-style.
The plymouthcfg Calamares module is optional. Distributions which
write filesystems with a full plymouth configuration won't even
want to use it (see plymouthcfg docs).
However, now grubcfg depends on plymouthcfg to run because
the globalstorage value to trigger setting 'splash' in grub,
is set in the plymouthcfg module.
Just check for plymouth existence separately in the grub module.
Fixes ea1c8a0e5d
Since this is a new language, it is currently 0% translated.
That is why it goes into _tx_bad. It will move to one of the
other categories once some translation has happened. Add the
(still empty) Transifex files already.
- Add a TODO for allowing modules to come from somewhere other
than the module loader (this would allow "internal" modules
that are always present)
- Warnings are warnings
- Add -v (verbose) and -b (load via bytearray)
- Verbose prints the keys read from the file,
- Bytes reads via an indirection through QByteArray, like Settings does
- Collect the failed modules, instead of bailing out on the first one
(this also prevents crashes caused by quit() called from a timer).
- Introduce a slot to report on failed module loading (no UI yet).
- Fudge the numbers in the North, to improve location
of the pins and lines of latitude.
- Inuvik, Yellowknife, Cambridge Bay, Resolute look ok
- Thule, Scoresbysund look ok; Danmarkshavn a pixel or so too far North
- Reykjavik is a bit too far North
- Longyearbyen is a bit too far North
Since these places are off by one or two pixels, this becomes
invisible when a large pin + text label is placed on it.
The scaling on the map was a little off; the degrees of latitude
are a little wider there than around the equator and Europe.
- Johannesburg is in the right spot
- Hobart is no longer a suburb of Melbourne
- Punta Arenas is in Chile
Replace pin and text label with just a dot (to pinpoint where
locations are) and draw latitude lines on the globe when
DEbUG_TIMEZONE is set at compile time. Since there's probably
still timezone-related bugs (in particular in the images that
map points on the globe to timezones), leave this in the codebase.
This is orthogonal to the SKIP_* mechanism already documented
for avoiding modules. If the module is enabled, but its dependencies
are not present, don't bother building the module. This follows
e.g. plasmalnf as an "avoidably heavy dependency".
Related to a misplaced comment in ISSUE #956
This test-application should load a single module and execute it --
that can be used to quickly test configurations, loading, etc. This
is preparation for loading all sorts of Python modules by PythonQt.
The loader does some internals initialization and gets the module,
but doesn't actually run it yet.
- Add a *userShell* key, which can be left out (default, backwards-
compatible) to retain the old /bin/bash behavior, or explicitly
set to empty to defer to useradd-configuration, or explicitly
set to something non-empty to use that shell.
- This is prep-work for #964, which was caused by #955
- Original assumption was that distro's would have a working
useradd configuration; @abucodonosor already pointed out that
this was probably not the case, but I ignored that.
- document accessors
- put all path and directory accessors together
- make simple accessors inline
- rename "pathprefix" to "directory" to be consistent with others
- more flexible way to keep (all kinds of) files from the host
system, into the target system.
- WIP: substitutions like in shellprocess (@@ROOT@@, @@HOME@@ probably)
- WIP: creating a JSON file from global settings
Extensive go-over on the partitioning code. #622 is maybe "possibly fixed",
but there's no real indication of what constitutes an invalid combination
of flags.
FIXES#884FIXES#951FIXES#953FIXES#622
The substitution of @@ROOT@@ should happen when running in the
host, not in the target, system. Also only complain about it
if @@ROOT@@ is actually used in the commands.
FIXES#954
- If there is a partition already (newly) created, then pass that
to the dialog so that it can use the setings previously applied
(e.g. mount point and flags).
- This avoids the case where you create or format a partition,
then click on it again to edit it and the previous settings are lost.
- Setup the lsit of flags consistently, by providing the available
and to-be-checked flags.
- In CreatePartitionDialog, assume that ~0 is all the flags.
This file is full of helper functions for the partition-editing
dialogs. At first it was just mount-point helper functions,
but there is other functionality that can be refactored.
- If we're changing the flags to enable EFI boot, then that's
enough to satisfy the (future) EFI bootability check.
This is for issue #622 as well. Fixes#884.
- Use the desired (future) flags, if set, to initialize the
flags checkboxes. If there are no future flags set, this
returns active flags as before.
- This fixes the situation where editing a partition, changing
flags, then editing it *again* re-starts with the original
flags instead of the modified flags.
- PartitionInfo maintains information on "what is desired" for
a given Partition. Now we can set desired flags, alongside
the flags already supported by Partition (where activeFlags()
gives you the flags currently set on that partition).
- Avoids case where you edit a partition with a mountpoint
set; previously, calling setText() would update the text
but leave the selected index unchanged (usually 0), so that
later calling selectedMountPoint() would return empty.
- Move to one place which handles the standard mount points
- While here, introduce explicit "(no mount point)" string
into the combo box. This is prep-work for issue #951.
- Some locales have no nativeLanguageName(), so instead display
the locale id (e.g. "eo") and the resulting language in English
(which, if it is really unsupported, will be "C").
