- the module is 'unpackfs', not 'unsquash'
- add a warning + specific error if there is no unpack key in the config
- the 'doing nothing' part isn't true: the module errors out instead
of doing nothing.
SEE #1870
- make the installation work,
- special case because rsync can return error 23 (which throws, from
inside the Python API) which still means "it was ok".
SEE #1740
By processing each line in turn (and just counting is) rather
than collecting all of the lines of output from the tools,
we end up with lower memory usage.
Apparently everyone shipping a squashfs image also has the tools
installed, because the error message reporting that the tools-are-
missing contained a reference to an undefined variable.
Fix that, and while here improve the error message so you
don't get a whole path as a title in the error message.
The slightly weird error-message construction is so that no
messages change and no translation work is needed.
All **other** modules fully specify libcalamares; only unpackfs
was importing shortcuts. Change to conventional usage (partly
because that's easier on the pylint implementation, partly because
it's then consistent with the rest).
The module preserves the extended attributes at rsync and the overlay
filesystem stores extended attributes by inodes.
The overlay filesystem keeps traces of the lower directory by encoding
and storing its UUID to the attribute trusted.overlay.origin. If the
index feature is on, that attribute is compared to the UUID of the lower
directory at every subsequent mounts and causes mount to fail with
ESTATE if it does not match.
This filters the namespace trusted.overlay.* by using the rsync option
--filter='-x trusted.overlay.*' to make sure the overlays extended
attributes are not preserved.
Fixes:
# mount -t overlay -o lowerdir=...,upperdir,...,workdir= overlay /mnt/etc
mount: /var/mnt/etc: mount(2) system call failed: Stale file handle.
# dmesg
(...)
overlayfs: "xino" feature enabled using 32 upper inode bits.
overlayfs: failed to verify origin (/etc, ino=524292, err=-116)
overlayfs: failed to verify upper root origin
This still won't help if there's one really huge file that takes
several seconds to write, but if there's a bunch of files together
that is less than a file_chunk_count but take more than a half-
second to write, update anyway
If there's thousands of files in a squashfs (e.g. 400000 like on
some ArcoLinux ISOs) then progress would be reported every
4000 files, which can take quite some time to write. Reduce
file_chunk_count to at most 500, so that progress is reported
more often even if that wouldn't lead to a visible change
in the percentage progress: instead we **do** get a change
in files-transferred numbers.
- The total weight is only needed by the UnpackOperation,
not by each entry.
- Use a chunk size of 107 so that the number-complete seems busy:
the whole larger-or-smaller chunk size doesn't really matter.
- The progress-report was missing the weight of the current
module, so would report way too low if weight > 1. This affects
ArcoLinux configurations where one entry is huge and one is a
single file, so weights 50 and 1 are appropriate.
When there are multiple entries, the overall weight of the
module is divided between the entries: currently each entry
takes an equal amount of space in the overall progress.
When there are multiple entries which take wildly different
amounts of time (e.g. a squash-fs and a single file) then
the progress overall looks weird: the squash-fs gets half
of this module's weight, and the single file does too.
With the new *weight* key for entries, that division can
be tweaked so that progress looks more "even".
- point to main Calamares site in the 'part of' headers instead
of to github (this is the "this file is part of Calamares"
opening line for most files).
- remove boilerplate from all source files, CMake modules and completions,
this is the 3-paragraph summary of the GPL-3.0-or-later, which has
a meaning entirely covered by the SPDX tag.
In spite of there being considerable documentation sometimes in the
config file, we go with CC0 because we don't want the notion of
'derived work' of a config file.
The example `settings.conf` is also CC0. Add some docs to
it while we're at it.
There's lots of (YAML) test data that is just trivial configurations
for modules. Since the configurations themselves are **also** CC0-1.0,
and the tests are less interesting, license them equally liberally.
- a handful of modules had an unused *requires* key in module.desc;
this is probably from previous intentions around
prerequisites-testing. Since the settings were empty anyway,
they have been removed.
- [unpackfs] Compacted the way *requiredModules* list is written
- These have **not** been fixed for validation, so the schema's themselves
will fail to load. This is a consequence of variations in JSON-Schema
representations through various drafts. Fixing the schemata is
fairly straightforward.
This gives us 19 new tests, all of which fail.
- Slice overall progress into chunks, with each chunk of equal size
(as long as we have no overall count information) and place
the progress of the current chunk into its own slice.
- The entry knows where it should be mounted, and can remember that
- mount_entry() didn't use self, so made no sense as a method
of the Operation class
- Follow original patch from Gabriel Craciunescu: just drop
the *bufsize* parameter and stick to binary reads.
Text mode was associated in my testing with multiple hangs,
which didn't show up during binary-reads.
- Give the whole entry to file_copy, not just the
destination. This will allow file_copy to work
with local excludes.
- Pluck entry.destination out immediately, to keep
code changes minimal.
- Document the parameters.
- list_excludes() turns the extra mounts from global storage
into --exclude parameters for rsync; it doesn't do anything
with the destination parameter.
- while here rename to something more descriptive
