Commit Graph

72 Commits

Author SHA1 Message Date
Adriaan de Groot
4370669fc3 [unpackfs] Put exclusion settings into the rsync command line 2019-10-11 17:46:01 +02:00
Adriaan de Groot
9d47716e5d [unpackfs] Add settings per-item for exclude and excludeFile
- support excluding specific files / lists when rsyncing, in case
   the squashfs has more in it than you really want to copy over.
2019-10-11 17:19:56 +02:00
Adriaan de Groot
4552631086 [unpackfs] Prep for more information in file_copy
- Give the whole entry to file_copy, not just the
   destination. This will allow file_copy to work
   with local excludes.
 - Pluck entry.destination out immediately, to keep
   code changes minimal.
 - Document the parameters.
2019-10-11 16:56:04 +02:00
Adriaan de Groot
311c65737f [unpackfs] Remove unused parameter
- list_excludes() turns the extra mounts from global storage
   into --exclude parameters for rsync; it doesn't do anything
   with the destination parameter.
 - while here rename to something more descriptive
2019-10-11 16:47:54 +02:00
Adriaan de Groot
fe1ddb8ead [unpackfs] Expand on the documentation 2019-10-11 16:32:39 +02:00
Adriaan de Groot
57c0b2088c [unpackfs] Allow naming a non-directory for single files
- this enables renaming files on copy
2019-10-11 15:48:17 +02:00
Adriaan de Groot
4acf63fd9a [unpackfs] Document fields of the UnpackEntry
- Document fields
 - While here, remove code for special-casing single-file copies.
   Those already work, based on rsync.
2019-10-08 17:31:51 +02:00
Adriaan de Groot
05c0398126 [unpackfs] Support multiple items better
- it's ok if item one creates directories where item two will write,
   so don't check for existence of all directories on start-up.

Reported by ArcoLinux.
2019-10-08 16:19:39 +02:00
Adriaan de Groot
d58fdaa2cd [unpackfs] Stub of special handling for a single file 2019-10-08 15:00:25 +02:00
Adriaan de Groot
95f725831f [unpackfs] Shuffle the documentation
- move all the examples to an EXAMPLES section
 - leave one single-file example enabled in the configuration
2019-09-30 17:35:31 +02:00
Adriaan de Groot
6fb2563c75 [unpackfs] Copy files when sourcefs = "file"
- Just use the existing rsync code, which can do both
   files and directory trees.
 - The existing code assumed we were always copying directories.
   Now double-check beforehand.
2019-09-30 17:14:23 +02:00
Adriaan de Groot
27cdaba8b2 [unpackfs] Special-case file sources 2019-09-30 17:14:23 +02:00
Adriaan de Groot
7856c1a6a8 [unpackfs] Do more checks before doing any work 2019-09-30 17:14:23 +02:00
Adriaan de Groot
784bc5b86e [unpackfs] Expand documentation
- comment out the squashfs example, since it's not readily
   available even on developer systems
 - add entries for the upcoming "file" type.
2019-09-30 17:14:23 +02:00
Arnaud Ferraris
0f6b2dbe2d [unpackfs] Enable to use "/" as a source
If we don't have/need an image for the rootfs, we might want to
configure the `/` directory as a source for unpackfs. Unfortunately,
this raises an error:
  - unpackfs first creates a temporary directory
  - it then creates a subdirectory for each source, using the source
path's basename
  - when the source is `/`, the basename is an empty string, therefore
the module tries to create an already existing directory

In order to prevent this error, we use the `os.makedirs` function with
parameter `exist_ok=True` instead of `os.mkdir`.

Signed-off-by: Arnaud Ferraris <arnaud.ferraris@collabora.com>
2019-08-23 17:26:45 +02:00
Arnaud Rebillout
2ab45cf4f0 Make sure that variable num_files_total_local is declared
This variable is declared in `if m:`. Of course if this codepath doesn't
run, the variable is not declared an Python doesn't like this kind of
surprise...

Signed-off-by: Arnaud Rebillout <arnaud.rebillout@collabora.com>
2019-08-12 15:37:02 +02:00
Adriaan de Groot
b0bcdba867 [unpackfs] Needs mount before it
- The mount module must happen before unpackfs because that (mount)
   module sets up the root mount point (in /tmp) and some other
   variables needed later.
2019-08-01 12:37:03 +02:00
Kevin Kofler
71d991e2e4 [unpackfs] Do not use -o loop on a device
src/modules/unpackfs/main.py (UnpackOperation.mount_image): Check
whether entry.source is a regular file or a device and only use
`-o loop` on regular files, not devices.

At least on Fedora >= 29, `-o loop` fails on the read-only device
`/dev/mapper/live-base` (though `-o loop,ro` would be accepted).
2019-05-06 15:04:58 +02:00
Adriaan de Groot
fb412c177c [unpackfs] Improve human-readable name 2019-03-29 17:32:52 -04:00
Adriaan de Groot
5c4f2423f8 [unpackfs] Fix error in error-handling (warn -> warning) 2019-03-29 17:27:26 -04:00
Adriaan de Groot
d87badbf45 [unpackfs] Add test with too-small destination FS 2019-03-29 17:22:47 -04:00
Adriaan de Groot
2a6bf50621 [unpackfs] Don't let ir-phase reduce total number of files 2019-03-29 16:53:02 -04:00
Adriaan de Groot
fae0b8c2f8 [unpackfs] Re-jig progress reporting
- rsync reports its own progress, and reports on files that
   find -type f doesn't. This meant that the numbers didn't
   match what was stored in entry.total
 - The ir-phase adds files to be handled; to-phase happens once
   ir-phase is over and the remaining files are processed.
   By adding the to-phase files, percentages over 100% were
   reported (in part because the number of files doesn't match).
 - Update expected entries total from rsync output.
 - Re-jig computation of how done everything is: tally it
   up in integers, and do only one global progress percentage.
2019-03-29 16:48:46 -04:00
Adriaan de Groot
6d85fd3586 [unpackfs] One last progress call afterwards 2019-03-29 16:24:22 -04:00
Adriaan de Groot
9ce34782ee [unpackfs] Avoid double / at end 2019-03-29 16:09:11 -04:00
Adriaan de Groot
dc2fafe324 [unpackfs] to-chk is also progress information
- ir-chk happens first, and then there's a phase with to-chk
   messages; use those as well.
2019-03-29 12:51:00 -04:00
Adriaan de Groot
38821f47ba [unpackfs] Make comment match code
- The mismatch between "ir-chk" and the comment "to-check" led me
   to check (ha!) the output of rsync, and it outputs "to-chk"
   during small transfers; make sure the comment reflects what
   is actually being used to track progress (which is "ir-chk").
2019-03-29 12:39:45 -04:00
Adriaan de Groot
b3a7545217 [unpackfs] One more test case 2019-01-25 11:43:33 -05:00
Adriaan de Groot
d95193f646 [unpackfs] Translate and log other error messages 2019-01-25 08:02:56 -05:00
Adriaan de Groot
79ce2c2d0f [unpackfs] Translate pretty module name 2019-01-25 07:54:51 -05:00
Adriaan de Groot
28efe2b85a [unpackfs] Log messages in English, too
- For debugging purposes, need to log the message in English before
   returning the (user-visible) error description.
2019-01-25 07:53:39 -05:00
Adriaan de Groot
8ddf801bfb [unpackfs] Switch on translations for user-visible strings 2019-01-25 07:46:19 -05:00
Adriaan de Groot
18f172e80c [unpackfs] Add tests
- Add global- and job-configurations for test runs.
 - Add a driver script that sets up some assumptions on the host
   system so that the tests can complete.
 - The idea is that these tests together get a decent code-coverage
   for the module.
2019-01-25 07:20:16 -05:00
Adriaan de Groot
2d684bb385 [unpackfs] Be more resilient to globalStorage weirdness
- If no extraMount is defined, just don't mount anything
2019-01-25 07:14:23 -05:00
Adriaan de Groot
e96a29b1b9 [unpackfs] Improve error message wording 2019-01-25 06:08:56 -05:00
Adriaan de Groot
56d05d5834 [unpackfs] Reduce file-reading
- Only need to get the list of supported filesystems *once*,
   not for each and every filesystem that is going to be unpacked.
 - Be more Python-idiomatic.
2019-01-25 06:01:49 -05:00
Alf Gaida
784bbd3bc8 finally !must! never fail unhandled
but it does without help
2017-12-26 22:25:40 +01:00
Adriaan de Groot
762ad54344 Documentation: change http links to GitHub to https 2017-12-20 08:39:09 -05:00
Adriaan de Groot
63f9c25611 Update unpackfs.conf examples and documentation 2017-08-23 06:57:11 -04:00
Adriaan de Groot
ca3b10002b Modules: minor cleanup + docs 2017-08-09 05:45:09 +09:00
Adriaan de Groot
ae6a9cd822 Python: move unsquash documentation to the config file 2017-07-10 11:58:58 -04:00
Alf Gaida
5f7b1dda26 Fixed pep8 whining in module unpackfs
Added myself to copyright
2017-06-06 17:44:53 +02:00
Kevin Kofler
0131d7fbf1 [unpackfs] Do not fail if rsync returns exit code 23.
Unfortunately, rsync returns exit code 23 (Partial transfer due to
error) if it cannot write extended attributes (with -X) because the
target file system does not support it, e.g., the FAT EFI system
partition. We need -X because distributions using file system
capabilities and/or SELinux require the extended attributes. But
distributions using SELinux may also have SELinux labels set on files
under /boot/efi, and rsync complains about those. The only clean way
would be to split the rsync into one with -X and --exclude /boot/efi and
a separate one without -X for /boot/efi, but only if /boot/efi is
actually an EFI system partition. For now, this hack will have to do.

See also:
https://bugzilla.redhat.com/show_bug.cgi?id=868755#c50
for the same issue in Anaconda, which uses a similar workaround.
2016-11-04 01:37:08 +01:00
hmikihth
42ceffa2e9 . 2016-06-05 13:16:31 +01:00
hmikihth
f18dc02459 A comment has been deleted 2016-06-04 12:23:59 +01:00
hmikihth
062e9b764c blackPanther OS and Fedora compatibility 2016-06-04 12:11:02 +01:00
hmikihth
377a1c3a17 Support the already mounted loop fs (example in the blackPanther OS) 2016-06-04 03:15:43 +01:00
Teo Mrnjavac
98b5085e6e Make sure we print the error message before the finally: kicks in. 2016-04-28 16:37:21 +02:00
Gormogon
9f8a8f6a64 PEP and Unused Import 2015-06-14 08:22:35 -04:00
Gormogon
bc66502fda Some more PEP 2015-06-14 07:25:37 -04:00