feat(iso): split the live medium into semantic SquashFS layers

Booting via BMC virtual CD reads the ~2.8 GB filesystem squashfs
sequentially during the live-boot toram copy; a mid-read drop of the
redirected medium loses the whole copy and fails the boot (v14). Split
the rootfs into self-contained semantic layers so a retry re-reads at
most one ~500-700 MiB layer, not everything. This is a resilience /
reduced-re-read mechanism, not a fix for the virtual-media instability.

NVIDIA variants now ship 7 layers (00-base, 05-firmware, 08-desktop,
10-nvidia-driver, 20-nvidia-platform, 30-nvidia-cuda-libs,
40-nvidia-dcgm-cuda) plus an explicit live/filesystem.module that fixes
their OverlayFS order; amd/nogpu keep a single squashfs.

- lib/squashfs-layers.sh: deterministic classifier (dpkg file ownership
  plus explicit rules for build.sh-injected files, never a path
  substring), per-layer mksquashfs, 800 MiB hard ceiling, unsquashfs -s
  plus strict extraction of every layer, merged-rootfs bootability check.
- build.sh: split the monolith after the full lb build, verify and merge,
  write the module file, delete the monolith only then; abort before ISO
  assembly on any failure. Runs the builder test suites up front.
- fast-path: force a full build for a multi-layer medium;
  fast_path_repack_squashfs hard-refuses (it would drop layers).
- iso-validation.sh: validate_iso_squashfs_layers (module vs layer set
  match, size ceiling, no lone giant squashfs) and
  validate_iso_media_integrity (xorriso -check_media).
- bee-install: honour filesystem.module order, abort on any layer failure.
- 9013-toram-retry: record the real rsync exit code (it printed a false
  rc=0) and correct the "resumes the tail" comment (rsync without
  --partial keeps only fully-copied layers). No unsafe partial resume.
- tests: test-squashfs-layers.sh plus a multi-layer guard in
  test-build-libs.sh; both run at the top of every build.
- docs: bible-local architecture and decision, iso/README, iso-build-rules.

Verified by a full nvidia build: 7 layers 622/199/256/466/37/567/562 MiB,
every validator passes, xorriso -check_media good, merged rootfs bootable.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Mikhail Chusavitin
2026-09-04 15:38:14 +03:00
co-authored by Claude Sonnet 5
parent bc57b85d3f
commit b8c45d54c1
13 changed files with 1056 additions and 13 deletions
@@ -12,8 +12,15 @@
# adds several full attempts with a geometrically growing pause between them
# (15s, 30s, 60s, 120s, 240s, 480s, 900s), giving the virtual media time to
# re-enumerate. Between attempts the medium is unmounted, waited on, and
# remounted. rsync resumes (already-copied files are skipped), so a retry that
# only needs the tail of the squashfs finishes quickly.
# remounted.
#
# rsync here runs without --partial, so it keeps only files it copied in full;
# a file interrupted mid-transfer is discarded and re-read from the start on the
# next attempt. The medium is split into semantic squashfs layers
# (filesystem-v<ver>-NN-*.squashfs, see lib/squashfs-layers.sh), so a retry only
# re-reads the layer that was in flight, not the whole ~2.8 GB rootfs. We do
# NOT enable --partial / --append: resuming a partial squashfs without a
# post-copy integrity check would risk booting a truncated layer.
set -e
TORAM_SCRIPT="/usr/lib/live/boot/9990-toram-todisk.sh"
@@ -56,12 +63,19 @@ bee_rsync_retry ()
echo " * bee: toram copy attempt ${_try}/${_maxtry} (medium ${_dev:-?} ${_fst:-?})" 1>/dev/console
if rsync -a --progress --timeout=180 ${_src}/* ${_dst} 1>/dev/console
then
_rc=0
else
_rc=$?
fi
if [ "${_rc}" -eq 0 ]
then
echo " * bee: toram copy completed on attempt ${_try}" 1>/dev/console
return 0
fi
echo " * bee: toram copy FAILED (rsync rc=$?) on attempt ${_try}" 1>/dev/console
echo " * bee: toram copy FAILED (rsync rc=${_rc}) on attempt ${_try}" 1>/dev/console
if [ "${_try}" -ge "${_maxtry}" ]
then