Booting via BMC virtual CD reads the ~2.8 GB filesystem squashfs sequentially during the live-boot toram copy; a mid-read drop of the redirected medium loses the whole copy and fails the boot (v14). Split the rootfs into self-contained semantic layers so a retry re-reads at most one ~500-700 MiB layer, not everything. This is a resilience / reduced-re-read mechanism, not a fix for the virtual-media instability. NVIDIA variants now ship 7 layers (00-base, 05-firmware, 08-desktop, 10-nvidia-driver, 20-nvidia-platform, 30-nvidia-cuda-libs, 40-nvidia-dcgm-cuda) plus an explicit live/filesystem.module that fixes their OverlayFS order; amd/nogpu keep a single squashfs. - lib/squashfs-layers.sh: deterministic classifier (dpkg file ownership plus explicit rules for build.sh-injected files, never a path substring), per-layer mksquashfs, 800 MiB hard ceiling, unsquashfs -s plus strict extraction of every layer, merged-rootfs bootability check. - build.sh: split the monolith after the full lb build, verify and merge, write the module file, delete the monolith only then; abort before ISO assembly on any failure. Runs the builder test suites up front. - fast-path: force a full build for a multi-layer medium; fast_path_repack_squashfs hard-refuses (it would drop layers). - iso-validation.sh: validate_iso_squashfs_layers (module vs layer set match, size ceiling, no lone giant squashfs) and validate_iso_media_integrity (xorriso -check_media). - bee-install: honour filesystem.module order, abort on any layer failure. - 9013-toram-retry: record the real rsync exit code (it printed a false rc=0) and correct the "resumes the tail" comment (rsync without --partial keeps only fully-copied layers). No unsafe partial resume. - tests: test-squashfs-layers.sh plus a multi-layer guard in test-build-libs.sh; both run at the top of every build. - docs: bible-local architecture and decision, iso/README, iso-build-rules. Verified by a full nvidia build: 7 layers 622/199/256/466/37/567/562 MiB, every validator passes, xorriso -check_media good, merged rootfs bootable. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
6.0 KiB
ISO Build Rules
Verify package names before use
ISO builds take 30–60 minutes. A wrong package name wastes an entire build cycle.
Rule: before adding any Debian package name to the ISO config, verify it exists and check its file list.
Use one of:
https://packages.debian.org/bookworm/<package-name>— existence + descriptionhttps://packages.debian.org/bookworm/amd64/<package-name>/filelist— exact files installedapt-cache show <package>inside a Debian bookworm container
This applies to:
iso/builder/config/package-lists/*.list.chroot- Any package referenced in bootloader configs, hooks, or overlay scripts
SquashFS layer rule
The NVIDIA live medium is not one squashfs. build.sh splits the monolith into
semantic layers (filesystem-v<ver>-NN-*.squashfs) after the full lb build,
driven by iso/builder/lib/squashfs-layers.sh. See
bible-local/architecture/squashfs-layers.md for the layer model and
decisions/2026-09-04-squashfs-semantic-layers.md for why.
Rules:
- Classify by dpkg file ownership, not path substring. New non-
.debfiles injected bybuild.shmust get an explicit entry inbee_layer_injected_rules(or they fall to00-base). live/filesystem.moduleis authoritative for layer order; keep it in sync with theNN-prefixes. Later layer wins a conflict, in live-boot,bee-install, and any rebuild.- Keep each layer a self-contained valid squashfs under the 800 MiB ceiling. Split a semantic layer further rather than blindly slicing bytes.
- Never let an upper layer carry a real top-level
bin/sbin/lib/lib64path - that breaks merged-usr the same way the fast-path bug did. - The builder deletes the monolith only after every layer verifies and re-merges into a bootable rootfs. Any failure must abort before ISO assembly.
- The fast path stays disabled for a multi-layer medium until a layer-aware repack exists. Do not "take the first squashfs".
test-squashfs-layers.shandtest-build-libs.shrun at the top of every build; keep them green.
Bootloader sync rule
The ISO has two canonical bootloader templates whose live entries must remain semantically equivalent:
| File | Used by |
|---|---|
config/bootloaders/grub-efi/grub.cfg |
UEFI (all modern servers) |
config/bootloaders/isolinux/live.cfg.in |
CSM / legacy BIOS (syslinux) |
live-build does not derive one from the other. lib/bootloader.sh renders both
templates into the final binary/ tree after live-build has created it, and the
ISO validator checks every live entry. Any menu or kernel-parameter change must
still be made in both templates.
Canonical entry list (both files must have all of these):
| Label | Key params |
|---|---|
| normal (default) | nomodeset bee.nvidia.mode=normal + full param set |
| load to RAM | toram nomodeset bee.nvidia.mode=normal + full param set |
| no GUI | nomodeset bee.gui=off bee.nvidia.mode=normal + full param set |
| fail-safe | normal parameters plus pci=realloc iommu.strict=1 |
| wipe | toram nomodeset bee.gui=off bee.wipe=all + reduced performance params |
Full standard param set (append after @APPEND_LIVE@ / nomodeset flags):
net.ifnames=0 biosdevname=0 mitigations=off transparent_hugepage=always
numa_balancing=disable pcie_aspm=off intel_idle.max_cstate=1 processor.max_cstate=1
nowatchdog nosoftlockup udev.children_max=1 intel_iommu=on
iommu.passthrough=0 efi=disable_early_pci_dma
The fail-safe entry alone adds pci=realloc iommu.strict=1. pci=realloc must
not be copied into normal entries. Memtest and firmware-setup entries are not
Linux live entries and therefore do not carry these parameters.
Historical note: grub-pc/ was mistakenly used instead of grub-efi/ until v8.25.
live-build reads config/bootloaders/grub-efi/ for UEFI because the build is
configured with --bootloaders "grub-efi,syslinux". Directory grub-pc is ignored.
Memtest rule
Do not assume live-build's built-in memtest integration is sufficient for bee.
We already tried that path and regressed again on 2026-04-01: lb binary_memtest
ran, but the final ISO still lacked memtest binaries and menu entries.
For this project, memtest is accepted only when the produced ISO actually contains all of the following:
boot/memtest86+x64.binboot/memtest86+x64.efi- a memtest entry in
boot/grub/grub.cfg - a memtest entry in
isolinux/live.cfg
Rules:
- Keep explicit post-build memtest validation in
build.sh. - Treat ISO reader success as a separate prerequisite from memtest content. If the reader cannot list or extract from the ISO, that is a validator failure, not proof that memtest is missing.
- If built-in integration does not produce the artifacts above, use a deterministic project-owned copy/extract step instead of hoping live-build will "start working".
- Do not switch back to built-in-only memtest without fresh build evidence from a real ISO.
- If you reference memtest files manually, verify the exact package file list first for the target Debian release.
Known bad loops for this repository:
- Do not retry built-in-only memtest without new evidence. We already proved
that
lb binary_memtestcan run while the final ISO still has no memtest. - Do not assume fixing memtest file names is enough. Correct names did not fix the final artifact path.
- Do not assume
chroot/boot/contains memtest payloads at the time hooks run. - Do not assume source
grub.cfg/live.cfg.inare the final writers of ISO bootloader configs. - Do not assume the current
config/hooks/normal/9100-memtest.hook.binarytiming is late enough to patch finalbinary/boot/grub/grub.cfgorbinary/isolinux/live.cfg; logs from 2026-04-01 showed those files were not present yet when the hook executed. - Do not treat a validator warning as ground truth until you have confirmed the ISO reader actually succeeded. On 2026-04-01 we misdiagnosed another memtest regression because the final ISO was correct but the validator produced a false negative.