Files
Mikhail ChusavitinandClaude Sonnet 5 b8c45d54c1 feat(iso): split the live medium into semantic SquashFS layers
Booting via BMC virtual CD reads the ~2.8 GB filesystem squashfs
sequentially during the live-boot toram copy; a mid-read drop of the
redirected medium loses the whole copy and fails the boot (v14). Split
the rootfs into self-contained semantic layers so a retry re-reads at
most one ~500-700 MiB layer, not everything. This is a resilience /
reduced-re-read mechanism, not a fix for the virtual-media instability.

NVIDIA variants now ship 7 layers (00-base, 05-firmware, 08-desktop,
10-nvidia-driver, 20-nvidia-platform, 30-nvidia-cuda-libs,
40-nvidia-dcgm-cuda) plus an explicit live/filesystem.module that fixes
their OverlayFS order; amd/nogpu keep a single squashfs.

- lib/squashfs-layers.sh: deterministic classifier (dpkg file ownership
  plus explicit rules for build.sh-injected files, never a path
  substring), per-layer mksquashfs, 800 MiB hard ceiling, unsquashfs -s
  plus strict extraction of every layer, merged-rootfs bootability check.
- build.sh: split the monolith after the full lb build, verify and merge,
  write the module file, delete the monolith only then; abort before ISO
  assembly on any failure. Runs the builder test suites up front.
- fast-path: force a full build for a multi-layer medium;
  fast_path_repack_squashfs hard-refuses (it would drop layers).
- iso-validation.sh: validate_iso_squashfs_layers (module vs layer set
  match, size ceiling, no lone giant squashfs) and
  validate_iso_media_integrity (xorriso -check_media).
- bee-install: honour filesystem.module order, abort on any layer failure.
- 9013-toram-retry: record the real rsync exit code (it printed a false
  rc=0) and correct the "resumes the tail" comment (rsync without
  --partial keeps only fully-copied layers). No unsafe partial resume.
- tests: test-squashfs-layers.sh plus a multi-layer guard in
  test-build-libs.sh; both run at the top of every build.
- docs: bible-local architecture and decision, iso/README, iso-build-rules.

Verified by a full nvidia build: 7 layers 622/199/256/466/37/567/562 MiB,
every validator passes, xorriso -check_media good, merged rootfs bootable.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-04 15:38:14 +03:00
..

bee — Project Bible

Project-specific architecture, decisions, and runtime contracts. Generic engineering rules live in bible/rules/patterns/.

Files

File Contents
architecture/system-overview.md What bee does, scope, tech stack
architecture/runtime-flows.md Boot sequence, audit flow, service order
architecture/squashfs-layers.md Semantic SquashFS layer model, ownership, order, verification
docs/customer-gpu-test-methodology.md Customer-facing GPU PCIe Validate / Validate -> Stress test list
docs/hardware-ingest-contract.md Current Reanimator hardware ingest JSON contract
docs/validate-vs-burn.md Validate and Validate -> Stress hardware test policy
decisions/ Architectural decision log, including read-only submodule policy
proposals/ RFCs and contract change proposals for Reanimator Core

Validate Test Matrix

Validate

  • CPU check
    • lscpu
    • sensors
    • stress-ng
  • TPM check (read-only)
    • tpm2_getcap properties-fixed
    • tpm2_getcap pcrs
    • tpm2_pcrread
    • tpm2_gettestresult (reads the existing result; does not start TPM2_SelfTest)
  • Memory check
    • free
    • timeout <timeout_sec> memtester
    • free
  • NVMe storage check
    • nvme id-ctrl
    • nvme smart-log
    • nvme device-self-test
  • SATA/SAS storage check
    • smartctl -H -A
    • smartctl -t short
  • Basic NVIDIA GPU check
    • nvidia-smi -pm 1
    • nvidia-smi -q
    • dmidecode -t baseboard
    • dmidecode -t system
    • dcgmi diag -r 2
  • Inter-GPU communication check
    • all_reduce_perf
  • GPU bandwidth check
    • dcgmi diag -r nvbandwidth (per CPU socket, then all selected GPUs, on multi-socket systems -- see decisions/2026-07-27-nvbandwidth-per-socket-split.md)

Validate -> Stress

  • Extended NVIDIA GPU check
    • nvidia-smi -pm 1
    • nvidia-smi -q
    • dmidecode -t baseboard
    • dmidecode -t system
    • dcgmi diag -r 3
  • NVIDIA targeted stress
    • nvidia-smi -pm 1
    • nvidia-smi -q
    • dcgmi diag -r targeted_stress
  • NVIDIA targeted power
    • nvidia-smi -pm 1
    • nvidia-smi -q
    • dcgmi diag -r targeted_power
  • NVIDIA pulse test
    • nvidia-smi -pm 1
    • nvidia-smi -q
    • dcgmi diag -r pulse_test
  • Inter-GPU communication check
    • all_reduce_perf
  • GPU bandwidth check
    • dcgmi diag -r nvbandwidth (per CPU socket, then all selected GPUs, on multi-socket systems -- see decisions/2026-07-27-nvbandwidth-per-socket-split.md)
  • Fan ceiling check (Load tier / 3. Load only)
    • stressapptest (or stress-ng) + the hottest GPU load — dcgmproftester -t 1004 / targeted_power (Power/Thermal Fit engine), not bee-gpu-burn — run simultaneously
    • ipmitool sdr type Fan on an adaptive interval (1 s floor, backs off to 30 s when the BMC gets slow, every read time-boxed) until every fan plateaus (~1 min flat, only trusted while telemetry is healthy) or the 15 min cap
    • records each fan's observed peak RPM to the fan-observation store (used by the Topology fan tiles: size ∝ ceiling, fill ∝ duty cycle); FAIL only on a fan at 0 RPM / IPMI cr-nr under load; cancelled ("not applicable"), never failed, if the host cannot be loaded or has no fan sensors
    • see decisions/2026-09-04-fan-ceiling-check.md