# Decision: Move pci=realloc out of the default/toram/no-GUI GRUB entries

**Date:** 2026-07-28
**Status:** active

## Context

`pci=realloc` was added to the default, toram, and no-GUI/gsp-off GRUB menu
entries by commit 8bf8dfa ("default to KMS + pci=realloc, drop nomodeset from
main entries") as a workaround for GPU BAR assignment failures on servers
where the BIOS lacks "Above 4G Decoding". It forces the Linux kernel to
fully reallocate all PCI resources instead of trusting the BIOS-assigned
addresses.

On a dual-socket AMD EPYC 9655 server (MSI CG480-S6053, 6x GPU + 8x NVMe
behind 2-port hot-plug-capable PCIe switches), this same flag caused a
different failure: the OS only exposed 2 of 8 NVMe drives, even though
`lspci` and the BIOS both enumerated all 6 controllers actually present in
the affected chassis. `dmesg` showed the kernel's bridge-window reallocation
pass exhausting available MMIO32 address space:

```
pci 0000:f2:03.0: bridge window [mem size 0x00200000 64bit pref]: can't assign; no space
pci 0000:f7:00.0: BAR 0 [mem size 0x00004000 64bit]: can't assign; no space
```

The BIOS had already assigned working BARs to all 6 NVMe controllers at
boot; `pci=realloc` threw those away and failed to re-assign 4 of them,
so the `nvme` driver never bound to those 4 (no `nvme nvmeX: pci function`
line), and they never became block devices. The dense NVMe backplane, not
the GPUs, hit the address-space ceiling that `pci=realloc` re-triggered on
every device in the topology, not just the GPUs it was meant to fix.

`pci=realloc` is a blanket flag: it was applied globally to the default and
toram entries (the ones almost every operator actually boots), so any
server with a BIOS whose GPU BARs are fine, but whose PCIe topology is
dense enough elsewhere (many NVMe bays, many switches), pays this cost
unconditionally.

## Decision

`pci=realloc` is removed from the default and toram menu entries in both
`iso/builder/build.sh` (`write_canonical_grub_cfg`) and the static
`iso/builder/config/bootloaders/grub-efi/grub.cfg` template. Rather than
adding a new single-purpose menu entry for this one workaround, it is
folded into the existing no-GUI/gsp-off entry, which is already the
troubleshooting/fail-safe-style option operators reach for when the
default entry does not work cleanly:

```
EASY-BEE v<version> -- no GUI / troubleshooting (gsp-off, pci=realloc)
```

An operator who hits either a GSP-related GPU issue or a GPU BAR
assignment failure (BIOS lacking Above 4G Decoding) selects this single
entry manually, instead of the menu growing one dedicated item per
individual failure mode. This keeps the total entry count in
`grub.cfg` unchanged from before the incident (still 3 real boot
entries plus wipe/memtest/firmware-settings), rather than letting it
grow linearly with every workaround discovered.

As of v13.0, GRUB and isolinux both expose exactly one troubleshooting entry
with `pci=realloc`. It remains absent from every normal, toram, no-GUI, and
wipe entry. The same v13.0 amendment replaces `iommu=pt` in that entry with
translated strict IOMMU mode; this changes DMA diagnostics, not the scope of
the PCI reallocation workaround.

## Consequences

- Servers with dense PCIe topologies (many NVMe bays, many GPUs) no longer
  silently lose drives to a workaround they didn't need, by default.
- Servers that genuinely need the GPU BAR workaround (BIOS lacking Above 4G
  Decoding) now require the operator to notice GPU BAR failures and
  deliberately pick the workaround entry, instead of getting it for free --
  document this tradeoff in the support runbook / vendor-facing material
  if GPU BAR failures turn out to be common enough to warrant it.
- DO NOT move `pci=realloc` back into the default/toram entries without
  first checking whether the affected server's topology can tolerate a
  full PCI reallocation pass -- this was the exact failure mode of the
  incident that prompted this decision.