# Decision: Stream SAT job output live to disk and kick blackbox sync on job completion

**Date:** 2026-07-27
**Status:** active

## Context

Investigating a hard reboot mid-`nvbandwidth` (see
`2026-07-27-nvbandwidth-per-socket-split.md`) showed that the SAT job runner
had no way to preserve a crashing job's own output. `streamExecOutput`
(`audit/internal/platform/sat.go`) buffered a job's stdout/stderr entirely
in memory; the job's named log file (`NN-<job>.log`) was written to disk
only once, after the process exited. A crash mid-command meant that job's
file never existed on disk at all -- `verbose.log`'s `start`/`finish`
markers survived (they are appended per-line already), but the command's
own output did not.

This matters more than it would on a normal install: the live ISO boots
with `toram` (`live-boot` copies the read-only medium into RAM and mounts
an overlay on top of it -- see `runtime-flows.md`'s Boot sequence). The
export directory bee writes to lives on that RAM-backed overlay, so a
local write surviving to "disk" is not itself durable across a hard reset
-- the only real persistence boundary is `bee-blackbox.service` mirroring
the export directory out to removable media on its own adaptive schedule
(1-30s flush period). Even a perfectly-written local file is lost if
blackbox hasn't copied it off before the crash.

## Decision

Two changes, in `audit/internal/platform/sat.go` and
`audit/internal/app/blackbox.go`:

1. `streamExecOutput` takes an optional `livePath`; when set, each output
   line is written straight to that file as it arrives, in addition to the
   in-memory buffer used for the final return value.
   `runSATCommandCtx`/`runSATCommandWithMetrics` thread this through from
   `runAcceptancePackCtx`'s job loop and the storage SAT loop, pointing at
   each job's own named log file. A crash mid-command now leaves whatever
   had printed up to that point instead of nothing.
2. `platform.SetJobBoundaryHook(func(jobName string))` is called right
   after each job's output file is written (success or failure, every
   job). `app.New()` wires this to touch a marker file
   (`.blackbox-kick` under `exportDir`); `blackboxWorker.run()` polls that
   marker's mtime (`blackboxKickPollInterval`, default 250ms) alongside its
   normal adaptive timer and syncs immediately on a kick instead of waiting
   out the current flush period.

`platform` does not import `app` or know about blackbox; the job-boundary
hook is a plain `func(string)` seam, consistent with the existing
`satExecCommand`/`satStat`/`satReadFile` seams in the same file.

## Consequences

- A job that completes cleanly reaches removable media within one poll
  interval instead of waiting out whatever the adaptive flush period
  happens to be (up to 30s) -- closing most of the historical gap for
  postmortem analysis of a later crash.
- The job that is in flight when a crash happens still only has whatever
  had streamed to its live file by that point -- this does not make output
  capture instantaneous, it makes it incremental instead of all-or-nothing.
- DO NOT revert `streamExecOutput`'s per-job file write to a single
  end-of-command `os.WriteFile` -- that was the exact gap that lost the
  `nvbandwidth` crash's own output in the original incident.
- DO NOT assume a local write under the live-ISO's export directory is
  durable on its own; anything that must survive a hard reset needs
  blackbox to have copied it off first (this is why the kick exists at
  all, not just the live write).