fix(archive): open zips with a missing or truncated central directory

A BEE-SP v14.23 blackbox (Dell PowerEdge R760) streamed from the live CD was
cut off inside the central directory: all 1920 members intact, no
end-of-central-directory record, so archive/zip rejected the whole file
("zip: not a valid zip file"). ADL-079:
- when archive/zip cannot open a .zip, read it front to back by local file
  headers; intact archives are unaffected
- streamed members (data descriptors) are delimited from the data itself,
  deflate and stored; every member is CRC32/size checked
- recovery stops at the first damaged, truncated or encrypted member and
  keeps the entries before it; the original error is returned when nothing
  could be read
- same 10 MB per-file / 1 GB total limits as the regular zip path

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Mikhail ChusavitinandClaude Opus 5.5 committed 2026-10-02 15:00:39 +03:00
1 parent b4b15e58a2
commit 1cc2faa567
5 files changed
+521

No files matched your search

+4
View File
@@ -20,6 +20,10 @@ extension: `.tar`/`.sds`, `.tar.gz`/`.tgz`/`.gz`, `.zip`, `.rar`, `.ahs`, and pl
volume, including solid archives. Password-protected archives, multi-volume sets, self-extracting
archives and the legacy RAR 4.x format are rejected with an explicit error.
A `.zip` that `archive/zip` rejects (central directory missing or cut off, e.g. a streamed
blackbox whose download was interrupted in the trailer) is read front to back by its local file
headers (ADL-079); members are CRC-checked and recovery stops at the first damaged one.
`VendorParser` contract:
```go
+43
View File
@@ -2328,3 +2328,46 @@ and no parser matched the content.
now parses as `linux-generic` with partial inventory instead of failing with "no parser found".
- Not parsed yet: `ip`/`ethtool` (MAC addresses), `nvidia-smi` (GPU serials), `ipmitool`
(FRU/sensors). Each is one more entry in the utility table.
## ADL-079 — A zip without a usable central directory is read by its local file headers
**Date:** 2026-10-02
**Context:** A BEE-SP v14.23 blackbox of a Dell PowerEdge R760 (1XWT5J4) could not be opened:
`open zip: zip: not a valid zip file`. The live CD streams the zip over HTTP, and the stream was
cut while the trailer was being sent — the file ends 39 bytes into the 1251st of 1920 central
directory records, with no end-of-central-directory record. All 1920 members (local header,
deflate data, data descriptor) are intact. `archive/zip` reads an archive only through the
end-of-central-directory record, so it rejects the whole file; so does Info-ZIP `unzip`.
**Decision:**
- When `archive/zip` cannot open a `.zip`, `extractZip` / `extractZipFromReader` fall back to
`recoverZipLocalEntries` (`internal/parser/archive_zip_recover.go`), which reads the archive
front to back by local file headers and ignores the central directory. If the fallback
recovers no entry, the original `archive/zip` error is returned unchanged.
- The fallback is used only after `archive/zip` fails. For an intact archive the central
directory stays authoritative.
- Streamed members (general-purpose flag bit 3) have no sizes in the local header, so the end of
a member is found from the data: deflate is self-terminating; a stored member ends at the first
data descriptor signature whose CRC32 and size match the bytes read so far. Every recovered
member is verified against its CRC32 and size (data descriptor or local header).
- Recovery stops at the first thing that is not an intact local entry — the central directory, a
truncated or CRC-mismatching member, an encrypted member, a streamed member in a compression
method other than store/deflate — and keeps the entries before it.
- Limits match the regular zip path: members over 10 MB are skipped, extraction stops at 1 GB
total, the upload path keeps its 50 MB archive cap.
- Member mtime comes from the extended-timestamp extra field (UTC), else the DOS time, the same
as `archive/zip`.
**Consequences:**
- Truncated streamed blackbox bundles open as long as the member data arrived; the cut-off
trailer costs nothing.
- An archive cut inside the member data opens with the members before the cut and no warning:
the extractor has no channel for "archive incomplete" (`ExtractedFile.Truncated` is per-file and
is reported as "input too large"). Parsers see a partial bundle.
- Local headers can disagree with the central directory in a crafted or appended-to archive
(deleted or superseded members remain as local entries). This is why the fallback never runs
for an archive `archive/zip` accepts.
- Not covered: nested zips opened by parsers themselves (`dell`, `redfishtree`),
`internal/sanitize` and `internal/server/raw_export.go` still require an intact archive.
- The producer-side bug (live CD closing the stream before the zip trailer is flushed) is not
fixed by this and should be fixed in bee.