fix(archive): open zips with a missing or truncated central directory
A BEE-SP v14.23 blackbox (Dell PowerEdge R760) streamed from the live CD was
cut off inside the central directory: all 1920 members intact, no
end-of-central-directory record, so archive/zip rejected the whole file
("zip: not a valid zip file"). ADL-079:
- when archive/zip cannot open a .zip, read it front to back by local file
headers; intact archives are unaffected
- streamed members (data descriptors) are delimited from the data itself,
deflate and stored; every member is CRC32/size checked
- recovery stops at the first damaged, truncated or encrypted member and
keeps the entries before it; the original error is returned when nothing
could be read
- same 10 MB per-file / 1 GB total limits as the regular zip path
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
b4b15e58a2
commit
1cc2faa567
5 files changed
+521
No files matched your search
@@ -20,6 +20,10 @@ extension: `.tar`/`.sds`, `.tar.gz`/`.tgz`/`.gz`, `.zip`, `.rar`, `.ahs`, and pl
|
||||
volume, including solid archives. Password-protected archives, multi-volume sets, self-extracting
|
||||
archives and the legacy RAR 4.x format are rejected with an explicit error.
|
||||
|
||||
A `.zip` that `archive/zip` rejects (central directory missing or cut off, e.g. a streamed
|
||||
blackbox whose download was interrupted in the trailer) is read front to back by its local file
|
||||
headers (ADL-079); members are CRC-checked and recovery stops at the first damaged one.
|
||||
|
||||
`VendorParser` contract:
|
||||
|
||||
```go
|
||||
|
||||
@@ -2328,3 +2328,46 @@ and no parser matched the content.
|
||||
now parses as `linux-generic` with partial inventory instead of failing with "no parser found".
|
||||
- Not parsed yet: `ip`/`ethtool` (MAC addresses), `nvidia-smi` (GPU serials), `ipmitool`
|
||||
(FRU/sensors). Each is one more entry in the utility table.
|
||||
|
||||
## ADL-079 — A zip without a usable central directory is read by its local file headers
|
||||
|
||||
**Date:** 2026-10-02
|
||||
**Context:** A BEE-SP v14.23 blackbox of a Dell PowerEdge R760 (1XWT5J4) could not be opened:
|
||||
`open zip: zip: not a valid zip file`. The live CD streams the zip over HTTP, and the stream was
|
||||
cut while the trailer was being sent — the file ends 39 bytes into the 1251st of 1920 central
|
||||
directory records, with no end-of-central-directory record. All 1920 members (local header,
|
||||
deflate data, data descriptor) are intact. `archive/zip` reads an archive only through the
|
||||
end-of-central-directory record, so it rejects the whole file; so does Info-ZIP `unzip`.
|
||||
|
||||
**Decision:**
|
||||
- When `archive/zip` cannot open a `.zip`, `extractZip` / `extractZipFromReader` fall back to
|
||||
`recoverZipLocalEntries` (`internal/parser/archive_zip_recover.go`), which reads the archive
|
||||
front to back by local file headers and ignores the central directory. If the fallback
|
||||
recovers no entry, the original `archive/zip` error is returned unchanged.
|
||||
- The fallback is used only after `archive/zip` fails. For an intact archive the central
|
||||
directory stays authoritative.
|
||||
- Streamed members (general-purpose flag bit 3) have no sizes in the local header, so the end of
|
||||
a member is found from the data: deflate is self-terminating; a stored member ends at the first
|
||||
data descriptor signature whose CRC32 and size match the bytes read so far. Every recovered
|
||||
member is verified against its CRC32 and size (data descriptor or local header).
|
||||
- Recovery stops at the first thing that is not an intact local entry — the central directory, a
|
||||
truncated or CRC-mismatching member, an encrypted member, a streamed member in a compression
|
||||
method other than store/deflate — and keeps the entries before it.
|
||||
- Limits match the regular zip path: members over 10 MB are skipped, extraction stops at 1 GB
|
||||
total, the upload path keeps its 50 MB archive cap.
|
||||
- Member mtime comes from the extended-timestamp extra field (UTC), else the DOS time, the same
|
||||
as `archive/zip`.
|
||||
|
||||
**Consequences:**
|
||||
- Truncated streamed blackbox bundles open as long as the member data arrived; the cut-off
|
||||
trailer costs nothing.
|
||||
- An archive cut inside the member data opens with the members before the cut and no warning:
|
||||
the extractor has no channel for "archive incomplete" (`ExtractedFile.Truncated` is per-file and
|
||||
is reported as "input too large"). Parsers see a partial bundle.
|
||||
- Local headers can disagree with the central directory in a crafted or appended-to archive
|
||||
(deleted or superseded members remain as local entries). This is why the fallback never runs
|
||||
for an archive `archive/zip` accepts.
|
||||
- Not covered: nested zips opened by parsers themselves (`dell`, `redfishtree`),
|
||||
`internal/sanitize` and `internal/server/raw_export.go` still require an intact archive.
|
||||
- The producer-side bug (live CD closing the stream before the zip trailer is flushed) is not
|
||||
fixed by this and should be fixed in bee.
|
||||
Reference in new issue
Block a user