Split bee-bench into perf and power workflows
This commit is contained in:
@@ -1,5 +1,34 @@
|
||||
# Benchmark clock calibration research
|
||||
|
||||
## Benchmark methodology versioning
|
||||
|
||||
Every benchmark methodology change must bump the benchmark version constant in
|
||||
source code by exactly `+1`.
|
||||
|
||||
Methodology change means any change that affects comparability of benchmark
|
||||
results, including for example:
|
||||
- phase durations or phase order
|
||||
- enabled/disabled precisions
|
||||
- fallback rules
|
||||
- normalization rules
|
||||
- score formulas or weights
|
||||
- degradation thresholds
|
||||
- power calibration logic
|
||||
- thermal/power penalty logic
|
||||
|
||||
Requirements:
|
||||
- benchmark version must be stored in source code as an explicit version
|
||||
constant, not inferred from git tag or build metadata
|
||||
- benchmark report must always print the benchmark version
|
||||
- `result.json` must always include the benchmark version
|
||||
- results from different benchmark versions must be treated as non-comparable by
|
||||
default
|
||||
|
||||
Purpose:
|
||||
- prevent accidental comparison of runs produced by different methodologies
|
||||
- make historical benchmark archives self-describing even when detached from git
|
||||
- force deliberate version bumps whenever scoring or execution semantics change
|
||||
|
||||
## Status
|
||||
In progress. Baseline data from production servers pending.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user