Drift trials detected across four perturbation regimes.
NADIR / Technology / Validation
Validation dossier · rev 03
Scientific validation, with the boundary left on.
Measured performance across synthetic perturbations and public-motion shadow replay — split by class, sensor pack, operating point, and time-to-detection. No customer fleet PII. No certification language.
01 · Synthetic perturbation
02 · Public-motion replay
03 · Frozen thresholds
04 · Subgroup reporting
Frozen operating point · 800 labeled perturbation trials
Detected events that corresponded to injected drift.
Clean controls remaining below the action boundary.
Equal weighting of drift and clean-control performance.
Source cohort: mixed realistic + harsh synthetic perturbations, public-motion replay, 185,664 frames. Values are research benchmarks, not fleet-wide production guarantees.
01 · Discrimination and calibration
A detector is not a single accuracy number.
The ROC shows every available sensitivity trade-off. The selected shadow-mode point is frozen before evaluation; beside it, split-conformal calibration tests whether the uncertainty statement remains honest on held-out frames.
AUC 0.952. Frozen operating point: 89.5% recall at 7.0% false-positive rate.
95.79% empirical coverage against a 95% target on 2,496 held-out evaluation frames.
CALIBRATION SET 1,344 frames
EVALUATION SET 2,496 frames
FALSE CAUTION RATE 4.21%
TARGET BAND 95% ± 2 pp
02 · Subgroup behavior
The average never gets dispatched.
Cars, trucks, and buses carry different geometry and motion profiles. Sensor availability changes the evidence again. We report both cuts instead of hiding the weakest cell inside a pooled score.
TABLE 02ABy vehicle classFrozen test cohort
| Class | n | Recall | Precision | FPR | MTTD |
|---|---|---|---|---|---|
| Cars | 312 | 90.7% | 96.9% | 6.4% | 34 fr |
| Trucks | 296 | 89.5% | 97.5% | 7.1% | 39 fr |
| Buses | 192 | 87.5% | 97.1% | 7.9% | 45 fr |
TABLE 02BBy sensor packCoverage ≠ accuracy
| Available inputs | Coverage | Recall | FPR |
|---|---|---|---|
| Camera + IMU | 96.2% | 82.3% | 0.0% |
| Camera + radar | 94.1% | 89.3% | 7.0% |
| Camera + radar + IMU | 95.8% | 89.5% | 7.0% |
| Public-motion camera only | 90.3% | 78.1% | 8.3% |
Interpretation boundary. Vehicle-class and sensor-pack rows are stratified research slices used to expose performance variation; sample sizes are not large enough to establish deployment guarantees for a specific make, model, route, or weather regime.
03 · Detection latency
How long the fault stays invisible.
MTTD measures frames from perturbation onset to sustained detection. MTTFF-RE measures the operational clock from repair-order close to first flag. One isolates detector response; the other tests whether a pilot changes fleet exposure.
Severe steps surface in 3 median frames; micro-gradual drift takes 73.5.
Median 76 hours; 92.1% of labeled events land inside the 168-hour pilot target.
P50 INFERENCE0.0487 msper frame
P95 INFERENCE0.0551 msper frame
MTTFF-RE MEDIAN76 hrepair close → first flag
INSIDE TARGET92.1%≤ 168 hours
04 · Algorithm lift
Every added layer has to earn its complexity.
The comparison begins with an observable-output binary baseline, then adds residual normalization, robust joint gating, and the combined persistence policy. On the shared overlay cohort the final stack lifts recall by 40.6 percentage points and F1 by 28.9 points. A separate micro-gradual campaign then measures Page-Hinkley slope boosting: detect rate moves from 30.8% to 90.8% (+60pp) with a 25-frame earlier median flag and 0% clean-segment false alerts.
Shared 800-trial overlay cohort. Baseline refers only to observable output behavior; no claim is made about proprietary internal calibration.
05 · Shadow stream wall
What the benchmark feels like at frame rate.
Six simulated sensor panels render generated road geometry and residual traces. They are not vehicle videos, customer footage, or recorded fleet routes.
PUB-0142 · CAM + RADARNOMINAL
SYN-0881 · CAM + IMUCAUTION
PUB-0317 · CAM + RADARNOMINAL
SYN-1044 · TRI-SENSORCRITICAL
PUB-0208 · CAM ONLYNOMINAL
SYN-1160 · CAM + IMUCAUTION
Simulation clock live · deterministic canvas scenes · no external media loaded
06 · Protocol and boundary
What was tested. What was not.
Thresholds were frozen before scoring. Perturbation labels were held outside the detector. Public-motion sequences contribute motion and scene variation; synthetic injection provides known onset and magnitude. Customer telemetry is not present in this result set.
- 01Assemble motion
Public-motion sequences and deterministic trajectories establish scene dynamics without importing private fleet video.
- 02Inject perturbations
Micro-gradual, moderate, severe-step, and harsh-noisy yaw regimes provide known onset and ground truth.
- 03Freeze policy
Normalization, robust gate, persistence, and tier thresholds are locked before the evaluation cohort is scored.
- 04Report every slice
ROC, coverage, latency, vehicle class, sensor availability, clean-control FPR, and algorithm lift travel together.
Claim boundary
Research evidence for a shadow-mode monitoring layer.
This supports benchmarked detection behavior, calibrated uncertainty, subgroup inspection, and a pilot hypothesis measured with MTTFF-RE.
This does not support ASIL certification, OEM homologation, autonomous actuation, physical calibration replacement, or claims about any proprietary vendor’s internal algorithm.
Technology
Follow the method end to end.
Bring your own denominator
Run the same readout on your fleet.
A shadow pilot replaces research slices with your mix of vehicles, sensors, routes, and repair events — while keeping the same frozen success metric.