Measured on apps it never saw.

Every number here comes from held-out test sets: simulated apps the model was not trained on, and real apps run in Chromium that were kept out of training. Each recall sits next to its false-intervention rate, so you can see what you are trading. The shipped runtime is 0.2.1 with model 0.2.0 (r17-v2dT) at the balanced profile, report threshold 0.90; numbers from the previous model, 0.1.0 (r17-v2b), are labelled as such.

3 in 4

duplicate submits detected on unseen real apps, with 30% of stale overwrites and 6% of broken state

model 0.2.0, balanced (shipped)

0.13% · 0.59%

guard and heal false interventions at the shipped profile, held-out simulated apps; the targets are 0.1% and 0.5%, met at cautious (0.005% · 0.26%)

model 0.2.0

false interventions on held-out real apps, at every profile

model 0.2.0 · guard and heal

non-passive actions on correct apps in 1,760 guard and heal runs; 0 bugs introduced in 3,510 runs

compatibility matrix · runtime 0.1.0-beta.4 (shipped as 0.2.1), model 0.2.0, balanced

Previous model 0.1.0, observe mode at report threshold 0.70: 96% detection precision on the same 16,600 held-out situations, with 0.62% false detections on clean traffic. Precision for the shipped model 0.2.0 at its 0.90 threshold has not been published yet.

Results

The full picture.

Accuracy by training stage

Held-out simulated apps

8.0

37.6

91.1

78.3

90.5

81.9

Base model, no runtime dataCurriculum, 1.2M rowsSimulator pilot, 224k rowsRound 1, 448k rows

diagnosis %action %

Fig. 1 Round 1 used situation format v1. The stricter v2 format, with honest labels, scores 84.2% / 77.8% on simulated apps and 83.6% / 80.0% on real apps.

Detection on real apps it never trained on (previous model)

Model 0.1.0, not the shipped one: observe mode, report threshold 0.70, 16,600 held-out situations, precision 0.96

Duplicate submits detected76%

Stale overwrites detected43%

Broken state detected27%

False detections on clean traffic0.62%

Fig. 2 The previous model, 0.1.0, at report threshold 0.70. The shipped model 0.2.0 at its default threshold of 0.90 detects 75% of duplicate submits, 30% of stale overwrites and 6% of broken state on the same held-out apps (RESULTS.md §1). Its gates were fitted under a 2% false-detection limit on clean real-app traffic; its measured precision and false-detection rate at 0.90 have not been published.

Three settings, one trade-off

Model 0.2.0, held-out test. No false interventions on held-out real apps at any setting.

	cautious	balanced	eager
Guard, false interventions	0.005%	0.13%	0.54%
Guard, recall on real apps	1.4%	7.5%	21.0%
Heal, false interventions	0.26%	0.59%	1.84%
Heal, recall on real apps	3.8%	10.4%	24.2%
Report threshold	0.95	0.90	0.70

Fig. 3 Balanced is the default. False-intervention rates in the table are on simulated apps; the targets are 0.1% for guard and 0.5% for heal.

Precision and recall by class

Held-out simulated apps, report threshold 0.70

Class	Precision	Recall
slow	0.94	0.91
transient	0.88	0.81
failing	0.95	0.78
stale	0.87	0.57
conflict	0.89	0.54
inconsistent	0.97	0.53
duplicate	0.80	0.49

Table 1 Model 0.1.0 (r17-v2b) on the sim2e test set, report threshold 0.70. Expected calibration error on the action question 0.021–0.023; browser and PyTorch outputs agree on 223 of 223 decisions.

More caught at the same safety, each round

Heal-mode gain captured at a fixed false-intervention budget

Fig. 4 x: false-intervention budget. y: share of the available gain captured. v2dT ships as model 0.2.0.

Small enough for every device

Model file size, int8

R17, shipped (0.2.0 q8)10.2 MB

R3222.5 MB

GenClass 0.1, original57 MB

One WASM thread, 500-token situation: about 180 ms for R17 (182 ms for the shipped 0.2.0 export), 569 ms for R32. WebGPU and multi-threaded WASM are faster, and normal traffic makes no model calls.

Fig. 5 R17 matches R32's accuracy at about a third of the size and latency, so it is the default everywhere.

Never make a correct app worse

Plumbing test of the runtime with an always-passive model: real apps in headless Chromium, final page text and server state with and without the runtime installed

194 / 198

clean runs untouched by the v1 runtime, which held store writes. Not good enough: the React/Redux RealWorld app never rendered its home feed.

396 / 396

clean runs unchanged with the v2 runtime installed, which decides at the network boundary: 66 real apps × 6 seeds, always-passive model. With the real model, the compatibility matrix counts 0 non-passive actions on correct apps in 1,760 guard and heal runs.

Where it still misses: six demo apps under chaos

Model 0.2.0 on the 0.2.0 runtime code. Bug rate over the six demos, 60 chaos trials per cell; false interventions (FI) counted on 30 clean trials per mode.

Setting	off	observe	guard	heal	actions guard / heal	FI guard / heal
balanced (default)	72%	70%	72%	73%	2 / 3	0 / 0
cautious	–	–	72%	77%	0 / 0	0 / 0
eager	–	72%	73%	75%	6 / 12	2 / 0

Table 2 Nothing is healed at the shipped gates: the demos' bug rate is the same with GenClass on or off, and differences under about 5 points are noise. The compatibility matrix says the same: of 250 runs that showed a latent bug without GenClass, guard left 249 and heal 240 buggy. Today the value is in what it reports, not in what it fixes.

Observe mode on correct apps: what you see when nothing is wrong

False flags are the noise that gets a console reporter muted; here is how much to expect.

Compatibility matrix, 880 observe-mode runs on correct apps: runs with a false finding (49, 45 of them "work is firing far more often than usual" on the page that polls and fans out)5.6%

Model 0.1.0, held-out real apps: clean decisions flagged, report threshold 0.85 (305 of 8,079)3.78%

The same without one app that was flagged on a single field a quarter of the time1.05%

Model 0.1.0, held-out simulated apps: clean decisions flagged, report threshold 0.85 (149 of 10,582)1.41%

Table 3 Sources: compat/RESULTS.md and the package README's model-quality table. The shipped model 0.2.0 has not been re-measured at its 0.90 threshold. Identical lines within a minute are folded into one with a count; requests.ignore drops traffic you do not want observed and policy.thresholds.report trims the rest.

Where it is today

Precise when it acts, and it rarely acts.

GenClass is tuned to almost never be wrong when it acts. At the default setting guard acts on 7.5% of the real cases where acting would help; the eager setting reaches 21%. Each training round so far has lifted recall at the same safety level, and the main value today is in what it reports.

Target	Now (model 0.2.0, held-out test)
Guard false interventions ≤ 0.1%	missed at balanced, the default (0.13%); met at cautious (0.005%)
Heal false interventions ≤ 0.5%	missed at balanced, the default (0.59%); met at cautious (0.26%)
False interventions on held-out real apps: 0	met, every profile (0.00%)
Calibration error ≤ 0.02	0.021–0.023, model 0.1.0 (not yet published for 0.2.0)
Diagnosis accuracy ≥ 95%	84–86%, model 0.2.0
Recall on clear cases ≥ 80%	7.8%, guard at balanced, model 0.2.0
Detection precision published for the shipped model	not yet (0.96 for model 0.1.0 at threshold 0.70)

One production pilot so far (model 0.2.0, runtime 0.1.0-beta.4): a Next.js ordering site, nine fault scenarios, three repetitions in each of four modes. GenClass flagged each injected server failure, took 0 actions, intervened wrongly 0 times and added no measurable latency. The one real bug, a duplicate order after a lost-commit 502 and a re-tap, happened in every mode: detected as a transient failure, not prevented.

Raw tables: RESULTS.md, the compatibility matrix and the model card.