Version ledger

Markdown: versions.md · Data: versions.json

LoRA fine-tunes of unsloth/gemma-3-4b-it as a closed-set command router (one JSON tool call per utterance), trained and served on 8 GB NVIDIA consumer cards, 13-25 Sep 2026.

Version ledger

One row per trained model or variant. Test sets differ between eras, so compare accuracy only within one test set. Each version links to its full record below.

Test-set eras

The test set grew over the program. A number from one era does not compare with a number from another.

Definitions

Accuracy definition

passed / runnable cases. A case passes when the router returns the asserted capability and arguments, and none of the case's forbidden capabilities.

Held out definition

cases with no training label in the corpus the model was trained on. The held-out slice changed size as the case file grew, so compare held-out numbers only within one test-set era.

Fair slice definition

cases carrying no train key at all, which neither compared model ever saw (used for v16 and v17).

Guard definition

must-not-fire cases: utterances that must NOT route to a named capability (negations, quoted or pasted content, vague phrasing). Counts only; case text is excluded.

Speed definition

mean seconds per case through the production routing prompt (~5.6k tokens), model fully on GPU unless noted.

Dates

UTC. measured_on is the file timestamp of the result, or the date the program notes records.

Shared recipe

see recipe.json; per-version config below lists only what differs or what the training log confirms.

Per-version records

v1

Ollama tag

gemma3-4b-router-finetuned

base model

unsloth/gemma-3-4b-it

Corpus examples

152

trained

Start: 2026-09-13; End: 2026-09-14T00:26Z

what changed

First LoRA, on 152 curated examples mostly mined from the untuned router's own logged decisions plus a first batch of hand-authored gold cases.

outcome

Regressed: 4.9 pp below the untuned 12B baseline (85.3%). Not caught until v2's bake, because v1 was never evaluated after training.

Config

Rank

16

Alpha

16

RsLoRA

no

Dropout

0

Epochs

3

Optimizer steps

57

Lr

0.0002

Measured on

2026-09-14

Training

Train loss

0.3685

Peak VRAM MB

4926

Runtime s

4616

Measured on

2026-09-14

Results

v2

Ollama tag

gemma3-4b-router-v2

base model

unsloth/gemma-3-4b-it

Corpus examples

490

trained

End: 2026-09-14T07:08Z

what changed

Corpus tripled with hand-authored gold cases weighted toward vague and ambiguous phrasing and full per-capability coverage.

outcome

First fine-tune to beat the untuned baseline: +4.3 pp over gemma4:12b-it-qat on the same 550 cases.

Config

Rank

16

Alpha

16

RsLoRA

no

Dropout

0

Epochs

3

Optimizer steps

186

Lr

0.0002

Measured on

2026-09-14

Training

Final segment train loss

0.0274

Peak VRAM MB

5053

Relaunches

9

Note

train_loss covers only the last resumed segment, not the run

Measured on

2026-09-14

Results

v3

Ollama tag

gemma3-4b-router-v3

base model

unsloth/gemma-3-4b-it

Corpus examples

696

trained

End: 2026-09-14T23:57Z

what changed

240 new gold cases for the capabilities v2 missed most; full corpus, fresh from base.

outcome

Held: same score as v2 on a set 240 cases harder. Not comparable to v2's number (different test set).

Config

Rank

16

Alpha

16

RsLoRA

no

Dropout

0

Epochs

3

Optimizer steps

261

Lr

0.0002

Measured on

2026-09-14

Training

Train loss

0.1502

Peak VRAM MB

5040

Runtime s

21162

Relaunches

0

Measured on

2026-09-14

Results

v4

Ollama tag

gemma3-4b-router-v4

base model

v3's adapter (continued, not fresh from base)

Corpus examples

59

trained

End: 2026-09-15T02:32Z; Conflict: The program notes and the training script's docstring date the run 2026-09-16; file timestamps (train log 09-15 02:32, bake summary 09-15 02:54) say 2026-09-15. Trust the file timestamps.

what changed

Incremental patch: only the 59 new or corrected examples, 8 epochs, continued from v3's adapter.

outcome

Catastrophic forgetting. Capabilities the patch never touched collapsed to 25-56%. Never deployed.

Config

Rank

16

Alpha

16

RsLoRA

no

Dropout

0

Epochs

8

Optimizer steps

64

Lr

0.0002

Continued from

v3 adapter

Measured on

2026-09-15

Training

Train loss

0.0522

Peak VRAM MB

4889

Runtime s

4778

Measured on

2026-09-15

Results

v5

alias

marvin-v1.0

Ollama tag

gemma3-4b-router-v5

base model

unsloth/gemma-3-4b-it

Corpus examples

757

trained

End: 2026-09-15T11:32Z

what changed

v4's 59 examples mixed into the full corpus and retrained fresh from base.

outcome

Repaired v4's damage (run_agent, resume_agent, dispatch_code, calendar_availability all back to 100% on the 790 subset). Deployed 15 Sep.

Config

Rank

16

Alpha

16

RsLoRA

no

Dropout

0

Epochs

3

Optimizer steps

285

Lr

0.0002

Measured on

2026-09-15

Training

Train loss

0.0729

Peak VRAM MB

5117

Kills by low memory monitor

4

Note

train_loss and runtime cover the last resumed segment

Measured on

2026-09-15

Results

v5-q4

Ollama tag

gemma3-4b-router-v5-q4

base model

same merged weights as v5

what changed

Quantization only.

outcome

Rejected: -3.2 pp for ~15% speed. Q8_0 stays.

Config

Quant

Q4_0

GGUF size GB

2.37

Measured on

2026-09-16

Results

v6-wide

Ollama tag

gemma3-4b-router-v6-wide

base model

unsloth/gemma-3-4b-it

Corpus examples

757

trained

End: 2026-09-17T01:46Z

what changed

Rank 16 to 64 (alpha kept equal to rank). Same 757 examples, same recipe, fresh from base.

outcome

+4.0 pp over v5 from rank alone. The two passes a day apart (93.1% / 93.9%) set the noise floor for this era.

Config

Rank

64

Alpha

64

RsLoRA

no

Dropout

0

Epochs

3

Optimizer steps

285

Lr

0.0002

Measured on

2026-09-17

Training

Train loss

0.1227

Peak VRAM MB

5796

Runtime s

23944

Measured on

2026-09-17

Results

v7

alias

marvin-v1.1

Ollama tag

gemma3-4b-router-v7-r112

base model

unsloth/gemma-3-4b-it

Corpus examples

757

trained

End: 2026-09-17T23:06Z

what changed

Rank 64 to 112, the highest rank that fit on the 8 GB card.

outcome

+0.5 pp over v6-wide, inside v6-wide's own day-to-day spread. Shipped 18 Sep, replaced by v9 on 19 Sep.

Config

Rank

112

Alpha

112

RsLoRA

no

Dropout

0

Epochs

3

Optimizer steps

285

Lr

0.0002

Measured on

2026-09-17

Training

Train loss

0.1251

Peak VRAM MB

6647

Runtime s

24217

Measured on

2026-09-17

Results

v8

Ollama tag

gemma3-4b-router-v8-r112

base model

unsloth/gemma-3-4b-it

Corpus examples

1050

trained

End: 2026-09-18T11:19Z

what changed

Enriched corpus: 126 negative-guard cases given targets, setup context rendered into prompts (27 context cases had trained context-free), clarify-chain turns exported, 171 gold cases including the 9 capabilities that had zero examples.

outcome

Mixed, not deployed. Guards rose 19-50 pp, but four new clarify labels on bare confirmation words contradicted older gold and dropped gate_approve from 100% to 70%.

Config

Rank

112

Alpha

112

RsLoRA

no

Dropout

0

Epochs

3

Optimizer steps

396

Lr

0.0002

Measured on

2026-09-18

Training

Train loss

0.1117

Peak VRAM MB

6643

Runtime s

34881

Measured on

2026-09-18

Results

v9-step175

Ollama tag

gemma3-4b-router-v9-r112-ckpt175

base model

unsloth/gemma-3-4b-it

Corpus examples

1080

what changed

Same run as v9, earlier checkpoint.

outcome

Rejected. Tied the final checkpoint on held-out, lost 6.5 pp overall. Eval loss on partial-argument targets does not select for accuracy.

Config

Rank

112

Alpha

112

RsLoRA

no

Dropout

0

Checkpoint step

175

Of steps

405

Selected by

lowest held-out eval loss (0.1306)

Measured on

2026-09-19

Results

v9

alias

marvin-v1.2

Ollama tag

gemma3-4b-router-v9-r112

base model

unsloth/gemma-3-4b-it

Corpus examples

1080

Eval examples

55

trained

End: 2026-09-19T04:41Z

what changed

v8's corpus with the conflicting labels removed and 35 counterweight cases added; held-out eval loss logged every 25 steps, every checkpoint kept.

outcome

Deployed 19 Sep. First fine-tune to beat the untuned 12B on held-out once the harness was fixed (91.7% vs 88.9%).

Config

Rank

112

Alpha

112

RsLoRA

no

Dropout

0

Epochs

3

Optimizer steps

405

Lr

0.0002

Measured on

2026-09-19

Training

Train loss

0.1115

Peak VRAM MB

6643

Runtime s

38218

Eval loss best

Step: 175; Loss: 0.1306

Eval loss final

0.1861

Measured on

2026-09-19

Results

v10-r64

Ollama tag

gemma3-4b-router-v10-r64

base model

unsloth/gemma-3-4b-it

Corpus examples

1080

trained

End: 2026-09-20T06:03Z

what changed

v9's exact data and recipe at rank 64. Rank the only variable.

outcome

-1.4 pp vs v9 at rank 112 on the same data; clarify 86% to 73%. Rank 112 stays.

Config

Rank

64

Alpha

64

RsLoRA

no

Dropout

0

Epochs

3

Optimizer steps

405

Lr

0.0002

Measured on

2026-09-20

Training

Train loss

0.1103

Peak VRAM MB

5767

Runtime s

37684

Eval loss best

Step: 125; Loss: 0.1372

Eval loss final

0.1839

Measured on

2026-09-20

Results

v11

alias

marvin-v1.3 (25 Sep 06:42 to 08:10 UTC, then re-pointed to v17)

Ollama tag

gemma3-4b-router-v11-r112

base model

unsloth/gemma-3-4b-it

Corpus examples

1167

Eval examples

72

trained

Start: 2026-09-20T13:27Z; End: 2026-09-21T01:18Z

what changed

Corpus 1,167: new compound and correct_last capabilities, 121 context cases, clarify chains, 40 dispatch_answer positives.

outcome

Best and most stable accuracy of the program. Won the accuracy rule against six challengers (v12, v13, v15, v16, v17 and both averages). Released as marvin-v1.3, then replaced the same morning by v17 on guard behaviour.

Config

Rank

112

Alpha

112

RsLoRA

no

Dropout

0

Epochs

3

Optimizer steps

438

Lr

0.0002

Measured on

2026-09-21

Training

Train loss

0.1073

Peak VRAM MB

6648

Runtime s

42633

Eval loss best

Step: 75; Loss: 0.2727

Eval loss final

0.3997

Measured on

2026-09-21

Results

v12-e2

Ollama tag

gemma3-4b-router-v12-r112-e2

base model

unsloth/gemma-3-4b-it

Corpus examples

1167

trained

End: 2026-09-21T14:55Z

what changed

v11's exact data and recipe at 2 epochs. Epochs the only variable.

outcome

-0.8 pp and -4 held-out cases vs v11, mostly clarify (wrong answers went to converse). Lowest final eval loss of any run, and still worse. Three epochs stays.

Config

Rank

112

Alpha

112

RsLoRA

no

Dropout

0

Epochs

2

Optimizer steps

292

Lr

0.0002

Measured on

2026-09-21

Training

Train loss

0.1417

Peak VRAM MB

6648

Runtime s

28440

Eval loss best

Step: 125; Loss: 0.2589

Eval loss final

0.2934

Measured on

2026-09-21

Results

v11+grammar

Ollama tag

gemma3-4b-router-v11-r112 (same weights)

what changed

Decoding only.

outcome

+0.1 pp once the enum was correct. Turned on in production with the 25 Sep release.

Config

Change

Ollama structured output: JSON schema with a capability enum enforced at decode time

Measured on

2026-09-21

Results

v11-avg

Ollama tag

gemma3-4b-router-v11-avg

what changed

No training; weight averaging only.

outcome

Noise: +0.4 pp on the first bake, +0.0 on the second; held-out identical both times.

Config

Method

exact mean of v11 checkpoints 300, 325, 350, 375, 400, 425, 438 by concatenating LoRA A and B along the rank axis (r = 7 x 112, alpha unchanged)

Measured on

2026-09-21

Results

v13-rslora-a112 (killed)

what changed

rsLoRA with alpha left equal to rank.

outcome

Killed early. Eval loss flatlined; scale 10.6 is too hot at lr 2e-4.

Config

Rank

112

Alpha

112

RsLoRA

yes

Effective scale

10.6

Dropout

0

Lr

0.0002

Measured on

2026-09-21

Training

Eval loss step25

0.5734

Eval loss step50

0.577

V11 eval loss same steps

0.35, 0.33

Train loss peak

3.85

Measured on

2026-09-21

v13

Ollama tag

gemma3-4b-router-v13-r112-rslora-a32

base model

unsloth/gemma-3-4b-it

Corpus examples

1172

Eval examples

74

trained

End: 2026-09-22T06:35Z

what changed

rsLoRA at alpha 32 on the 21 Sep corpus (1,172 examples: 6 new two-turn-back context cases, 2 label fixes, 18 conflicts excluded).

outcome

Rejected. Highest overall number of the program, but every gained case was train-flagged and it lost held-out safety guards v11 passes.

Config

Rank

112

Alpha

32

RsLoRA

yes

Effective scale

3.02

Dropout

0

Epochs

3

Optimizer steps

441

Lr

0.0002

Measured on

2026-09-22

Training

Train loss

0.1246

Peak VRAM MB

6653

Runtime s

42946

Eval loss best

Step: 100; Loss: 0.2519

Eval loss final

0.3497

Measured on

2026-09-22

Results

v13-avg

Ollama tag

gemma3-4b-router-v13-avg

what changed

No training; weight averaging only.

outcome

-0.4 pp vs v13 with one more held-out miss. Did not recover the guard losses.

Config

Method

exact checkpoint mean, with B scaled by a further 1/sqrt(n) because rsLoRA scales by alpha/sqrt(r)

Measured on

2026-09-22

Results

v13-ck225

Ollama tag

gemma3-4b-router-v13-ck225

what changed

Mid-run checkpoint.

outcome

Only the delta to v14-ck225 means anything; absolute scores are for a half-trained model.

Config

Checkpoint step

225

Of steps

441

Dropout

0

Note

about 1.5 epochs; control arm of the dropout A/B

Measured on

2026-09-23

Results

v14

Ollama tag

gemma3-4b-router-v14-ck225 (only the step-225 checkpoint was deployed)

base model

unsloth/gemma-3-4b-it

Corpus examples

1172

trained

Attempts: 2026-09-22T20:16Z, 2026-09-23T01:23Z, 2026-09-23T03:06Z

what changed

v13's recipe plus LoRA dropout 0.1.

outcome

Would not fit at rank 112. At equal steps dropout added 3.2 pp, all on train-flagged cases, and widened the train-to-held-out gap from 1.2 to 6.8 pp. Guards improved 148 to 157.

Config

Rank

112

Alpha

32

RsLoRA

yes

Dropout

0.1

Epochs

3

Planned steps

441

Lr

0.0002

Measured on

2026-09-23

Training

Died at steps

184, 236, 239

Causes

CUDA illegal memory access (Xid 31 MMU fault per the program notes), CUDA OOM, 6.98 GiB allocated + 394 MiB reserved of 7.65 GiB, CUDA OOM, 6.90 GiB allocated + 457 MiB reserved

Measured on

2026-09-23

Results

v15

Ollama tag

gemma3-4b-router-v15-r96-drop10

base model

unsloth/gemma-3-4b-it

Corpus examples

1172

trained

End: 2026-09-23T17:51Z

what changed

Rank 112 to 96 to pay for dropout's memory; alpha 30 to hold rsLoRA's effective scale near v13's.

outcome

Vetoed on guards. Dropout bought one held-out case over v13 and cost 0.7 pp overall. Closed the rsLoRA branch.

Config

Rank

96

Alpha

30

RsLoRA

yes

Effective scale

3.06

Dropout

0.1

Epochs

3

Optimizer steps

441

Lr

0.0002

Measured on

2026-09-23

Training

Train loss

0.1218

Peak VRAM MB

6629

Predicted peak MB

6793

Runtime s

43420

Eval loss best

Step: 300; Loss: 0.2499

Eval loss final

0.3385

Measured on

2026-09-23

Results

v16

Ollama tag

gemma3-4b-router-v16-disjoint

base model

unsloth/gemma-3-4b-it

Corpus examples

873

trained

Start: 2026-09-23T22:18Z; End: 2026-09-24T07:12Z

what changed

v11's recipe on a disjoint corpus with a quarter of the gold set held back.

outcome

-8.5 pp on the fair slice vs v11. Disproved its own premise: v11's headline was not inflated by memorization.

Config

Rank

112

Alpha

112

RsLoRA

no

Dropout

0

Epochs

3

Optimizer steps

330

Lr

0.0002

Holdout

273-case stratified slice (25%) plus 58 cases dropped because a deterministic pre-router step now answers them

Measured on

2026-09-24

Training

Train loss

0.1316

Peak VRAM MB

6653

Runtime s

32031

Eval loss best

Step: 100; Loss: 0.2603

Eval loss final

0.3877

Measured on

2026-09-24

Results

v17

alias

marvin-v1.3 (production from 2026-09-25 08:10 UTC)

Ollama tag

gemma3-4b-router-v17-full

base model

unsloth/gemma-3-4b-it

Corpus examples

1189

Eval examples

78

trained

Start: 2026-09-24T17:10Z; End: 2026-09-25T05:38Z

what changed

v11's exact recipe on the largest corpus: 86 adversarial guards, 41 session-capability cases, 33 guard training targets and 6 conflict rulings added.

outcome

Lost the pre-agreed fair-slice rule by 3 cases (90 vs 93), so the scripted decision picked v11. Then chosen deliberately for guards: 256/267 vs 250/267 over three repeated passes. Live as marvin-v1.3 from 25 Sep.

Config

Rank

112

Alpha

112

RsLoRA

no

Dropout

0

Epochs

3

Optimizer steps

447

Lr

0.0002

Holdout

about 10%

Measured on

2026-09-25

Training

Train loss

0.1105

Peak VRAM MB

6658

Runtime s

44847

Eval loss best

Step: 100; Loss: 0.2526

Eval loss final

0.4378

Measured on

2026-09-25

Results