Transfer benchmark

Question

Does ImageNet pretraining help on 32×32 CIFAR-10, at what compute cost, and how much of the benefit comes from the head versus the backbone?

The benchmark is a fairness anchor: three ResNet-18 runs with the same architecture, data, augmentation, budget, and evaluation. Pretraining and freezing are the changed variables.

The three runs

Config	Weights	Trainable	Budget
resnet18_scratch_cifar.yaml	none	all	30 epochs, AdamW 1e-3, cosine, early stop 10
resnet18_linear_probe_cifar.yaml	ImageNet	head only	max 30 epochs, AdamW head 1e-3, early stop 10
resnet18_finetune_cifar.yaml	ImageNet	all	30 epochs, backbone 1e-4 / head 1e-3

All three use keep_7x7_stride1 so the stem sees 32×32 inputs without changing the architecture between runs. The linear probe freezes the entire backbone, including the pretrained stem, so its features are genuinely ImageNet features. Its head has exactly 5,130 trainable parameters before training.

Normalization difference

The transfer anchor uses ImageNet mean/std because the pretrained model expects it. The scratch reference baseline uses CIFAR mean/std. That difference is labeled in the comparison, not hidden: the anchor answers a different question from the reference reproduction.

Model design

The schematic shows the linear-probe configuration: the ImageNet stem and layers 1–4 are frozen, and only the 5,130-parameter head is trainable. The fine-tune configuration keeps the same stage layout but unfreezes every block.

Result

All three runs share the CIFAR-10 test set and test fingerprint.

Run	Test accuracy	95% CI	Macro F1	Trainable params	Train time
Scratch	90.88%	90.33–91.43%	90.90%	11,181,642	19.6 min
Linear probe	62.66%	61.75–63.63%	62.54%	5,130	8.4 min
Fine-tune	94.98%	94.57–95.41%	94.97%	11,181,642	19.5 min

Paired bootstrap on the shared test predictions:

Comparison	Difference	95% CI
Fine-tune − scratch	+4.10 pp	+3.49 to +4.69 pp
Fine-tune − linear probe	+32.32 pp	+31.35 to +33.23 pp
Linear probe − scratch	−28.22 pp	−29.19 to −27.21 pp

The comparison records test fingerprints, per-run bootstrap intervals, parameter counts, training time, latency, and peak memory. BatchNorm running statistics update even while probe parameters are frozen, so the linear probe is a frozen-parameter probe, not a strict frozen-feature probe.

• Runner
• Configs
• Comparison

What to notice

• Same data and budget. The fairness anchor is structural: shared split, augmentation, epochs, and evaluation.
• Test evaluation happens after training. All three runs train first with test evaluation disabled; test is evaluated only after the suite completes.
• Compute is part of the result. A model that wins by a fraction of a point while costing much more training time is reported as such.
• No significance claim from overlap. Per-run intervals are reported; a paired test is required for a real significance claim.

Caveat

This is a single 32×32 protocol, not a universal statement about transfer learning. This site covers the 32×32 benchmark; a 224×224 study is outside its scope. Results from the reference baseline and the transfer anchor are separate and should not be merged into one leaderboard.