Skip to main content

DINOv2 — Fine-tuned

Key results​

83.9%
Balanced accuracy
test set, 2,660 images
86.5%
Accuracy
test set
84.7%
Best validation
epoch 6 of 10
86.6
Parameters (M)
all trainable
Provenance. Model scores in this section are copied from the output cells of the AstroVision notebooks (and, for DINOv2 per-class recall, read from an exported figure). The documentation data pack contains no metrics file for them, so they cannot be re-derived from the pack; the cell of origin is listed for each value in _facts/astrovision/FACTS_INDEX.md §3 and shown when hovering over a number.

Recipe​

ItemValue
BackboneDINOv2 ViT-B/14 (torch.hub, dinov2_vitb14), CLS token → LayerNorm → Dropout(0.3) → Linear(768→10)
Phase 1linear probe on cached features (about 20 s) initialises the head — validation balanced accuracy 55.0%
Phase 2full fine-tuning, 10 epochs, learning rate 5 × 10⁻⁶, weight decay 1 × 10⁻⁴, batch size 32, FP16 mixed precision
Augmentationresize 256, random crop 224, flips, rotation 180°, colour jitter

The learning rate is deliberately very low: large rates destabilise a ViT that is already well initialised. The head is initialised from the probe so the first epochs do not spend their gradient on a random classifier. The class DINOv2Classifier is defined in notebook 04 (not in src/astrovision/models.py).

Per-class recall​

Recall per class, DINOv2 fine-tuned (test)
0 · Disturbed53%
1 · Merging86%
2 · Round Smooth94%
3 · In-between Round Smooth95%
4 · Cigar Shaped Smooth72%
5 · Barred Spiral87%
6 · Unbarred Tight Spiral83%
7 · Unbarred Loose Spiral78%
8 · Edge-on without Bulge96%
9 · Edge-on with Bulge93%

Most classes exceed 0.85 recall. The weak ones are Disturbed (0.53, often confused with loose spirals and in-between smooth galaxies), Cigar Shaped Smooth (0.72, confused with In-between Round Smooth) and Unbarred Loose Spiral (0.78). Per-class values are read from the exported confusion matrix (two decimals).

What changes inside the network​

Before fine-tuning, attention of the CLS token is partly drawn to bright background stars; afterwards it covers the galaxy structure (qualitative, figure below; see Attention & Grad-CAM).

◎
Confusion matrix
Confusion matrix
Row-normalised recall, test set.
◎
Comparison with other models
Comparison with other models
Balanced accuracy of the benchmark models.
◎
Attention before / after fine-tuning
Attention before / after fine-tuning
CLS attention of the zero-shot and fine-tuned backbone.
Reading the test score

The validation maximum (84.7%, epoch 6) is higher than the final test balanced accuracy (83.9%); the validation curve is noisy (it dips in the middle of training), so differences of one to two points between models on 2,660 test images should not be over-interpreted. One training run per model, no seeds repeated.