Skip to main content

DINOv2 — Linear Probe

DINOv2 ViT-B/14 (86.6 M parameters, 12 blocks, 12 heads, patch size 14) was trained by self-supervision on natural images. The probe asks: without any astronomical training, how much morphology is linearly decodable from its features?

Protocol​

  1. Resize each image to 224×224 and take the CLS token (768-dimensional) of the frozen backbone for all images; cache in dinov2_features.npz.
  2. Fit a logistic-regression probe on the training split; evaluate on the test split.

Key results​

55.5%
Balanced accuracy
linear probe · test set
63.6%
Accuracy
linear probe · test set
59.5%
SimpleCNN, for comparison
trained from scratch
Provenance. Model scores in this section are copied from the output cells of the AstroVision notebooks (and, for DINOv2 per-class recall, read from an exported figure). The documentation data pack contains no metrics file for them, so they cannot be re-derived from the pack; the cell of origin is listed for each value in _facts/astrovision/FACTS_INDEX.md §3 and shown when hovering over a number.
A measurable domain gap

Frozen DINOv2 features give 55.5% balanced accuracy, below a small CNN trained from scratch (59.5%). Natural-image features alone are not enough for galaxy morphology; fine-tuning recovers and exceeds the CNN baselines.

The same features nevertheless show structure without supervision: neighbours in feature space are often of the same class, and a UMAP of DINOv2 features separates the classes more cleanly than a UMAP of raw pixels (qualitative, figures below). The first principal components also correlate with morphometry and colour — see Physical Interpretation.

Which number is "the probe"?

Three values circulate: 0.5545 (test set, notebook 03, retained here), 0.550 (validation set, used as head initialisation in notebook 04) and a rounded 0.555 in the project README.

◎
UMAP of DINOv2 features
UMAP of DINOv2 features
Zero-shot CLS features, coloured by class.
◎
Pixels vs DINOv2
Pixels vs DINOv2
Same UMAP on raw pixels and on DINOv2 features.
◎
Nearest neighbours
Nearest neighbours
Visually similar galaxies in feature space.
◎
Attention heads
Attention heads
CLS attention of the 12 heads.
◎
Attention by class
Attention by class
Zero-shot attention maps per class.
◎
Probe vs baselines
Probe vs baselines
Balanced accuracy comparison.