Key Results at a Glance

Run context

Classifier: XGBoost ST_350k_xgb_AFGKM_v6 · saved 20260522T161236Z · spectro_only mode · G2, 349,998 spectra processed · classes A / F / G / K / M.

Dimensionality reduction: PHY-3500 · G1 · 43,019 spectra (a different, earlier extraction — the two are never combined in a metric).

⚠v6 · includes 4 Gaia/photometric features — see Data Leakage Audit

Classification performance

Held-out test set of 60,947 spectra, reference run (v6, G2).

Metric	Value	Notes
Balanced accuracy	89.2%	5-class: A, F, G, K, M
Accuracy	87.4%	Plain accuracy — not the same metric as balanced accuracy
F1 (macro)	0.883	Unweighted mean over the five classes
ROC-AUC (macro)	0.986	One-vs-rest
F/G boundary confusion	Expected	Continuous physical transition at 5 500–6 200 K

Run v6 includes four Gaia/photometric features, so these metrics are not purely spectroscopic; see the Data Leakage Audit.

Dimensionality reduction

PHY-3500 · G1 · 43,019 spectra — these analyses were run on the earlier PHY-3500 extraction, not on the data of the classifier above.

Method	Key result
PCA	91 components → 95 % variance · PC1: ρ(T_eff) = +0.831
UMAP	Harvard sequence emerges without labels · 20 HDBSCAN clusters
t-SNE	60× more stable than UMAP (Procrustes, init='pca')
Autoencoder	Lower MSE than PCA at K = 2 · interpretable latent arithmetic

⭐ Central scientific finding (SHAP)

Key finding

Current SHAP results for the reference model suggest that a small set of spectroscopic features carries most of the learned classification structure, alongside classical Balmer temperature indicators.

• 96.8% of the total mean |SHAP| comes from spectroscopic features; 29 of the 30 most important features are spectroscopic (the rest of the attribution belongs to Gaia/photometric columns)
• Top-ranked features: feature_Mg_b_eq_width (rank 1), feature_index_CaH2 (rank 2), feature_slope_deep_red (rank 3); feature_CaIIK_eq_width is rank 4
• This contrasts with PCA, where Balmer lines dominate PC1 (the temperature axis)

⚠v6 · includes 4 Gaia/photometric features — see Data Leakage Audit

Physical interpretation: PCA maximises variance → temperature dominates. XGBoost minimises classification error → metallicity-sensitive indices help discriminate between adjacent spectral classes. These two results are complementary, not contradictory.

Historical pipeline progression

Balanced accuracy of successive runs (the earlier 100k runs processed 99,999 spectra; v6 processed 349,998):

Run	Experiment	Balanced accuracy	Notes
20260306T011253Z	ST_100k_xgb_AFGKM_v3	86.4%	100k spectra
20260306T013341Z	ST_100k_xgb_AFGKM_v4	86.4%	100k, spectro_only
20260306T020848Z	ST_100k_xgb_AFGKM_v5	87.3%	100k, spectro_only + isotonic calibration
20260522T161236Z	ST_350k_xgb_AFGKM_v6	89.2%	Reference run, 350k spectra, isotonic calibration

The "87 %" quoted in older material is the balanced accuracy of the v5 row above; the reference run is v6. Earlier runs also kept Gaia/photometric columns among their selected features, so this progression does not isolate a purely spectroscopic model. Earlier figures for the pre-2026 pipeline and the claimed effect of removing ra, dec and redshift are not backed by the exported run reports and are no longer quoted here.