AstroSpectro
Interpretable machine-learning pipeline for stellar spectral classification.
LAMOST DR5 × Gaia DR3 · 183 engineered features · XGBoost · TreeSHAP
When a modern ML classifier is trained on large-scale stellar spectroscopy, which physical indicators actually drive its decisions?
AstroSpectro was built to answer this — with physics-motivated descriptors, a reproducible pipeline, and post-hoc SHAP interpretability on N ≈ 350k LAMOST DR5 × Gaia DR3 spectra.
AstroSpectro is actively evolving. Current metrics, SHAP rankings, and model comparisons are intentionally centralised in the dedicated results, model-card, and claims-register pages — not here. This introduction page is designed to stay stable as the project grows.
Scientific purpose
AstroSpectro exists to investigate a deceptively simple question: when a machine learning classifier is trained on a large stellar spectroscopic survey, which physical indicators actually determine its decisions?
The classical MK spectral classification system places effective temperature — traced by Balmer series strength — at the centre of the A/F/G/K/M hierarchy. Modern surveys like LAMOST DR5 now make it possible to train classifiers on hundreds of thousands of spectra and then interrogate those models via post-hoc interpretability methods. AstroSpectro does exactly this, with four explicit scientific goals:
- Classify stellar spectra into five Harvard types (A, F, G, K, M) with calibrated confidence, using only features derivable from the spectrum itself.
- Extract physically meaningful descriptors that encode astrophysical observables — line strengths, profile shapes, molecular band indices — rather than raw flux pixels.
- Validate what the model learns against independent stellar parameters from Gaia DR3, confirming or challenging the astrophysical coherence of the classification structure.
- Avoid observational shortcuts: positional, temporal, and survey-specific metadata
are excluded at the architectural level via
spectro_only = True.
What AstroSpectro does
The pipeline operates in six stages, from raw FITS files to astrophysical interpretation.
- 01LAMOST DR5 spectraSmartDownloader
Raw .fits.gz spectra are downloaded from the LAMOST DR5 public archive. The SmartDownloader module implements idempotent, resumable parallel downloading organised by observation plan — progress is preserved across sessions.
LAMOST DR5 → - 02Preprocessing
Each spectrum is wavelength-normalised onto a common λ grid (3,690–9,100 Å, 3,921 channels), continuum-divided, and subjected to quality filters (SNRg threshold, Gaia RUWE filter). Spectra failing quality criteria are rejected before any features are computed.
Preprocessing → - 03Feature engineering183 descriptors
The FeatureEngineer module extracts 183 spectroscopic descriptors spanning six physical families: Balmer lines, Ca II H&K and IR triplet, Mg b and α-elements, iron-peak metals, Lick/SDSS band indices, and continuum slopes. Each descriptor carries a direct astrophysical interpretation before the classifier ever sees it.
Feature engineering → - 04Classifier trainingspectro_only = True
An XGBoost gradient-boosted classifier is trained on the feature matrix with spectro_only = True enforced throughout. The calibrated model outputs class-conditional probability vectors, and every run is logged to Weights & Biases with a reproducible session report and MD5 hash.
Classification → - 05Dimensionality reductionPCA · UMAP · t-SNE · AE
Independently of the supervised pipeline, the same 183-feature matrix is explored via PCA, UMAP + HDBSCAN, t-SNE, and a symmetric autoencoder. This multi-method analysis reveals what physical structure the high-dimensional feature space contains and whether different projection methods recover equivalent information.
Dimensionality reduction → - 06Scientific interpretationTreeSHAP
TreeSHAP is applied post-hoc to the trained classifier to decompose every prediction into per-feature contributions. The global importance ranking is then cross-validated against Gaia DR3 stellar parameters — T_eff, log g, [Fe/H] — to assess the physical coherence of what the model has learned.
SHAP Interpretability →
Project philosophy
The classifier ingests only spectroscopic descriptors — line strengths, profiles, and indices derived from the stellar spectrum itself. Positional coordinates, redshift, and Gaia metadata are excluded at training time. The model must earn its accuracy from physics alone.
A high accuracy score that cannot be explained by astrophysics is not sufficient. Every classification decision should be traceable to known stellar physics via SHAP values, and the resulting rankings should be validated against independent stellar parameters from Gaia DR3.
Observational shortcuts — features that correlate with labels for non-physical reasons — are tracked and excluded explicitly. A Data Leakage Audit documents every exclusion decision, and the spectro_only mode enforces the boundary at runtime.
Every training run is logged to Weights & Biases and produces a JSON session report containing hyperparameters, feature lists, and a model MD5 hash. Results are never stated without a pointer to the specific experiment that produced them.
Scientific findings are tracked in a dedicated Claims Register with an explicit evidence level — suggests vs. confirms — and a pointer to the analysis that supports each claim. No finding is promoted to the documentation without it.
Pipeline at a glance
Three-tier architecture
The project is organised in three tiers that build on each other and can be read independently.
Tier 1 — Core pipeline Preprocessing → Feature engineering → Classification
The foundational tier: from raw FITS spectra to a trained, calibrated classifier.
It covers the SmartDownloader, the FeatureEngineer (183 descriptors), the
GaiaCrossmatcher (Gaia DR3 positional match), and the SpectralClassifier
(XGBoost, isotonic calibration, SHAP output).
spectro_only = True is the central design constraint of this tier.
See: Core Pipeline
Tier 2 — Dimensionality reduction PCA · UMAP · t-SNE · Autoencoder · HDBSCAN
An independent multi-method exploration of the 183-feature matrix in low dimensions. This tier covers the algorithmic foundations of each method, their hyperparameter sensitivity, their Procrustes stability, and the physical interpretation of their axes and clusters. It constitutes the analytical core of the PHY-3500 course project (Baker, Caillat & Jean, Université Laval, 2026).
Tier 3 — Scientific validation SHAP · Physical cross-checks · Claims register
Post-hoc interpretability and astrophysical validation of the classifier's decisions. This tier contains the SHAP global importance analysis, per-class beeswarm plots, feature interaction matrices, physical cross-validation against Gaia DR3 parameters, and the Scientific Claims Register that tracks every finding with its evidence level.
See: Scientific Analysis
Research sections
Where to find current results
The current SHAP analysis suggests that metallicity-sensitive spectroscopic descriptors, including Mg b and Ca II features, play a major role in the learned classification structure — above classical Balmer temperature indicators.
This finding is tracked in the dedicated SHAP and Claims Register pages, which are the authoritative sources for all quantitative claims.