Dataset — LAMOST DR5 × Gaia DR3

This document describes the dataset used in the PHY-3500 project and the article "Does dimensionality reduction preserve the essential physical information of stellar spectra?" (Baker, Caillat & Jean, 28 April 2026). It forms the basis of all dimensionality reduction analyses presented in this project.

N

43,019

Retained spectra

PHY-3500 · G1 · 43,019 spectra

p

183

Features

pure spectroscopic

λ

3690–9100

Spectral coverage

Ångströms

R

≈ 1800

Resolution

low resolution

---

2.1 Sources

LAMOST DR5 — Large Sky Area Multi-Object Fiber Spectroscopic Telescope

LAMOST (Cui et al., 2012) is a Chinese telescope dedicated to large-scale multi-object spectroscopy. The fifth data release (Data Release 5) covers:

Parameter	Value
Spectral coverage	3690–9100 Å
Spectral resolution	R ≈ 1800 (low resolution)
Total number of spectra in DR5	> 9 million
Number of channels per spectrum (raw flux)	3,921

Spectra are distributed in FITS format, with a header containing observation metadata (coordinates, identifiers, LAMOST spectral type, measured redshift, SNR per band).

At this resolution, individual visual inspection is impossible: automated analysis is a necessity, not a choice. This is precisely what motivates the development of dimensionality reduction and automatic classification methods.

---

Gaia DR3 — Gaia Data Release 3

Gaia DR3 (Gaia Collaboration, Vallenari et al., 2023) provides the reference astrophysical parameters used to validate and interpret the results:

Parameter	Symbol	Unit	Role in the project
Effective temperature	T_eff	K	PC1 validation (ρ = +0.831)
Surface gravity	log g	dex	Dwarf/giant distinction
Metallicity	[Fe/H]	dex	PC2 validation
BP/RP photometry	G_BP, G_RP	mag	Photometric colours
Parallax	π	mas	Distance, absolute magnitude
Proper motion	μα, μδ	mas/yr	Tangential velocity
Extinction	A_G	mag	Reddening correction

---

Positional cross-match LAMOST × Gaia

The cross-match is performed by angular coordinates with a tolerance radius of 1 arcsecond. This radius is conservative enough to avoid false associations in sparse fields, while ensuring a high match rate for Milky Way stars.

The cross-match is managed by gaia_crossmatcher.py in the AstroSpectro pipeline. It produces a master catalogue merging LAMOST and Gaia columns for each matched spectrum.

---

2.2 Final dataset

Quality filters applied

Two filters are applied sequentially before any analysis:

Input:LAMOST DR5 candidates→→SNR→RUWE→→43,019retained

1

LAMOST DR5

Signal-to-noise ratio

SNR_r > 10

Only spectra with a red-band SNR above 10 are retained. This threshold ensures that spectral lines are detectable and that equivalent width measurements are reliable.

↓

2

Gaia DR3

Astrometric quality

RUWE < 1.4

The Gaia Renormalised Unit Weight Error (RUWE) measures the quality of the astrometric fit. A value < 1.4 guarantees a simple proper motion solution, excluding unresolved binary stars.

Final dataset: N = 43,019 spectra · p = 183 features · matrix X ∈ ℝ^(43019 × 183)

---

Removed variables

Before building the feature matrix, the following variables were explicitly removed:

Variable	Source	Reason
ra	LAMOST	Angular coordinate — observational geographic bias
dec	LAMOST	Angular coordinate — same reason
redshift	LAMOST	Instrumental metadata — non-physical bias

Feature leakage

These three variables have artificial discriminative power: they encode where and when the spectrum was observed, not the physical properties of the star. A model trained with ra or dec learns to classify stars by sky position — correlated with spectral types via LAMOST observation programs — rather than by their true physics. In the AstroSpectro pipeline, this exclusion mode is activated via spectro_only=True.

---

Final composition

99.85% stars

★

Stars42,956(99.85%)

Main population — all analyses

◎

Galaxies56(0.13%)

Too rare — excluded from HDBSCAN clustering

◉

QSO7(0.02%)

Median reconstruction error ×4.2 vs stars (autoencoder)

The vast majority of spectra being stars, dimensionality reduction and classification analyses focus on the stellar population. Galaxies and QSOs are too rare (63 combined objects) to be included in the HDBSCAN analysis in a statistically significant way.

---

The 183 features

The 183 features are distributed across six physical families. Colours correspond to the approximate spectral colours of the relevant lines:

Balmer lines

Hα – Hθ

~35

Hydrogen absorption series. Primary thermal indicator: maximum strength around 10,000 K (type A), weakens toward M and O.

Hα 6563 Å · Hβ 4861 Å · Hγ 4341 Å · Hδ 4102 Å · Hθ 3798 Å

→ PCA:PC1 (Balmer: 29.7% of top-10 |loadings|)

Ca II H&K + IR triplet

H 3968 · K 3933 · IR 8498–8662

~20

Ionised calcium in absorption. Sensitive to both metallicity ([Ca/H]) and stellar magnetic activity (emission core filling).

Ca II H 3968 Å · Ca II K 3933 Å · IR triplet 8498, 8542, 8662 Å

→ PCA:PC1 (Ca II: 9.3% of top-10 |loadings|)

Magnesium & α-elements

Mg b · Mg I 5184 · Mg II

~15

Mg b triplet and magnesium lines. Tracers of α-element enrichment — discriminates halo and Galactic disk populations.

Mg b 5167–5183 Å · Mg I 5184 Å · Mg II 4481 Å

→ PCA:PC1 (W(Mgb) = −0.171)

Iron peak metals

Fe, Cr, V, Ni, Ba, Sr...

~30

Iron peak element lines. Direct measurement of iron abundance and heavy metals. Dominate PC2.

Fe I 5270, 5335 Å · Cr I · V I · Ni I · Ba II · Sr II

→ PCA:PC2 ([Fe/H]proxy: +0.205)

Lick/SDSS/CaH indices

Lick IDS indices · CaH · TiO

~18

Standardised spectral indices. Narrow-band combinations calibrated to measure age and metallicity of stellar populations.

Lick IDS indices · CaH1, CaH2 · TiO1, TiO2 · SDSS composites

→ PCA:PC2 (metal index: +0.198)

Continuum · slopes · colours

Flux ratios · photom. indices

~65

Global spectral continuum shape. Largest family by count. Encodes temperature via blackbody slope and luminosity via spectral breaks.

fblue/fred · synthetic B−V · local slopes · photometric indices

→ PCA:PC1 (continuum: fblue/fred = +0.161)

183 features · 6 spectroscopic families · complete catalogue →

---

2.3 Standardisation

Z-score transformation

Before any analysis, each feature $j$ is standardised to z-scores:

$$\tilde{x}_{ij} = \frac{x_{ij} - \bar{x}_j}{s_j}$$

where $\bar{x}_j$ and $s_j$ are the empirical mean and standard deviation computed over all $N = 43{,}019$ spectra.

This step is essential for PCA and strongly recommended for UMAP and t-SNE.

Why standardise?

Without standardisation, features with large variance (typically raw fluxes integrated over wide windows) would artificially dominate the first principal components, regardless of their actual physical content. For example, a continuum index computed over 100 Å mechanically has more absolute variance than an equivalent width measured over 2 Å, without reflecting any difference in physical importance.

Standardisation places all features on an equal footing in terms of variance (variance = 1 after transformation), leaving PCA to discover real variance directions in physical space.

Implementation

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)   # X: (43019, 183)

Standardisation is computed only on the training set and applied without recomputation to any new spectrum to classify, in accordance with best practices to avoid information leakage (data leakage).

---

Data access

Source	URL	Format
LAMOST DR5 spectra	http://dr5.lamost.org/	FITS
Gaia DR3 archive	https://gea.esac.esa.int/archive/	VOTable / CSV

Automated download of FITS spectra is managed by dr5_downloader.py in the AstroSpectro pipeline. See the download guide for detailed instructions.

---

Related pages

◎Catalogue of 183 features

183 features

Complete physical definitions — 6 families, PCA loadings, λ positions

◎Dimensionality Reduction

PCA, UMAP, t-SNE, HDBSCAN — applied on this dataset

◎Physical interpretation of PCA

ρ = +0.831

Explained variance, eigenspectra, Gaia DR3 correlations

▶Download guide

Retrieve LAMOST DR5 spectra with SmartDownloader

▶Preprocessing guide

Median normalisation, λ reconstruction, SNR/RUWE filters