Skip to main content

Roadmap

This page is a living document. It lists what the pipeline does today, what is planned, and what is known to be unfinished. Contributions that tackle any open item are welcome: see Contributing.

How the statuses were set

A box is ticked only if the corresponding code exists in the AstroSpectro repository (src/, app.py, .devcontainer/, .github/). The repository also keeps a more detailed ROADMAP.md, in French; some of its boxes are older than the code and may not match it.

Axis 1 — Scientific content​

Extract more, and more reliable, information from each spectrum.

  • Base measurements on six lines (Hα, Hβ, Ca II K and H, Mg b, Na D): prominence, FWHM and equivalent width.
  • Line ratios and spectral indices (among them TiO5, Dn4000, G4300, CaH2, CaH3).
  • Extended extractor: molecular bands, extended Balmer series, metal lines, extended indices, continuum analysis, line profiles, synthetic colours and composite indices. See Feature engineering theory.
  • Gaia DR3 cross-match and derived quantities (distance, absolute magnitudes, tangential velocity).
  • spectro_only mode, which removes non-spectroscopic columns by name. See Spectro-only mode and the Data Leakage Audit for its present limits.
  • SHAP interpretability (notebook 04_shap_interpretability). See SHAP interpretability.
  • Anomaly detection: intersect the HDBSCAN noise points with the spectra of highest autoencoder reconstruction error.
  • Characterise the HDBSCAN clusters against rare objects (variables, binaries, red-giant-branch stars).

Axis 2 — Models and data​

Explore better models and richer data handling.

  • XGBoost as the main model, with RandomForest as an alternative.
  • Other estimators selectable in SpectralClassifier: LightGBM, CatBoost, ExtraTrees, SVM, k-NN, MLP, soft voting and others. See the Classifier API.
  • Grid and randomised hyper-parameter search with cross-validation.
  • Probability calibration (sigmoid, isotonic) and an Expected Calibration Error helper.
  • Dimensionality reduction and clustering: PCA, UMAP, t-SNE, HDBSCAN, and a spectral autoencoder (PyTorch) in src/dimred/. See the dimensionality reduction section.
  • Opt-in Weights & Biases logging (use_wandb). See Experiment tracking.
  • Partial: class-imbalance oversampling (SMOTE and variants) is integrated as an option of SpectralClassifier; the default is no oversampling.
  • Optuna-based search: not in the code today; hyper-parameters are searched with GridSearchCV or RandomizedSearchCV.
  • Extension to other LAMOST releases or to SDSS (see multi-survey support below).
  • 1D convolutional network on the raw flux: planned, not implemented.

Axis 3 — Infrastructure​

Make the pipeline more robust, more accessible and easier to use.

  • Modular layout (src/pipeline/, src/dimred/, src/tools/) packaged with pyproject.toml. See Project structure.
  • GitHub Codespaces dev container: no local installation needed.
  • Documentation deployed to GitHub Pages with GitHub Actions.
  • Streamlit application (app.py).
  • Parallel feature extraction: a thread pool for reading files, a process pool for the CPU-bound stage. See Processing pipeline.
  • Timestamped artefacts for every run: model pickle plus JSON metadata.
  • Automated tests: the repository has no test suite yet.
  • REST API for predictions (a /predict endpoint).
  • Workflow orchestration (Snakemake or Prefect) to chain download, preprocessing, features, classification and reporting.
  • Multi-survey support (SDSS, Gaia RVS).

Upcoming releases​

An earlier version of this page gave Q3 2026 as the target for v1.1. That date has passed and is not repeated here.

v1.1​

  • Optuna hyper-parameter optimisation, tracked with Weights & Biases sweeps.
  • Scale the processing to the full set of downloaded FITS files, once the Gaia cross-match issue below is solved.
  • Archive the canonical model artefact with its full metadata.
  • Submit the manuscript (see Citing AstroSpectro).

v1.2​

  • Extension to a later LAMOST release or to SDSS.
  • Anomaly-detection pipeline (HDBSCAN noise intersected with high reconstruction error).
  • Identification of binary-star candidates.

Known issues​

  • Gaia cross-match instability blocking the large-scale extension.
  • Feature-count discrepancy: resolved in the documentation, model artefact still to be regenerated.
  • Optuna is not integrated: searches use GridSearchCV.

See also​