Act I: The 32% Baseline — A Proof of Concept​

Our starting point was a simple model trained on 5,000 spectra with only four features: the presence or absence of the Hα, Hβ, and CaII H & K lines. The result, 32%, was more than three times better than chance and validated our pipeline. However, analysing the results revealed two major weaknesses:

1. a strong class imbalance,
2. features that were too often absent, especially Hα.

(For more details, see my first results post).

Act II: The Feature Engineering Era — Toward 75%​

The roadmap was clear: we needed to enrich our features. Each new idea was tested methodically:

1. From Presence to Strength (Prominence): Measuring line depth raised accuracy to 40%.
2. The Power of Ratios: Adding strength ratios between lines (e.g. CaII K / Hβ) pushed the score to 50%.
3. Context Through Metadata: Including FITS header metadata (redshift, SNR, seeing) produced a spectacular breakthrough, reaching 71%.
4. Line Shape (FWHM & Skewness): Adding the full width at half maximum (FWHM) further refined the model. We also tested line asymmetry (skewness). While FWHM proved to be a highly relevant indicator, skewness did not improve performance. The analysis showed that this measure was too sensitive to residual noise, even after smoothing, so the model learned to ignore it. This is an important lesson: not all physical features are necessarily useful for an AI model.
5. Robustness Through Cross-Validation: Integrating GridSearchCV made it possible to find the best hyperparameters and stabilize performance around a very solid 75%.

Each step confirmed the same principle: the more physical and contextual information we gave the model, the better it performed.

The Key Tool: Interactive Debugging​

This progress would not have been possible without the visualization tools developed in parallel. Our Augmented Spectrum Analyzer became the core of our research process, allowing us to:

• Visualize any spectrum in our collection.
• Tune peak-detection parameters in real time.
• Compare a star's “true” class with the model prediction live.

This tool allowed us to identify that our initial detection parameters were too strict, causing us to miss crucial information.

Act III: The XGBoost Challenger — The Breakthrough to 84%​

With a rich and robust feature set, it was time to test a more powerful model. We replaced the RandomForest with XGBoost, a gradient boosting algorithm renowned for its performance.

The result was immediate and spectacular.

• Cross-Validation Score: 82.3%
• Final accuracy on the test set: 84%

--- Evaluation Report (XGBoost) ---
             precision  recall   f1-score   support
A              0.85      0.79      0.82        29
F              0.79      0.79      0.79        98
G              0.82      0.80      0.81       143
K              0.77      0.85      0.81       101
M              0.94      0.82      0.88        57
N              0.97      1.00      0.99        69

accuracy                           0.84       497
macro avg      0.86      0.84      0.85       497
weighted avg   0.84      0.84      0.84       497

The final confusion matrix shows a dominant diagonal, the sign of an extremely confident and accurate model. Confusion between adjacent classes was minimized, and performance is excellent across all major stellar types.

Anatomy of the Decision: Which Features Matter Most?​

The feature-importance plot for our final XGBoost model reveals a fascinating hierarchy. Contextual metadata such as redshift_error and the colour index dominate, but they are immediately followed by the physical measurements we extracted, such as the width (FWHM) and strength (prominence) of the Hα and Hβ lines. This confirms that the best performance is achieved by combining the observational context with the physical properties of the spectrum.

Conclusion: The Convergence of Physics and AI​

This journey from 32% to 84% is not just the story of an improving score. It demonstrates that a rigorous approach combining:

• A robust modular software architecture.
• Feature engineering guided by astrophysical domain knowledge.
• Interactive visualization tools for debugging and analysis.
• Rigorous statistical validation of models.

...can produce very high-quality results.

The AstroSpectro project has reached its main objective. The laboratory is built, the tools are sharpened, and the results are here. The adventure, of course, is only beginning. The Roadmap is still full of exciting challenges.

Perspectives​

• New models to test: CNNs, spectral transformers.
• Greater dataset diversity: extension to DR6, SDSS...
• Full pipeline automation (CI/CD).
• Streamlit app: toward an online interactive classification interface.

🚀 Want to contribute?​

Fork the project, suggest your ideas, or contact me!

Today, after several iterations of debugging, analysis, and optimization, we reached a significant milestone, increasing the model accuracy from 32% to a very respectable 56%. Here is the story of that progression.

Iteration 1: From Line Presence to Line Strength​

The first improvement consisted in moving from a binary feature (is the line present?) to a continuous measurement: the line strength, approximated by the "prominence" of the absorption peak. This change had an immediate impact, raising our overall accuracy from 32% to 40%. This step confirmed that the amount of physical information contained in the strength of a line is much richer than its mere presence.

Removal of overly rare classes: ['C']

Features used for training: ['feature_Hα', 'feature_Hβ', 'feature_CaIIK', 'feature_CaIIH']
Number of samples: 4999, Number of features: 4
> Training the model on 3749 samples...
> Model trained.

--- Evaluation Report ---
            precision    recall  f1-score   support

         A       0.58      0.64      0.61        73
         B       0.00      0.00      0.00         1
         D       0.00      0.00      0.00         1
         F       0.43      0.43      0.43       255
         G       0.43      0.35      0.38       394
         K       0.34      0.49      0.40       283
         M       0.28      0.18      0.22       158
         N       0.43      0.51      0.46        84
         W       0.00      0.00      0.00         1
         s       0.00      0.00      0.00         0

  accuracy                           0.40      1250
 macro avg       0.25      0.26      0.25      1250
weighted avg     0.40      0.40      0.40      1250

Iteration 2: The Power of Spectral Ratios​

With a solid new baseline, we introduced even more sophisticated features: strength ratios between different lines. In astrophysics, these ratios are extremely powerful temperature indicators.

• feature_ratio_CaK_Hbeta = CaII K / Hβ
• feature_ratio_Halpha_Hbeta = Hα / Hβ

Adding just these two ratios once again pushed the model forward, reaching 50% accuracy. The confusion matrix showed a clear improvement, but a feature-quality analysis revealed a critical weakness: the Hα line was missing in more than 50% of the spectra, making the ratios unreliable.

--- Loading dataset: features_20250723T225208Z.csv ---
> 250 rows with invalid or null labels removed.
> Removal of overly rare classes: ['W', 'C', 'D', 'B', 's']

Features used: ['feature_Hα', 'feature_Hβ', 'feature_CaIIK', 'feature_CaIIH', 'feature_ratio_CaK_Hbeta', 'feature_ratio_Halpha_Hbeta']
Final number of samples: 2740, Number of features: 6

--- STEP 5: Model Training and Evaluation ---
> Training the model on 2055 samples...
> Model trained.

--- Evaluation Report ---
            precision    recall  f1-score   support

         A       0.70      0.51      0.59        41
         F       0.54      0.60      0.57       157
         G       0.55      0.58      0.56       230
         K       0.41      0.40      0.41       163
         M       0.34      0.29      0.31        94

  accuracy                           0.50       685
 macro avg       0.51      0.48      0.49       685
weighted avg     0.49      0.50      0.49       685

Final Iteration: Tuning Through Visualization​

At this stage, our visualization tools became crucial. The analysis showed that our peak-detection parameter (prominence) was too strict and ignored many real but shallower lines.

Thanks to our interactive spectrum analyzer, we were able to tune this parameter in real time. An optimized value of prominence=0.23 was identified, allowing us to recover a large number of lines that had previously been ignored.

The Final Result: 56% Accuracy​

After rerunning the full pipeline with this optimized parameter on a new batch of 2607 spectra, the results exceeded our expectations:

• Overall Accuracy: 56%
• Feature Quality: The missing-value problem was almost eliminated, with all base features present in more than 95% of cases.
• Feature Relevance: Feature importance is now much better balanced. feature_Hβ and the ratio_CaK_Hbeta stand out as the most powerful indicators, but all features now contribute meaningfully.

The final confusion matrix shows a much clearer diagonal, a sign of a more confident and more accurate model, with F1 scores above 0.54 for all main classes.

--- Loading dataset: features_20250724T012617Z.csv ---
> 383 rows with invalid or null labels removed.
> Removal of overly rare classes: ['s', 'B', 'D', 'C']

Features used: ['feature_Hα', 'feature_Hβ', 'feature_CaIIK', 'feature_CaIIH', 'feature_ratio_CaK_Hbeta', 'feature_ratio_Halpha_Hbeta']
Final number of samples: 2607, Number of features: 6

--- STEP 5: Model Training and Evaluation ---
> Training the model on 1955 samples...
> Model trained.

--- Evaluation Report ---
            precision    recall  f1-score   support

         A       0.54      0.54      0.54        41
         F       0.54      0.55      0.55       140
         G       0.59      0.59      0.59       215
         K       0.59      0.58      0.58       168
         M       0.45      0.46      0.45        87
         W       0.00      0.00      0.00         1

  accuracy                           0.56       652
 macro avg       0.45      0.45      0.45       652
weighted avg     0.55      0.56      0.56       652

Conclusion and Next Adventures​

This research sprint was a striking demonstration of the power of the iterative method in data science. Starting from a simple model, and improving it step by step based on result analysis, we improved our baseline by 24 percentage points.

The AstroSpectro project now has a robust production pipeline and a suite of powerful analysis tools. The next exploration paths are already on the table:

• Enrich the features to better classify cool stars (type M) by adding molecular bands.
• Experiment with more powerful models such as XGBoost or neural networks.
• Move toward Deep Learning by using the full spectrum as input to a convolutional neural network (CNN).

The laboratory is built. The tools are sharpened. The adventure continues! 🚀

Refactoring Goals​

I undertook a complete refactor with three objectives in mind:

• Clarity: each tool must be readable, self-contained, and clearly named
• Reusability: functions must be modular and usable elsewhere
• Explorability: interactivity must make exploration more intuitive and scientific

Final Architecture​

All the logic now lives in src/tools/[visualizer.py](https://github.com/PhD-Brown/AstroSpectro/blob/main/src/tools/visualizer.py), in the form of a class:

visualizer = AstroVisualizer(paths)

Each interactive tool (header explorer, augmented spectrum, peak tuning, sky map, model inspector) becomes a simple call:

visualizer.interactive_spectrum_analyzer()
visualizer.plot_sky_coverage()

New Integrated Tools​

• Dynamic FITS Header Explorer
• Augmented Spectrum Analyzer (flux + lines + SNR + sliders)
• Interactive tuning of detection parameters (prominence, window)
• Analysis of zeros in the extracted features
• Sky coverage map (Mollweide projection)
• .pkl model inspector: hyperparameters + feature importance

Before / After​

For example, the code for displaying null features

Before:

• 500 lines of exploratory notebook (example of the null-feature exploration cell below)
• Spaghetti code that was difficult to reread

After:

• A dashboard notebook
• Each tool in its own cell, one call per tool
• A modular class importable elsewhere

Impact on the Project​

This refactor allows me to:

• Save time when iterating on features and models
• Work more like in a real lab
• Prepare the pipeline extension toward spectral cubes (SITELLE, etc.)

And above all… keep having fun. I have already spent more than 200h on AstroSpectro this summer — and I still do not feel the time passing!

Next Step​

• Add spectrum comparators
• Export analyses (PNG + PDF)
• Automatically generate a mini exploratory report
• Everything listed in the Roadmap

"Today I realize how enjoyable exploration becomes when the code disappears behind the tool."

Thanks to everyone following the AstroSpectro adventure 🚀

The First Iteration: A Simple but Honest Model​

To establish a baseline, the goal was to build the simplest possible model in order to validate the integrity of the processing chain. The strategy was as follows:

• Model : A RandomForestClassifier, a robust and well-understood algorithm.
• Features : Only four characteristics, based on the position of the best-known absorption lines Hα, Hβ, CaII H & K. The goal was to verify whether this minimal information already contained a predictive signal.

The Result: 32% Accuracy​

After training and evaluation on a test set of 1250 spectra, the model reached an overall accuracy of 32%.

Although this score may seem modest, it is an extremely promising starting point. In a classification problem with so many classes A, B, C, F, G, K, M, N, s, random prediction would only achieve about 9% correct answers. Our 32% score is therefore more than three times better than chance, which proves that our pipeline works and that the features, even simple ones, contain an exploitable signal.

Features used for training: ['feature_Hα', 'feature_Hβ', 'feature_CaIIK', 'feature_CaIIH']
Number of samples: 5000, Number of features: 4
Training the model on 3750 samples...

--- Evaluation Report ---

            precision    recall  f1-score   support

         A       0.40      0.59      0.48        79
         B       0.00      0.00      0.00         1
         C       0.00      0.00      0.00         1
         D       0.00      0.00      0.00         1
         F       0.41      0.51      0.46       239
         G       0.51      0.26      0.34       393
         K       0.37      0.24      0.29       294
         M       0.14      0.13      0.14       152
         N       0.30      0.37      0.33        89
         W       0.00      0.00      0.00         0
         s       0.00      0.00      0.00         1

  accuracy                           0.32      1250
 macro avg       0.19      0.19      0.18      1250
weighted avg     0.39      0.32      0.33      1250

But the most interesting part is not the score itself, but what the model's errors teach us.

The Analysis: The Confusion Matrix Speaks​

The confusion matrix is a true map of our model's "thinking". Two observations stand out immediately:

1. Data Imbalance: The model is unable to classify stars of type B, C, D or s, because it saw only a single example of each in the test set. This is a classic lesson in AI: a model cannot learn what it does not see.
2. "Physical Confusion": Where the model makes mistakes, it does so in a logical way. For example, it massively confuses G-type stars with F and K types. This is normal: these types are adjacent in the spectral sequence and look very similar. The model has therefore learned the stellar "geography" well, but struggles to draw precise boundaries.

The Root Cause: The Missing Features Problem​

Analysis of our extracted features revealed the heart of the problem. A graph showing the percentage of null values for each feature is quite clear:

This graph, which analyzes our entire dataset of 5000 spectra, reveals a surprising reality: the feature based on the Hα line is absent in more than 80% of cases. The Hβ and CaII H/K lines are detected more frequently, but they still remain missing in a non-negligible fraction of the observations.

The model therefore had to make its decisions based on very incomplete information. Reaching 32% under these conditions proves that the calcium lines and Hβ are powerful indicators, but also that the model is strongly limited by the difficulty of extracting a clear Hα signal with the current parameters.

Next Steps: The Hunt for Real Features Is On!​

This first result is not an endpoint, but a documented starting point. The roadmap for the next iteration is clear: we need to provide our model with better-quality "ingredients".

1. Measure Strength, Not Presence: The next version of the pipeline will no longer only determine whether a line is present, but will measure its depth (through peak prominence). This is much richer physical information.
2. Create Ratio Features: We will compute ratios between the depths of different lines. These ratios are well-known temperature indicators in astrophysics and should help the model better distinguish adjacent classes (such as F, G and K).
3. Clean the Dataset: For the next evaluation, we will focus on the classes for which we have a sufficient number of examples A, F, G, K, M in order to obtain more reliable and interpretable performance metrics.

This 32% baseline is the foundation on which we will build. Every future improvement can be measured against this starting point, demonstrating the direct impact of our feature engineering work.

Stay tuned!

It has been an incredible journey, going from a simple README.md to a full Docusaurus site, with a custom design, automated CI/CD deployment, and structured guides.

What has been accomplished:​

• An immersive homepage with a strong visual identity.
• Documentation structured by intent: Quick Start, Scientific Context, Practical Guides, etc.
• Polished pages with React/MDX for an interactive user experience.
• A fully automated deployment workflow via GitHub Actions.

This solid foundation will allow us to focus confidently on the next phase: continuously improving the pipeline and implementing the features from the Roadmap.

Thank you to everyone following the project. The journey is only just beginning!