Swandip's Optimal Fertilizer Prediction Ensemble 🌱

🧠 Overview

This repository contains a powerful Level-3 stacked ensemble model designed for optimal fertilizer prediction. The model architecture uses over 80 diverse base learners, multiple advanced ensemble strategies, and robust feature engineering techniques to achieve state-of-the-art log loss performance.

Best LogLoss: 1.86257 (Ridge Ensemble)
Hill Climbing LogLoss: 1.86554

📊 Evaluation Metric

LogLoss was used as the primary evaluation metric to assess model performance.

🧱 Architecture

The model follows a 3-tier ensemble stacking structure:

Level 1 - Diverse Base Models

Includes over 80+ models from the following families:

AutoGluon (27 models)
MLP (x2)
XGBoost (x20) – tuned with bagging and early stopping
LGBM GBDT (x4) and LGBM GOSS
TabTransformer (x3)
Neural Networks (NNx15) – including deep tabular variants
CatBoost (x2)
HistGradientBoost (HGBx2)
YDF (Yandex Decision Forest)

Feature Engineering Highlights:

✅ Binned Features – Numerical columns transformed into bins to capture non-linear effects.
✅ All Numerical → Categorical – Applied label encoding or one-hot encoding to convert features.
✅ Data Augmentation using Train+Orig blending.

Level 2 - Intermediate Ensembles

Various ensemble strategies were applied to Level-1 predictions:

🔁 Logistic Regression (LR)
🧱 Voting Classifier
🎯 Stacking Classifier
⚖️ Weighted Ensemble
📊 Cluster Averaging
🗳️ Weighted Voting Classifier

Level 3 - Final Meta-Ensemble

🔼 Hill Climbing
🧠 Ridge Ensemble (Best Performer)

🏆 Performance Summary

Level	Description	Score (LogLoss)
1	Base models	1.94 – 1.88
2	Intermediate ensembles	1.88 – 1.87
3	Ridge & Hill Climbing	1.86257 (best)

🖼️ Model Architecture

📁 Files Included Swandip_optimal_fertilizer_model.joblib – Trained ensemble model

README.md – This file

architecture.jpg – Visual representation of the architecture

👨‍🔬 Author Swandip Singha 📬 Kaggle: @SwandipSingha

📜 License MIT License – You are free to use, modify, and distribute this work with proper credit.