pages/4Logistic_Regression.py

import streamlit as st

# Configure the Streamlit page
st.set_page_config(page_title="Logistic Regression", page_icon="🤖", layout="wide")

# Custom CSS for styling
st.markdown("""
    <style>
        .stApp {
            background-color: #4A90E2; /* Light blue background */
        }
        h1, h2, h3 {
            color: #003366; /* Dark blue headers */
        }
        .custom-font, p {
            font-family: 'Arial', sans-serif;
            font-size: 18px;
            color: white;
            line-height: 1.6;
        }
    </style>
""", unsafe_allow_html=True)

# Title
st.markdown("<h1 style='color:#003366;'>Logistic Regression</h1>", unsafe_allow_html=True)

# Introduction
st.write("""
Logistic Regression is a supervised machine learning method primarily used for classification tasks. While it's ideal for binary classification, it can be extended to handle multiple classes using appropriate techniques. Its main objective is to find the best linear separator—line, plane, or hyperplane—that effectively divides different classes. It assumes that the classes can be linearly separated to perform optimally.
""")

# Step Function
st.markdown("<h2 style='color:#003366;'>1. Logistic Regression with Step Function</h2>", unsafe_allow_html=True)
st.write("""
The step function outputs either 0 or 1 based on a defined threshold. However, it comes with limitations:
- Not differentiable, making it unsuitable for gradient-based optimization.
- Cannot represent probabilities.
- Unresponsive to small changes in input.

To address these drawbacks, the sigmoid function is used.
""")

# Sigmoid Function
st.markdown("<h2 style='color:#003366;'>2. Logistic Regression with Sigmoid Function</h2>", unsafe_allow_html=True)
st.write(r'''
The sigmoid function is defined as:

\[
sigmoid(z) = \frac{1}{1 + e^{-z}}\
\]

where \( z = WX + b \).

**Advantages:**
- Smooth and continuous
- Outputs values between 0 and 1 (interpretable as probabilities)
- Differentiable and suitable for optimization
''')

# Loss Function
st.markdown("<h2 style='color:#003366;'>1. Loss Function in Logistic Regression</h2>", unsafe_allow_html=True)
st.write(r'''
Logistic Regression uses the cross-entropy (log) loss:

\[
L = -\sum_{i=1}^{N} [y_i \log P(y_i) + (1 - y_i) \log(1 - P(y_i))]
\]

where \( y_i \) is the true label and \( P(y_i) \) is the predicted probability.
''')

# Gradient Descent
st.markdown("<h2 style='color:#003366;'>2. Gradient Descent</h2>", unsafe_allow_html=True)
st.write(r'''
Gradient Descent is used to optimize the weights by minimizing the loss:

\[
W = W - \alpha \frac{\partial L}{\partial W}
\]

Here, \( \alpha \) is the learning rate.
''')

# Learning Rate
st.markdown("<h2 style='color:#003366;'>3. Learning Rate in Gradient Descent</h2>", unsafe_allow_html=True)
st.write("""
- A high learning rate may overshoot the minimum and cause instability.
- A low learning rate slows down convergence.
- A balanced rate (e.g., 0.01 or 0.1) is commonly used.
- Fixed learning rates may cause oscillations if not tuned properly.
""")

# Types of Gradient Descent
st.markdown("<h2 style='color:#003366;'>4. Types of Gradient Descent</h2>", unsafe_allow_html=True)
st.write("""
- **Batch Gradient Descent**: Uses the entire dataset per update. Fewer epochs, more computation per epoch.
- **Stochastic Gradient Descent (SGD)**: Updates weights after each sample. Faster updates but noisier convergence.
- **Mini-batch Gradient Descent**: Uses batches of samples. A balance between batch and SGD—widely used in practice.
""")

# Softmax Regression
st.markdown("<h2 style='color:#003366;'>Multiclass Logistic Regression</h2>", unsafe_allow_html=True)
st.subheader("1. Softmax Regression")
st.write(r'''
Softmax Regression extends logistic regression to multiclass classification:

\[
P(y = j | X) = \frac{e^{Z_j}}{\sum_{k=1}^{K} e^{Z_k}}
\]

### Softmax Regression Steps:
1. Compute scores: \( Z = WX + b \)
2. Apply softmax to get probabilities
3. Use cross-entropy loss
4. Optimize with gradient descent
5. Choose the class with the highest probability
''')

# Example table for softmax
st.write("""**Example:**
If the model predicts probabilities for classes 0 to 9 as follows:

| Class | Probability |
|-------|-------------|
| 0     | 0.02        |
| 1     | 0.05        |
| 2     | 0.07        |
| 3     | 0.10        |
| 4     | 0.08        |
| 5     | 0.12        |
| 6     | 0.10        |
| 7     | 0.30        |
| 8     | 0.05        |
| 9     | 0.11        |

The model predicts class **7** because it has the highest probability.
""")

# One-vs-Rest
st.markdown("<h2 style='color:#003366;'>2. One-vs-Rest (OvR) Classification</h2>", unsafe_allow_html=True)
st.write("""
OvR decomposes a multiclass problem into several binary problems:

### OvR Steps:
1. Train one classifier per class
2. Each predicts whether a sample belongs to its class
3. Choose the class with the highest score

**Example:** Classify 🍎 🍌 🍊
- Classifiers: Apple vs Others, Banana vs Others, Orange vs Others
- Predict based on the highest confidence
""")

# Regularization
st.markdown("<h2 style='color:#003366;'>Regularization in Logistic Regression</h2>", unsafe_allow_html=True)
st.write("""
Regularization helps reduce overfitting by penalizing large weights:

- **L1 (Lasso)**: \( \lambda \sum |w| \), encourages sparsity
- **L2 (Ridge)**: \( \lambda \sum w^2 \), smooths the weights
- **ElasticNet**: Combines both L1 and L2 penalties

Regularization leads to simpler, more generalizable models.
""")

# Multicollinearity
st.markdown("<h2 style='color:#003366;'>Detecting Multicollinearity</h2>", unsafe_allow_html=True)
st.write(r'''
High correlation among features can affect performance.

### Variance Inflation Factor (VIF):

\[
VIF_i = \frac{1}{1 - R^2_i}
\]

- VIF > 10 indicates multicollinearity
- Lower VIF is preferred
''')

# Hyperparameters
st.markdown("<h2 style='color:#003366;'>Hyperparameters in Logistic Regression</h2>", unsafe_allow_html=True)
st.table([
    ["penalty", "Regularization type ('l1', 'l2', 'elasticnet', None)"],
    ["dual", "Use dual formulation (for 'l2' with 'liblinear')"],
    ["tol", "Stopping tolerance"],
    ["C", "Inverse of regularization strength"],
    ["fit_intercept", "Add intercept term or not"],
    ["intercept_scaling", "Scaling of intercept (liblinear only)"],
    ["class_weight", "Class weights ('balanced' or dict)"],
    ["random_state", "Random seed"],
    ["solver", "Optimization algorithm ('lbfgs', etc.)"],
    ["max_iter", "Max iterations"],
    ["multi_class", "'ovr' or 'multinomial'"],
    ["verbose", "Verbosity"],
    ["warm_start", "Reuse previous solution"],
    ["n_jobs", "CPU cores used"],
    ["l1_ratio", "Ratio for ElasticNet"]
])

st.write("This interactive guide helps you understand logistic regression from basics to advanced concepts! 🚀")