pages/4Logistic_Regression.py

import streamlit as st

st.set_page_config(page_title="Logistic Regression", page_icon="🤖", layout="wide")

# Updated CSS styling
st.markdown("""
    <style>
        .stApp {
            background-color: #f2f6fa;
        }
        h1, h2, h3 {
            color: #1a237e;
        }
        .custom-font, p {
            font-family: 'Arial', sans-serif;
            font-size: 18px;
            color: #212121;
            line-height: 1.6;
        }
        code {
            background-color: #e3eaf5;
            color: #1a237e;
            padding: 4px 6px;
            border-radius: 6px;
            font-size: 16px;
        }
        pre {
            background-color: #e3eaf5 !important;
            color: #1a237e;
            padding: 10px;
            border-radius: 10px;
            overflow-x: auto;
        }
    </style>
""", unsafe_allow_html=True)

# App Content
st.markdown("<h1>Logistic Regression</h1>", unsafe_allow_html=True)

st.write("""
Logistic Regression is a supervised machine learning algorithm used for classification only. It is mainly used for binary classification, but with some extensions, it can handle multi-class labels as well.
The main task of logistic regression is to find the best line, plane, or hyperplane that separates the classes linearly. It assumes the data should be (almost) linearly separable for good performance.
""")

st.markdown("<h2>1. Logistic Regression with Step Function</h2>", unsafe_allow_html=True)
st.write("""
The step function assigns an output of either 0 or 1 based on a threshold value. However, it has the following disadvantages:
- Not differentiable, which makes optimization hard.
- Cannot measure the probability of class membership.
- Small input changes don’t change the output smoothly.

To solve this, we use the Sigmoid function.
""")

st.markdown("<h2>2. Logistic Regression with Sigmoid Function</h2>", unsafe_allow_html=True)
st.write(r"""
The sigmoid function is defined as:

\[
sigmoid(z) = \frac{1}{1 + e^{-z}}
\]

where \( z = WX + b \).

**Advantages**:
- Smooth and differentiable
- Outputs probability between 0 and 1
- Great for binary classification
""")

st.markdown("<h2>Loss used in Logistic Regression</h2>", unsafe_allow_html=True)
st.write(r"""
Logistic Regression uses **cross-entropy loss**:

\[
L = -\sum_{i=1}^{N} \left[y_i \log P(y_i) + (1 - y_i) \log (1 - P(y_i)) \right]
\]
""")

st.markdown("<h2>Gradient Descent</h2>", unsafe_allow_html=True)
st.write(r"""
Gradient Descent is used to minimize the loss function:

\[
W = W - \alpha \frac{\partial L}{\partial W}
\]

Where \( \alpha \) is the learning rate.
""")

st.markdown("<h2>Learning Rate in Gradient Descent</h2>", unsafe_allow_html=True)
st.write("""
- High learning rate → faster but unstable
- Low learning rate → stable but slow
- Too low → may never converge
- Typically used: 0.1 or 0.01
""")

st.markdown("<h2>Types of Gradient Descent</h2>", unsafe_allow_html=True)
st.write("""
- **Batch GD**: Full dataset per update — few epochs, slow
- **Stochastic GD (SGD)**: One sample per update — fast per epoch, more epochs
- **Mini-batch GD**: Small batch updates — balanced and commonly used
""")

st.markdown("<h2>Multiclass Logistic Regression</h2>", unsafe_allow_html=True)
st.subheader("1. Softmax Regression")
st.write(r"""
Softmax regression generalizes logistic regression for multi-class problems.

\[
P(y = j | X) = \frac{e^{Z_j}}{\sum_{k=1}^{K} e^{Z_k}}
\]

**Steps**:
1. Compute scores: \( Z = WX + b \)
2. Apply softmax
3. Use cross-entropy loss
4. Update with gradient descent

**Example**:
If class probabilities are:

| Class | Probability |
|-------|-------------|
| 0     | 0.02        |
| 1     | 0.05        |
| 7     | 0.30        |
| 9     | 0.11        |

Prediction = **7**
""")

st.markdown("<h2>2. One-vs-Rest (OvR) Classification</h2>", unsafe_allow_html=True)
st.write("""
OvR breaks a multi-class problem into many binary ones.

**Steps**:
1. Train N binary classifiers (one per class)
2. Each says: "Is this class or not?"
3. Pick the one with the highest score

**Example**:
For 🍎 🍌 🍊, you train:
- Apple vs Not Apple
- Banana vs Not Banana
- Orange vs Not Orange
""")

st.markdown("<h2>Regularization in Logistic Regression</h2>", unsafe_allow_html=True)
st.write(r"""
Regularization adds a penalty to reduce overfitting:

- **L1 (Lasso)**: \( \lambda \sum |w| \)
- **L2 (Ridge)**: \( \lambda \sum w^2 \)
- **ElasticNet**: Combination of both

**Why?**
- Reduces model complexity
- Encourages generalization
""")

st.markdown("<h2>Detecting Multicollinearity</h2>", unsafe_allow_html=True)
st.write(r"""
**Variance Inflation Factor (VIF)**:

\[
VIF_i = \frac{1}{1 - R^2_i}
\]

- VIF > 10 = high multicollinearity
""")

st.markdown("<h2>Hyperparameters in Logistic Regression</h2>", unsafe_allow_html=True)
st.table([
    ["Hyperparameter", "Description"],
    ["penalty", "Regularization type ('l1', 'l2', 'elasticnet', None)"],
    ["dual", "Use dual formulation (for 'l2' with 'liblinear')"],
    ["tol", "Tolerance for stopping"],
    ["C", "Inverse of regularization strength"],
    ["fit_intercept", "Add intercept term or not"],
    ["intercept_scaling", "Intercept scaling (for 'liblinear')"],
    ["class_weight", "Weights for classes ('balanced' or dict)"],
    ["random_state", "Seed for reproducibility"],
    ["solver", "Optimization algorithm (e.g., 'lbfgs', 'saga')"],
    ["max_iter", "Max number of iterations"],
    ["multi_class", "Strategy ('ovr' or 'multinomial')"],
    ["verbose", "Level of output verbosity"],
    ["warm_start", "Reuse previous solution"],
    ["n_jobs", "Cores used for training"],
    ["l1_ratio", "Mix ratio (for 'elasticnet')"],
])

st.write("🚀 This app helps you understand **Logistic Regression** step by step!")