Update pages/4Logistic_Regression.py

pages/4Logistic_Regression.py

@@ -0,0 +1,201 @@
+import streamlit as st
+
+# Configure the Streamlit page
+st.set_page_config(page_title="Logistic Regression", page_icon="🤖", layout="wide")
+
+# Custom CSS for styling
+st.markdown("""
+    <style>
+        .stApp {
+            background-color: #4A90E2; /* Light blue background */
+        }
+        h1, h2, h3 {
+            color: #003366; /* Dark blue headers */
+        }
+        .custom-font, p {
+            font-family: 'Arial', sans-serif;
+            font-size: 18px;
+            color: white;
+            line-height: 1.6;
+        }
+    </style>
+""", unsafe_allow_html=True)
+
+# Title
+st.markdown("<h1 style='color:#003366;'>Logistic Regression</h1>", unsafe_allow_html=True)
+
+# Introduction
+st.write("""
+Logistic Regression is a supervised machine learning method primarily used for classification tasks. While it's ideal for binary classification, it can be extended to handle multiple classes using appropriate techniques. Its main objective is to find the best linear separator—line, plane, or hyperplane—that effectively divides different classes. It assumes that the classes can be linearly separated to perform optimally.
+""")
+
+# Step Function
+st.markdown("<h2 style='color:#003366;'>1. Logistic Regression with Step Function</h2>", unsafe_allow_html=True)
+st.write("""
+The step function outputs either 0 or 1 based on a defined threshold. However, it comes with limitations:
+- Not differentiable, making it unsuitable for gradient-based optimization.
+- Cannot represent probabilities.
+- Unresponsive to small changes in input.
+
+To address these drawbacks, the sigmoid function is used.
+""")
+
+# Sigmoid Function
+st.markdown("<h2 style='color:#003366;'>2. Logistic Regression with Sigmoid Function</h2>", unsafe_allow_html=True)
+st.write(r'''
+The sigmoid function is defined as:
+
+\[
+sigmoid(z) = \frac{1}{1 + e^{-z}}\
+\]
+
+where \( z = WX + b \).
+
+**Advantages:**
+- Smooth and continuous
+- Outputs values between 0 and 1 (interpretable as probabilities)
+- Differentiable and suitable for optimization
+''')
+
+# Loss Function
+st.markdown("<h2 style='color:#003366;'>1. Loss Function in Logistic Regression</h2>", unsafe_allow_html=True)
+st.write(r'''
+Logistic Regression uses the cross-entropy (log) loss:
+
+\[
+L = -\sum_{i=1}^{N} [y_i \log P(y_i) + (1 - y_i) \log(1 - P(y_i))]
+\]
+
+where \( y_i \) is the true label and \( P(y_i) \) is the predicted probability.
+''')
+
+# Gradient Descent
+st.markdown("<h2 style='color:#003366;'>2. Gradient Descent</h2>", unsafe_allow_html=True)
+st.write(r'''
+Gradient Descent is used to optimize the weights by minimizing the loss:
+
+\[
+W = W - \alpha \frac{\partial L}{\partial W}
+\]
+
+Here, \( \alpha \) is the learning rate.
+''')
+
+# Learning Rate
+st.markdown("<h2 style='color:#003366;'>3. Learning Rate in Gradient Descent</h2>", unsafe_allow_html=True)
+st.write("""
+- A high learning rate may overshoot the minimum and cause instability.
+- A low learning rate slows down convergence.
+- A balanced rate (e.g., 0.01 or 0.1) is commonly used.
+- Fixed learning rates may cause oscillations if not tuned properly.
+""")
+
+# Types of Gradient Descent
+st.markdown("<h2 style='color:#003366;'>4. Types of Gradient Descent</h2>", unsafe_allow_html=True)
+st.write("""
+- **Batch Gradient Descent**: Uses the entire dataset per update. Fewer epochs, more computation per epoch.
+- **Stochastic Gradient Descent (SGD)**: Updates weights after each sample. Faster updates but noisier convergence.
+- **Mini-batch Gradient Descent**: Uses batches of samples. A balance between batch and SGD—widely used in practice.
+""")
+
+# Softmax Regression
+st.markdown("<h2 style='color:#003366;'>Multiclass Logistic Regression</h2>", unsafe_allow_html=True)
+st.subheader("1. Softmax Regression")
+st.write(r'''
+Softmax Regression extends logistic regression to multiclass classification:
+
+\[
+P(y = j | X) = \frac{e^{Z_j}}{\sum_{k=1}^{K} e^{Z_k}}
+\]
+
+### Softmax Regression Steps:
+1. Compute scores: \( Z = WX + b \)
+2. Apply softmax to get probabilities
+3. Use cross-entropy loss
+4. Optimize with gradient descent
+5. Choose the class with the highest probability
+''')
+
+# Example table for softmax
+st.write("""**Example:**
+If the model predicts probabilities for classes 0 to 9 as follows:
+
+| Class | Probability |
+|-------|-------------|
+| 0     | 0.02        |
+| 1     | 0.05        |
+| 2     | 0.07        |
+| 3     | 0.10        |
+| 4     | 0.08        |
+| 5     | 0.12        |
+| 6     | 0.10        |
+| 7     | 0.30        |
+| 8     | 0.05        |
+| 9     | 0.11        |
+
+The model predicts class **7** because it has the highest probability.
+""")
+
+# One-vs-Rest
+st.markdown("<h2 style='color:#003366;'>2. One-vs-Rest (OvR) Classification</h2>", unsafe_allow_html=True)
+st.write("""
+OvR decomposes a multiclass problem into several binary problems:
+
+### OvR Steps:
+1. Train one classifier per class
+2. Each predicts whether a sample belongs to its class
+3. Choose the class with the highest score
+
+**Example:** Classify 🍎 🍌 🍊
+- Classifiers: Apple vs Others, Banana vs Others, Orange vs Others
+- Predict based on the highest confidence
+""")
+
+# Regularization
+st.markdown("<h2 style='color:#003366;'>Regularization in Logistic Regression</h2>", unsafe_allow_html=True)
+st.write("""
+Regularization helps reduce overfitting by penalizing large weights:
+
+- **L1 (Lasso)**: \( \lambda \sum |w| \), encourages sparsity
+- **L2 (Ridge)**: \( \lambda \sum w^2 \), smooths the weights
+- **ElasticNet**: Combines both L1 and L2 penalties
+
+Regularization leads to simpler, more generalizable models.
+""")
+
+# Multicollinearity
+st.markdown("<h2 style='color:#003366;'>Detecting Multicollinearity</h2>", unsafe_allow_html=True)
+st.write(r'''
+High correlation among features can affect performance.
+
+### Variance Inflation Factor (VIF):
+
+\[
+VIF_i = \frac{1}{1 - R^2_i}
+\]
+
+- VIF > 10 indicates multicollinearity
+- Lower VIF is preferred
+''')
+
+# Hyperparameters
+st.markdown("<h2 style='color:#003366;'>Hyperparameters in Logistic Regression</h2>", unsafe_allow_html=True)
+st.table([
+    ["penalty", "Regularization type ('l1', 'l2', 'elasticnet', None)"],
+    ["dual", "Use dual formulation (for 'l2' with 'liblinear')"],
+    ["tol", "Stopping tolerance"],
+    ["C", "Inverse of regularization strength"],
+    ["fit_intercept", "Add intercept term or not"],
+    ["intercept_scaling", "Scaling of intercept (liblinear only)"],
+    ["class_weight", "Class weights ('balanced' or dict)"],
+    ["random_state", "Random seed"],
+    ["solver", "Optimization algorithm ('lbfgs', etc.)"],
+    ["max_iter", "Max iterations"],
+    ["multi_class", "'ovr' or 'multinomial'"],
+    ["verbose", "Verbosity"],
+    ["warm_start", "Reuse previous solution"],
+    ["n_jobs", "CPU cores used"],
+    ["l1_ratio", "Ratio for ElasticNet"]
+])
+
+st.write("This interactive guide helps you understand logistic regression from basics to advanced concepts! 🚀")