{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "def8e46c-7606-4991-9b29-9223ccacd54b",
      "metadata": {
        "id": "def8e46c-7606-4991-9b29-9223ccacd54b"
      },
      "source": [
        "# Project: Deep Learning - Pytorch"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "911a145c-7d01-4210-9699-0e1fef80ff19",
      "metadata": {
        "id": "911a145c-7d01-4210-9699-0e1fef80ff19"
      },
      "source": [
        "**Instructions for Students:**\n",
        "\n",
        "Please carefully follow these steps to complete and submit your project:\n",
        "\n",
        "1. **Make a copy of the Project**: Please make a copy of this project either to your own Google Drive or download locally. Work on the copy of the project. The master project is **Read-Only**, meaning you can edit, but it will not be saved when you close the master project. To avoid total loss of your work, remember to make a copy.\n",
        "\n",
        "2. **Completing the Project**: You are required to work on and complete all tasks in the provided project. Be disciplined and ensure that you thoroughly engage with each task.\n",
        "   \n",
        "3. **Creating a Google Drive Folder**: Each of you must create a new folder on your Google Drive. This will be the repository for all your completed project files, aiding you in keeping your work organized and accessible.\n",
        "   \n",
        "4. **Uploading Completed Project**: Upon completion of your project, make sure to upload all necessary files, involving codes, reports, and related documents into the created Google Drive folder. Save this link in the 'Student Identity' section and also provide it as the last parameter in the `submit` function that has been provided.\n",
        "   \n",
        "5. **Sharing Folder Link**: You're required to share the link to your project Google Drive folder. This is crucial for the submission and evaluation of your project.\n",
        "   \n",
        "6. **Setting Permission to Public**: Please make sure your Google Drive folder is set to public. This allows your instructor to access your solutions and assess your work correctly.\n",
        "\n",
        "Adhering to these procedures will facilitate a smooth project evaluation process for you and the reviewers."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "d3b9e0f7-75d2-476a-9b29-07c1a3800925",
      "metadata": {
        "id": "d3b9e0f7-75d2-476a-9b29-07c1a3800925"
      },
      "source": [
        "## Project Description\n",
        "\n",
        "The Deep Learning Projects are divided into two parts, the first is the Calculations worth 30% and the second one is Pytorch Project worth 70% in this notebook.\n",
        "\n",
        "The two projects will help you gain experience to learn about Deep Learning in detail.\n",
        "\n",
        "In this project, you will use what you learn to create your own Deep Learning model. We'll use a variety of datasets, each with different data types such as images, text, and numerical/categorical data. Your task is to choose one of these datasets and build a deep learning model upon it.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "d333f8fe-3ae6-4060-b597-e9f7d9e37582",
      "metadata": {
        "id": "d333f8fe-3ae6-4060-b597-e9f7d9e37582"
      },
      "source": [
        "## Datasets\n",
        "\n",
        "Choose one from the following datasets for your project:\n",
        "\n",
        "1. **Digits Dataset**: A simplified version of MNIST containing 8x8 images of hand-written digits. [Dataset Link](https://scikit-learn.org/stable/auto_examples/datasets/plot_digits_last_image.html)\n",
        "2. **Dogs vs. Cats Dataset from Kaggle**: A sizable dataset of 25,000 images with equal representation of dogs and cats. [Dataset Link](https://www.kaggle.com/c/dogs-vs-cats)\n",
        "3. **Breast Cancer Wisconsin (Diagnostic) Dataset**: Computed features from a digitized image of a fine needle aspirate (FNA) of a breast mass. [Dataset Link](https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+%28Diagnostic%29)\n",
        "4. **Spam Text Message Classification Dataset**: A collection of labeled SMS messages, categorized as \"spam\" or \"ham\". [Dataset Link](https://www.kaggle.com/uciml/sms-spam-collection-dataset)\n",
        "5. **German Credit Risk Dataset**: People's data categorized by various attributes to predict credit risk. [Dataset Link](https://archive.ics.uci.edu/ml/datasets/Statlog+%28German+Credit+Data%29)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "84fea308-2c09-472c-a5ec-e4ba0900d496",
      "metadata": {
        "id": "84fea308-2c09-472c-a5ec-e4ba0900d496"
      },
      "source": [
        "\n",
        "## Grading Criteria\n",
        "\n",
        "Your work will be evaluated based on both accuracy and loss value:\n",
        "\n",
        "100: The model has an accuracy of more than 80% and a Loss Value of less than 0.2. This model is excellent and demonstrates a strong understanding of the task.\n",
        "\n",
        "90: The model has an accuracy between 70% - 79% and a Loss Value between 0.2 - 0.3. This model is very good, with some room for improvement.\n",
        "\n",
        "80: The model has an accuracy between 60% - 69% and a Loss Value between 0.3 - 0.4. This model is fairly good but needs improvement in balancing accuracy and loss value.\n",
        "\n",
        "70: The model has an accuracy between 50% - 59% and a Loss Value between 0.4 - 0.5. This model is below average and needs significant improvement.\n",
        "\n",
        "60 or below: The model has an accuracy of less than 50% or a Loss Value of more than 0.5, or the student did not submit the accuracy and Loss Value. This model is poor and needs considerable improvement."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "915df21b-3a7e-444a-999d-c830acdbe7f2",
      "metadata": {
        "id": "915df21b-3a7e-444a-999d-c830acdbe7f2"
      },
      "source": [
        "Rmember to make a copy of this notebook in your Google Drive and work in your own copy.\n",
        "\n",
        "Let's start your deep learning journey! Choose your dataset and delve into the project! Happy modeling!"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "8018f21d-d661-4ae4-a50d-9fb66feb291d",
      "metadata": {
        "id": "8018f21d-d661-4ae4-a50d-9fb66feb291d"
      },
      "source": [
        "## Student Identity"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "e0b68327-fc7b-43de-9f16-b058d11d775b",
      "metadata": {
        "id": "e0b68327-fc7b-43de-9f16-b058d11d775b"
      },
      "source": [
        "## Project Structure\n",
        "\n",
        "Your project should be organized into five main sections."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "25ecc459-5019-42d8-84a6-8644cecafef3",
      "metadata": {
        "id": "25ecc459-5019-42d8-84a6-8644cecafef3"
      },
      "source": [
        "### 1. Package and Module Installation\n",
        "\n",
        "First, let's pool all package and module that you'll need in the installation section below."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "62606ca1-5868-4c65-93ef-1d6ee8d8d59a",
      "metadata": {
        "id": "62606ca1-5868-4c65-93ef-1d6ee8d8d59a"
      },
      "outputs": [],
      "source": [
        "# Write any package/module installation that you need\n",
        "# pip install goes here, this helps declutter your output below\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "from ucimlrepo import fetch_ucirepo \n",
        "import plotly.graph_objects as go\n",
        "from plotly.subplots import make_subplots\n",
        "from sklearn.preprocessing import LabelEncoder\n",
        "from sklearn.model_selection import train_test_split\n",
        "import torch\n",
        "import torch.nn as nn\n",
        "import torch.optim as optim\n",
        "from torch.utils.data import Dataset, DataLoader\n",
        "from sklearn.preprocessing import StandardScaler\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b0837717-de95-4193-9898-e2f4cadfcfc6",
      "metadata": {
        "id": "b0837717-de95-4193-9898-e2f4cadfcfc6"
      },
      "source": [
        "### 2. Data Loading and Preprocessing\n",
        "\n",
        "Load the chosen dataset and preprocess it for deep learning."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "6458c396",
      "metadata": {},
      "outputs": [],
      "source": [
        "# Fetch dataset from UCI repository based on documentation\n",
        "breast_cancer_wisconsin_diagnostic = fetch_ucirepo(id=17)\n",
        "\n",
        "# Extract features (X) and target labels (y) based on documentation\n",
        "X = breast_cancer_wisconsin_diagnostic.data.features  # Features as pandas DataFrame\n",
        "y = breast_cancer_wisconsin_diagnostic.data.targets  # Target as pandas DataFrame\n",
        "\n",
        "# Combine features and target into a single DataFrame for visualization\n",
        "df = pd.concat([X, y], axis=1)\n",
        "\n",
        "# Display first 5 rows of the full dataset\n",
        "print(df.head())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "f052beb7",
      "metadata": {},
      "outputs": [],
      "source": [
        "print(df.tail())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "db941354",
      "metadata": {},
      "outputs": [],
      "source": [
        "df.info()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "cd4056ec",
      "metadata": {},
      "outputs": [],
      "source": [
        "df.describe()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "26e0715f",
      "metadata": {},
      "outputs": [],
      "source": [
        "df.isnull().sum()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "b5512b49",
      "metadata": {},
      "outputs": [],
      "source": [
        "df.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "fa185a09",
      "metadata": {},
      "outputs": [],
      "source": [
        "# Visualize feature distributions in a grid layout\n",
        "features = df.columns[2:]  # Select the feature columns\n",
        "num_features = len(features)\n",
        "num_cols = 3  # Number of columns in the grid\n",
        "num_rows = (num_features + num_cols - 1) // num_cols  # Calculate the number of rows\n",
        "\n",
        "fig = make_subplots(rows=num_rows, cols=num_cols, subplot_titles=features)\n",
        "\n",
        "for i, feature in enumerate(features):\n",
        "    row = i // num_cols + 1\n",
        "    col = i % num_cols + 1\n",
        "    \n",
        "    # Plot the histogram for the current feature\n",
        "    fig.add_trace(go.Histogram(x=df[df['Diagnosis'] == 'M'][feature], name='Malignant', opacity=0.7), row=row, col=col)\n",
        "    fig.add_trace(go.Histogram(x=df[df['Diagnosis'] == 'B'][feature], name='Benign', opacity=0.7), row=row, col=col)\n",
        "    \n",
        "    # Set subplot title\n",
        "    fig.update_xaxes(title_text=feature, row=row, col=col)\n",
        "\n",
        "# Update layout\n",
        "fig.update_layout(showlegend=False, height=2000, width=1500, title_text=\"Feature Distributions\")\n",
        "\n",
        "fig.show()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "20848bd8",
      "metadata": {},
      "outputs": [],
      "source": [
        "# Calculate the correlation matrix\n",
        "cor = df.corr()\n",
        "\n",
        "# Create the heatmap trace\n",
        "heatmap = go.Heatmap(\n",
        "    z=cor.values,\n",
        "    x=cor.columns,\n",
        "    y=cor.columns,\n",
        "    colorscale='RdBu',\n",
        "    colorbar=dict(title='Correlation')\n",
        ")\n",
        "\n",
        "# Create the layout\n",
        "layout = go.Layout(\n",
        "    title='Correlation Matrix',\n",
        "    xaxis=dict(title='Features'),\n",
        "    yaxis=dict(title='Features'),\n",
        ")\n",
        "\n",
        "# Create the figure and add the trace\n",
        "fig = go.Figure(data=[heatmap], layout=layout)\n",
        "\n",
        "# Show the figure\n",
        "fig.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "30ce0c59-0e94-480a-81ae-528e68356a15",
      "metadata": {
        "id": "30ce0c59-0e94-480a-81ae-528e68356a15"
      },
      "source": [
        "### 3. Model Building\n",
        "\n",
        "Define your deep learning model's architecture."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "1aa3c3f7-c6c5-4419-9b25-e8d7a8ec345a",
      "metadata": {
        "id": "1aa3c3f7-c6c5-4419-9b25-e8d7a8ec345a"
      },
      "outputs": [],
      "source": [
        "# Write your code here for Model Building here\n",
        "label_encoder = LabelEncoder()\n",
        "df['Diagnosis'] = label_encoder.fit_transform(df['Diagnosis'])\n",
        "# define your Deep Learning Model here, training is in the next section.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "cfeaeee7",
      "metadata": {},
      "outputs": [],
      "source": [
        "features = df.drop(columns=['Diagnosis']).values\n",
        "labels = df['Diagnosis'].values\n",
        "\n",
        "X_train, X_test, y_train, y_test = train_test_split(features, labels, test_size=0.2, random_state=42)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "c94dbf5d",
      "metadata": {},
      "outputs": [],
      "source": [
        "# Convert the data to PyTorch tensors\n",
        "X_train = torch.tensor(X_train, dtype=torch.float32)\n",
        "X_test = torch.tensor(X_test, dtype=torch.float32)\n",
        "y_train = torch.tensor(y_train, dtype=torch.long)\n",
        "y_test = torch.tensor(y_test, dtype=torch.long)\n",
        "\n",
        "# Define the model architecture in PyTorch\n",
        "class DiseaseClassifier(nn.Module):\n",
        "    def __init__(self, input_dim, hidden_dim, output_dim):\n",
        "        super(DiseaseClassifier, self).__init__()\n",
        "        self.fc1 = nn.Linear(input_dim, hidden_dim)\n",
        "        self.relu = nn.ReLU()\n",
        "        self.fc2 = nn.Linear(hidden_dim, output_dim)\n",
        "        \n",
        "    def forward(self, x):\n",
        "        out = self.fc1(x)\n",
        "        out = self.relu(out)\n",
        "        out = self.fc2(out)\n",
        "        return out\n",
        "\n",
        "# Define the hyperparameters\n",
        "input_dim = X_train.shape[1]\n",
        "hidden_dim = 64\n",
        "output_dim = 2\n",
        "learning_rate = 0.001\n",
        "num_epochs = 10\n",
        "batch_size = 16\n",
        "\n",
        "# Create an instance of the model\n",
        "model = DiseaseClassifier(input_dim, hidden_dim, output_dim)\n",
        "\n",
        "# Define the loss function and optimizer\n",
        "criterion = nn.CrossEntropyLoss()\n",
        "optimizer = optim.Adam(model.parameters(), lr=learning_rate)\n",
        "\n",
        "# Create data loaders for batching\n",
        "train_dataset = torch.utils.data.TensorDataset(X_train, y_train)\n",
        "train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=batch_size, shuffle=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "585b925f-fcf0-4518-81d8-598dba65d646",
      "metadata": {
        "id": "585b925f-fcf0-4518-81d8-598dba65d646"
      },
      "source": [
        "### 4. Model Training\n",
        "\n",
        "Train your model and evaluate its performance using validation data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ee33d9bc-2b35-4f78-b5d8-3eb65db1e361",
      "metadata": {
        "id": "ee33d9bc-2b35-4f78-b5d8-3eb65db1e361"
      },
      "outputs": [],
      "source": [
        "# Write your code here for Model Training here\n",
        "#dl_loss_value = 1\n",
        "\n",
        "#define the iteration\n",
        "\n",
        "#create the training loop\n",
        "# Training loop\n",
        "for epoch in range(num_epochs):\n",
        "    running_loss = 0.0\n",
        "    for inputs, labels in train_loader:\n",
        "        # Zero the gradients\n",
        "        optimizer.zero_grad()\n",
        "        \n",
        "        # Forward pass\n",
        "        outputs = model(inputs)\n",
        "        \n",
        "        # Compute the loss\n",
        "        loss = criterion(outputs, labels)\n",
        "        \n",
        "        # Backward pass and optimization\n",
        "        loss.backward()\n",
        "        optimizer.step()\n",
        "        \n",
        "        # Track the loss\n",
        "        running_loss += loss.item()\n",
        "        \n",
        "    # Print the average loss for the epoch\n",
        "    print(f\"Epoch {epoch+1}/{num_epochs}, Loss: {running_loss / len(train_loader)}\")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1d391ec9-ca95-4c56-b4f6-fccf8e0be14a",
      "metadata": {
        "id": "1d391ec9-ca95-4c56-b4f6-fccf8e0be14a"
      },
      "source": [
        "### 5. Model Evaluation\n",
        "Evaluate your model's performance on the test data using the grading scheme defined above."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "c8ca8f78-e33c-4552-9f4d-8bed711e661f",
      "metadata": {
        "id": "c8ca8f78-e33c-4552-9f4d-8bed711e661f"
      },
      "outputs": [],
      "source": [
        "# Write your code here for Model Evaluation here\n",
        "# dl_accuracy = 0\n",
        "\n",
        "#define the iteration\n",
        "\n",
        "#create the training loop\n",
        "with torch.no_grad():\n",
        "    model.eval()\n",
        "    outputs = model(X_test)\n",
        "    _, predicted = torch.max(outputs.data, 1)\n",
        "    accuracy = (predicted == y_test).sum().item() / y_test.size(0)\n",
        "\n",
        "print(f\"\\nFinal Test Accuracy: {accuracy * 100:.2f}%\")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5d59af48-18c0-4e40-80ff-33a85030b205",
      "metadata": {
        "id": "5d59af48-18c0-4e40-80ff-33a85030b205"
      },
      "source": [
        "## Submission\n",
        "\n",
        "Once you are satisfied with the performance of your model, then you run the code block below to submit your project.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "aa89b9ff-0d31-416a-b3c4-851c725fadf7",
      "metadata": {
        "id": "aa89b9ff-0d31-416a-b3c4-851c725fadf7"
      },
      "source": [
        "## FIN"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.6"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}