update README.md - added ipynbs

README.md

@@ -46,5 +46,24 @@ python main.py -o config_files/options/baseline.json -a config_files/algorithm/<
 For reference of possible keys and values for each step, please see `config_files/algorithm/experiment_test.json`.
 To run the whole pipeline please create a new `.json` file, specifying all steps you want to run and specify desired keys and values for each step. 
 
+## Result plotting
+In the following, we describe the ipynb in the folder `\notebooks` to reproduce the illustrations from our paper. 
+
+
+### gedi_fig6_benchmark_boxplots.ipynb
+This notebook is used to visualize the metric distribution of real event logs compared to the generated ones. It shows 5 different metrics on 3 various process discovery techniques. We use 'fitness,', 'precision', 'fscore', 'size', 'cfc' (control-flow complexity) as metrics and as 'heuristic miner', 'ilp' (integer linear programming), and 'imf' (inductive miner infrequent) as miners. The notebook outputs the visualization shown in Fig.6 in the paper.
+
+### gedi_figs4and5_representativeness.ipynb
+In order to visualize the coverage of the feasible meta feature space of synthesied event logs compared to existing real-world benchmark datasets, in this notebook, we condcut a principal compoentn analysis on the meta features of both settings. The first two principal components are utilized to visualize the coverage which is further highlighted by computing a convex hull of the 2D-mapping.  Additionally, we visualize the distribution of each meta feature we used in the paper as a boxplot. Additional features can be extracted with FEEED. Therefore, the notebook contains the figures 4 and 5 in the paper. 
+
+### gedi_figs7and8_benchmarking_statisticalTests.ipynb
+
+This notebook is used to ansewr the question if there is a statistical signifacnt relation between feature similarity and performance metrics for the downstream tasks of process discovery. For that, we compute the pearson coefficient, as well as the kendall's tau coefficient. This elucidates the correlation between the meta features with metric scores being used for process discovery. Each coefficuent is calculated for three different settings: i) real-world datasets; ii) synthesized event log data with real-world targets; iii) synthesized event log data with grid objectives. The figures 7 and 8 shown in the paper refer to this notebook.
+
+### gedi_figs9and10_consistency.ipynb
+Likewise to the evaluation on the statistical tests in notebook `gedi_figs7and8_benchmarking_statisticalTests.ipynb`, this notebook is used to compute the differences between two correlation matrices $\Delta C = C_1 - C_2$. This logic is employed to evaluate and visualize the distance of two correlation matrices. Furthermore, we show how significant scores are retained from the correlations being evaluated on real-world datasets coompared to synthesized event log datasets with real-world targets. In Fig. 9 and 10 in the paper, the results of the notebook are shown. 
+
+
+
 ## References
 The framework used by `GEDI` is taken directly from the original paper by [...]. If you would like to discuss the paper, or corresponding research questions on benchmarking process mining tasks please email the authors.

notebooks/gedi_figs4and5_representativeness.ipynb

@@ -1,13 +1,19 @@
 {
  "cells": [
   {
-   "cell_type": "code",
-   "execution_count": 1,
-   "id": "b798e18b",
+   "cell_type": "markdown",
+   "id": "b2a7eff0-f134-4bfd-9729-604e312de968",
+   "metadata": {},
+   "source": [
+    "# Plot - Representativeness"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "73a8e182-f75b-4254-9bfe-766d7a758448",
    "metadata": {},
-   "outputs": [],
    "source": [
-    "import pandas as pd"
+    "#### load data"
    ]
   },
   {
@@ -25,6 +31,8 @@
     }
    ],
    "source": [
+    "import pandas as pd\n",
+    "\n",
     "df = pd.read_csv(\"../data/BaselineED_feat.csv\")\n",
     "gen =pd.read_csv(\"../data/GenED_feat.csv\")\n",
     "\n",
@@ -195,6 +203,14 @@
     "both_df.head()"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "789f649a-e2b7-415d-8b59-0253e2c968ed",
+   "metadata": {},
+   "source": [
+    "#### Data Preprocessing"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 4,
@@ -221,6 +237,14 @@
     "print(\"After:\", len(imp_df.loc[:, imp_df.isna().any()].columns))"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "99af2310-cfed-4668-83f2-8997a8dd4e2a",
+   "metadata": {},
+   "source": [
+    "#### Plot - Principal Component Analysis"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 5,
@@ -276,6 +300,14 @@
     "_ = plt.tight_layout()"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "f709c070-5c4b-4430-9a78-4e9c463708e4",
+   "metadata": {},
+   "source": [
+    "#### Plot - Convex hull of $1^{st}$ and $2^{nd}$ principal component"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 6,
@@ -432,6 +464,14 @@
     "plt.savefig(output_path, dpi=300, transparent=True)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "328721e8-68a9-4df5-96b6-f92f6b4e1ec1",
+   "metadata": {},
+   "source": [
+    "#### Reload module FeaturePlotter"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 8,
@@ -450,6 +490,14 @@
     "from gedi.plotter import FeaturesPlotter"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "de942786-25e1-45cd-aa4e-553809a88692",
+   "metadata": {},
+   "source": [
+    "#### Plot - feature distribution of real vs. generated data"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 9,
@@ -491,21 +539,13 @@
     "generated = gen.drop(['source'], axis=1)\n",
     "FeaturesPlotter( generated, model_params)"
    ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "53bdbc03",
-   "metadata": {},
-   "outputs": [],
-   "source": []
   }
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "tag",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
-   "name": "tag"
+   "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {
@@ -517,7 +557,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.18"
+   "version": "3.9.19"
   }
  },
  "nbformat": 4,

notebooks/gedi_figs7and8_benchmarking_statisticalTests.ipynb

@@ -1,5 +1,21 @@
 {
  "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "32241302-7f73-4756-b8a5-27f752de0dea",
+   "metadata": {},
+   "source": [
+    "# Plot - Statistical Tests"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "51cee5d6-2d4c-4bdd-bdbf-4b3a3b76e6d6",
+   "metadata": {},
+   "source": [
+    "#### load data"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 8,
@@ -64,6 +80,14 @@
     "    return data"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "f0d6e731-5f46-4747-82f8-a2f308d150ee",
+   "metadata": {},
+   "source": [
+    "#### Data Preprocessing"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 11,
@@ -110,7 +134,7 @@
    "id": "07370d54",
    "metadata": {},
    "source": [
-    "## Statistical test: Is there a statistical significant relation between feature similarity and performance metrics?"
+    "#### Statistical test: Is there a statistical significant relation between feature similarity and performance metrics?"
    ]
   },
   {
@@ -192,6 +216,14 @@
     "#df_tmp = statistical_test(DATA_SOURCE+\"_feat\", \"Gen\"+DATA_SOURCE+\"_bench\", TEST, IMPUTE)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "5e6ecc81-c14d-4859-ab04-49bbf458f7eb",
+   "metadata": {},
+   "source": [
+    "#### Plot - statistical Test of features vs metrics"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 62,
@@ -466,37 +498,13 @@
     "        plot_stat_test(masked_results, data_source+\"_feat\", data_source+\"_bench\", test, IMPUTE, cbar=cbar, ylabels=ylabels)\n",
     "        plt.clf()"
    ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "52c58c64",
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "3717a694",
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "c6afe4d9",
-   "metadata": {},
-   "outputs": [],
-   "source": []
   }
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "tag",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
-   "name": "tag"
+   "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {
@@ -508,7 +516,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.16"
+   "version": "3.9.19"
   }
  },
  "nbformat": 4,

notebooks/gedi_figs9and10_consistency.ipynb

@@ -1,5 +1,21 @@
 {
  "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "9c88600b-ccf7-4ed8-b2cc-094a7abb3e47",
+   "metadata": {},
+   "source": [
+    "# Plot - Consistency"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "40e77246-8ff1-4685-995f-b4f5aa34664e",
+   "metadata": {},
+   "source": [
+    "#### Load Data"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 1,
@@ -61,6 +77,14 @@
     "    return data"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "e7639ac5-865d-4a3e-ae75-627ec2fd31de",
+   "metadata": {},
+   "source": [
+    "#### Data Preprocessing"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 4,
@@ -107,7 +131,7 @@
    "id": "07370d54",
    "metadata": {},
    "source": [
-    "## Statistical test: Is there a statistical significant relation between feature similarity and performance metrics?"
+    "#### Statistical test: Is there a statistical significant relation between feature similarity and performance metrics?"
    ]
   },
   {
@@ -319,7 +343,7 @@
    "id": "d0a9ddb2",
    "metadata": {},
    "source": [
-    "## Figure 9: Correlation differences"
+    "#### Plot - Correlation differences (Figure 9)"
    ]
   },
   {
@@ -405,7 +429,7 @@
    "id": "962658c0",
    "metadata": {},
    "source": [
-    "## Figure 10: Limitations"
+    "#### Plot - Limitations (Figure 10)"
    ]
   },
   {
@@ -471,21 +495,13 @@
     "remark6_masked = statistical_test(DATA_SOURCE+\"_feat\", \"Gen\"+DATA_SOURCE+\"_bench\", test, IMPUTE, p_thresh=0.05)\n",
     "plot_stat_test(remark6_masked, DATA_SOURCE+\"_feat\", \"Gen\"+DATA_SOURCE+\"_bench\", test, IMPUTE, mask=True, cbar=True, ylabels=False, name_suffix=\"_filtered\") "
    ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "a2080811",
-   "metadata": {},
-   "outputs": [],
-   "source": []
   }
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "tag",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
-   "name": "tag"
+   "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {
@@ -497,7 +513,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.16"
+   "version": "3.9.19"
   }
  },
  "nbformat": 4,