Upload 20 files

add folder for pages, and stuff

APIs/README.md

@@ -0,0 +1,117 @@
+# Weather Dashboard for BYU Locations
+
+## Overview
+This Streamlit application provides weather insights for various BYU locations. Users can explore weather metrics, compare locations, and visualize key performance indicators (KPIs) and other weather variables. The app integrates with the Open-Meteo Weather API to fetch real-time and historical weather data.
+
+### Features
+1. **Single Location Weather:** View temperature data for a selected BYU location.
+2. **Compare Locations & KPIs:** Compare weather metrics like temperature across BYU Idaho, BYU Hawaii, and BYU Provo.
+3. **Variable Selection & Timezone:** Analyze multiple weather variables (e.g., Temperature, Wind Speed, Relative Humidity) with timezone options.
+4. **Summary Metrics Across Cities:** View combined metrics and boxplots for weather variables across cities.
+
+---
+
+## Technologies Used
+- **Streamlit**: Frontend interface for interactivity.
+- **Polars**: High-performance dataframe library for data manipulation.
+- **Requests**: Fetches weather data from the Open-Meteo API.
+- **Matplotlib**: Visualizes weather trends and comparisons.
+
+---
+
+## Setup and Installation
+Follow the steps below to get the app running locally:
+
+### 1. Prerequisites
+Ensure the following are installed:
+- Python 3.8 or higher
+- pip (Python package manager)
+
+### 2. Clone the Repository
+```bash
+git clone <repository-link>
+cd <repository-folder>
+```
+
+### 3. Install Dependencies
+Run the following command to install required Python packages:
+```bash
+pip install -r requirements.txt
+```
+**Note**: Ensure `streamlit`, `polars`, `matplotlib`, and `requests` are included in your `requirements.txt`.
+
+### 4. Project Directory Structure
+The app assumes the following directory structure:
+```
+project-folder/
+|└── APIs/
+|└── pages/
+|   |└── single_location.py
+|   |└── compare_locations.py
+|   |└── variable_selection.py
+|   |└── summary_metrics.py
+|└── screenshots/
+|└── utils/
+|   |└── weather_api.py
+|└── streamlit_app.py
+|└── requirements.txt
+|└── README.md
+|└── docker-compose.yaml
+|└── Dockerfile
+```
+
+### 5. Run the Application
+Launch the app using Streamlit:
+```bash
+streamlit run streamlit_app.py
+```
+
+The app will open in your browser at `http://localhost:8501`.
+
+---
+
+## Usage Instructions
+### Sidebar Navigation
+The sidebar allows you to navigate between different app sections:
+- **Single Location Weather**: Select a BYU location and view temperature data.
+- **Compare Locations & KPIs**: Compare weather metrics across multiple locations.
+- **Variable Selection & Timezone**: Analyze weather variables like temperature, wind speed, and humidity.
+- **Summary Metrics Across Cities**: View combined metrics and visualizations.
+
+### Input Parameters
+- **Date Range**: Select a start and end date for fetching weather data.
+- **Locations**: Choose from predefined BYU locations (Provo, Hawaii, Idaho).
+- **Timezone Selection**: For variable analysis, choose between `America/Denver` or `Pacific/Honolulu`.
+
+---
+
+## API Details
+- **Source**: [Open-Meteo API](https://open-meteo.com/)
+- **Endpoint**: `https://api.open-meteo.com/v1/forecast`
+- **Variables Supported**: Temperature, Wind Speed, Relative Humidity
+
+---
+
+## Screenshots
+### Single Location Weather
+![Single Location](screenshots/single_location.png)
+
+### Compare Locations
+![Compare Locations](screenshots/compare_locations.png)
+
+### Variable Selection
+![Variable Selection](screenshots/variable_selection.png)
+
+### Summary Metrics
+![Summary Metrics](screenshots/summary_metrics.png)
+
+---
+
+## Author
+Christian Landaverde
+
+---
+
+## Acknowledgements
+- Open-Meteo API for weather data.
+- Streamlit for the interactive dashboard.

APIs/example_api.ipynb

@@ -0,0 +1 @@
+{"cells":[{"cell_type":"code","source":["import pandas as pd\n\nimport requests\nimport json\nimport os\nimport datetime\nimport time\n\nimport pyspark.sql.functions as F"],"metadata":{"application/vnd.databricks.v1+cell":{"showTitle":false,"cellMetadata":{"rowLimit":10000,"byteLimit":2048000},"nuid":"846224e6-4452-43f8-9dad-a26705ff4f8c","inputWidgets":{},"title":""}},"outputs":[],"execution_count":0},{"cell_type":"markdown","source":["You will need to create a notebook with your keys in it in a notebook called `keys`. The next two lines provide examples of how to reference that notebook."],"metadata":{"application/vnd.databricks.v1+cell":{"showTitle":false,"cellMetadata":{},"nuid":"4d2b789e-2bd8-44f8-b24a-f03704d54eb9","inputWidgets":{},"title":""}}},{"cell_type":"code","source":["%run ./keys"],"metadata":{"application/vnd.databricks.v1+cell":{"showTitle":false,"cellMetadata":{"rowLimit":10000,"byteLimit":2048000},"nuid":"9fd037a7-d183-4e6a-bae7-0a657742bbbf","inputWidgets":{},"title":""}},"outputs":[],"execution_count":0},{"cell_type":"code","source":["%run ../sp23/keys"],"metadata":{"application/vnd.databricks.v1+cell":{"showTitle":false,"cellMetadata":{"rowLimit":10000,"byteLimit":2048000},"nuid":"bd674fa8-daa5-4317-9a57-6d5dc7e5a4c0","inputWidgets":{},"title":""}},"outputs":[{"output_type":"stream","output_type":"stream","name":"stdout","text":["ow_key is the variable\nd8a\n"]}],"execution_count":0},{"cell_type":"code","source":["# https://openweathermap.org/forecast16\napi_key = ow_key\n# https://api.openweathermap.org/data/2.5/forecast?lat=43.825386&lon=-111.792824&appid=d8aa64c57714e98a56057fa3bd9f478a&units=imperial&cnt=5\nlat = \"43.825386\"\nlon = \"-111.792824\"\nurl = \"https://api.openweathermap.org/data/2.5/forecast?lat=%s&lon=%s&appid=%s&units=imperial&cnt=5\" % (lat, lon, api_key)\nprint(url[0:85] + \"<YOURKEY>\" + url[117:])\n"],"metadata":{"application/vnd.databricks.v1+cell":{"showTitle":false,"cellMetadata":{"rowLimit":10000,"byteLimit":2048000},"nuid":"a7760181-3c8b-4183-bd40-9cd0a649fef2","inputWidgets":{},"title":""}},"outputs":[{"output_type":"stream","output_type":"stream","name":"stdout","text":["https://api.openweathermap.org/data/2.5/forecast?lat=43.825386&lon=-111.792824&appid=<YOURKEY>&units=imperial&cnt=5\n"]},{"output_type":"stream","output_type":"stream","name":"stdout","text":["ow_key is the variable\nd8a\n"]}],"execution_count":0},{"cell_type":"code","source":["# 200 is a good sign, #400 is a bad sign\nresponse = requests.get(url)\nresponse"],"metadata":{"application/vnd.databricks.v1+cell":{"showTitle":false,"cellMetadata":{"rowLimit":10000,"byteLimit":2048000},"nuid":"bac2a982-906a-446f-84df-aab0745a4780","inputWidgets":{},"title":""}},"outputs":[{"output_type":"stream","output_type":"stream","name":"stdout","text":["Out[51]: <Response [200]>"]}],"execution_count":0},{"cell_type":"code","source":["data = json.loads(response.text)\ndata"],"metadata":{"application/vnd.databricks.v1+cell":{"showTitle":false,"cellMetadata":{"rowLimit":10000,"byteLimit":2048000},"nuid":"4ec007c0-c8c7-4643-8436-33d09a8dd25c","inputWidgets":{},"title":""}},"outputs":[{"output_type":"stream","output_type":"stream","name":"stdout","text":["Out[52]: {'cod': '200',\n 'message': 0,\n 'cnt': 5,\n 'list': [{'dt': 1685394000,\n   'main': {'temp': 72.14,\n    'feels_like': 70.7,\n    'temp_min': 69.12,\n    'temp_max': 72.14,\n    'pressure': 1015,\n    'sea_level': 1015,\n    'grnd_level': 852,\n    'humidity': 35,\n    'temp_kf': 1.68},\n   'weather': [{'id': 800,\n     'main': 'Clear',\n     'description': 'clear sky',\n     'icon': '01d'}],\n   'clouds': {'all': 0},\n   'wind': {'speed': 11.14, 'deg': 217, 'gust': 14.92},\n   'visibility': 10000,\n   'pop': 0.31,\n   'sys': {'pod': 'd'},\n   'dt_txt': '2023-05-29 21:00:00'},\n  {'dt': 1685404800,\n   'main': {'temp': 71.44,\n    'feels_like': 70.07,\n    'temp_min': 70.03,\n    'temp_max': 71.44,\n    'pressure': 1012,\n    'sea_level': 1012,\n    'grnd_level': 850,\n    'humidity': 38,\n    'temp_kf': 0.78},\n   'weather': [{'id': 801,\n     'main': 'Clouds',\n     'description': 'few clouds',\n     'icon': '02d'}],\n   'clouds': {'all': 18},\n   'wind': {'speed': 10.94, 'deg': 213, 'gust': 14.47},\n   'visibility': 10000,\n   'pop': 0.34,\n   'sys': {'pod': 'd'},\n   'dt_txt': '2023-05-30 00:00:00'},\n  {'dt': 1685415600,\n   'main': {'temp': 62.2,\n    'feels_like': 60.89,\n    'temp_min': 57.24,\n    'temp_max': 62.2,\n    'pressure': 1012,\n    'sea_level': 1012,\n    'grnd_level': 850,\n    'humidity': 59,\n    'temp_kf': 2.76},\n   'weather': [{'id': 500,\n     'main': 'Rain',\n     'description': 'light rain',\n     'icon': '10n'}],\n   'clouds': {'all': 56},\n   'wind': {'speed': 8.81, 'deg': 209, 'gust': 17.58},\n   'visibility': 10000,\n   'pop': 0.71,\n   'rain': {'3h': 0.13},\n   'sys': {'pod': 'n'},\n   'dt_txt': '2023-05-30 03:00:00'},\n  {'dt': 1685426400,\n   'main': {'temp': 50.13,\n    'feels_like': 48.79,\n    'temp_min': 50.13,\n    'temp_max': 50.13,\n    'pressure': 1012,\n    'sea_level': 1012,\n    'grnd_level': 849,\n    'humidity': 84,\n    'temp_kf': 0},\n   'weather': [{'id': 803,\n     'main': 'Clouds',\n     'description': 'broken clouds',\n     'icon': '04n'}],\n   'clouds': {'all': 68},\n   'wind': {'speed': 7.38, 'deg': 135, 'gust': 10.76},\n   'visibility': 10000,\n   'pop': 0.42,\n   'sys': {'pod': 'n'},\n   'dt_txt': '2023-05-30 06:00:00'},\n  {'dt': 1685437200,\n   'main': {'temp': 48.56,\n    'feels_like': 46.87,\n    'temp_min': 48.56,\n    'temp_max': 48.56,\n    'pressure': 1012,\n    'sea_level': 1012,\n    'grnd_level': 848,\n    'humidity': 82,\n    'temp_kf': 0},\n   'weather': [{'id': 804,\n     'main': 'Clouds',\n     'description': 'overcast clouds',\n     'icon': '04n'}],\n   'clouds': {'all': 99},\n   'wind': {'speed': 4.47, 'deg': 71, 'gust': 4.76},\n   'visibility': 10000,\n   'pop': 0.05,\n   'sys': {'pod': 'n'},\n   'dt_txt': '2023-05-30 09:00:00'}],\n 'city': {'id': 5605242,\n  'name': 'Rexburg',\n  'coord': {'lat': 43.8254, 'lon': -111.7928},\n  'country': 'US',\n  'population': 25484,\n  'timezone': -21600,\n  'sunrise': 1685360996,\n  'sunset': 1685415573}}"]}],"execution_count":0},{"cell_type":"code","source":["# Google: python print json.loads object -> https://www.digitalocean.com/community/tutorials/python-pretty-print-json\njson_formatted_str = json.dumps(data, indent=3)\nprint(json_formatted_str)"],"metadata":{"application/vnd.databricks.v1+cell":{"showTitle":false,"cellMetadata":{"rowLimit":10000,"byteLimit":2048000},"nuid":"edec49b8-ef94-422b-af64-691090a95d1b","inputWidgets":{},"title":""}},"outputs":[{"output_type":"stream","output_type":"stream","name":"stdout","text":["{\n   \"cod\": \"200\",\n   \"message\": 0,\n   \"cnt\": 5,\n   \"list\": [\n      {\n         \"dt\": 1685394000,\n         \"main\": {\n            \"temp\": 72.14,\n            \"feels_like\": 70.7,\n            \"temp_min\": 69.12,\n            \"temp_max\": 72.14,\n            \"pressure\": 1015,\n            \"sea_level\": 1015,\n            \"grnd_level\": 852,\n            \"humidity\": 35,\n            \"temp_kf\": 1.68\n         },\n         \"weather\": [\n            {\n               \"id\": 800,\n               \"main\": \"Clear\",\n               \"description\": \"clear sky\",\n               \"icon\": \"01d\"\n            }\n         ],\n         \"clouds\": {\n            \"all\": 0\n         },\n         \"wind\": {\n            \"speed\": 11.14,\n            \"deg\": 217,\n            \"gust\": 14.92\n         },\n         \"visibility\": 10000,\n         \"pop\": 0.31,\n         \"sys\": {\n            \"pod\": \"d\"\n         },\n         \"dt_txt\": \"2023-05-29 21:00:00\"\n      },\n      {\n         \"dt\": 1685404800,\n         \"main\": {\n            \"temp\": 71.44,\n            \"feels_like\": 70.07,\n            \"temp_min\": 70.03,\n            \"temp_max\": 71.44,\n            \"pressure\": 1012,\n            \"sea_level\": 1012,\n            \"grnd_level\": 850,\n            \"humidity\": 38,\n            \"temp_kf\": 0.78\n         },\n         \"weather\": [\n            {\n               \"id\": 801,\n               \"main\": \"Clouds\",\n               \"description\": \"few clouds\",\n               \"icon\": \"02d\"\n            }\n         ],\n         \"clouds\": {\n            \"all\": 18\n         },\n         \"wind\": {\n            \"speed\": 10.94,\n            \"deg\": 213,\n            \"gust\": 14.47\n         },\n         \"visibility\": 10000,\n         \"pop\": 0.34,\n         \"sys\": {\n            \"pod\": \"d\"\n         },\n         \"dt_txt\": \"2023-05-30 00:00:00\"\n      },\n      {\n         \"dt\": 1685415600,\n         \"main\": {\n            \"temp\": 62.2,\n            \"feels_like\": 60.89,\n            \"temp_min\": 57.24,\n            \"temp_max\": 62.2,\n            \"pressure\": 1012,\n            \"sea_level\": 1012,\n            \"grnd_level\": 850,\n            \"humidity\": 59,\n            \"temp_kf\": 2.76\n         },\n         \"weather\": [\n            {\n               \"id\": 500,\n               \"main\": \"Rain\",\n               \"description\": \"light rain\",\n               \"icon\": \"10n\"\n            }\n         ],\n         \"clouds\": {\n            \"all\": 56\n         },\n         \"wind\": {\n            \"speed\": 8.81,\n            \"deg\": 209,\n            \"gust\": 17.58\n         },\n         \"visibility\": 10000,\n         \"pop\": 0.71,\n         \"rain\": {\n            \"3h\": 0.13\n         },\n         \"sys\": {\n            \"pod\": \"n\"\n         },\n         \"dt_txt\": \"2023-05-30 03:00:00\"\n      },\n      {\n         \"dt\": 1685426400,\n         \"main\": {\n            \"temp\": 50.13,\n            \"feels_like\": 48.79,\n            \"temp_min\": 50.13,\n            \"temp_max\": 50.13,\n            \"pressure\": 1012,\n            \"sea_level\": 1012,\n            \"grnd_level\": 849,\n            \"humidity\": 84,\n            \"temp_kf\": 0\n         },\n         \"weather\": [\n            {\n               \"id\": 803,\n               \"main\": \"Clouds\",\n               \"description\": \"broken clouds\",\n               \"icon\": \"04n\"\n            }\n         ],\n         \"clouds\": {\n            \"all\": 68\n         },\n         \"wind\": {\n            \"speed\": 7.38,\n            \"deg\": 135,\n            \"gust\": 10.76\n         },\n         \"visibility\": 10000,\n         \"pop\": 0.42,\n         \"sys\": {\n            \"pod\": \"n\"\n         },\n         \"dt_txt\": \"2023-05-30 06:00:00\"\n      },\n      {\n         \"dt\": 1685437200,\n         \"main\": {\n            \"temp\": 48.56,\n            \"feels_like\": 46.87,\n            \"temp_min\": 48.56,\n            \"temp_max\": 48.56,\n            \"pressure\": 1012,\n            \"sea_level\": 1012,\n            \"grnd_level\": 848,\n            \"humidity\": 82,\n            \"temp_kf\": 0\n         },\n         \"weather\": [\n            {\n               \"id\": 804,\n               \"main\": \"Clouds\",\n               \"description\": \"overcast clouds\",\n               \"icon\": \"04n\"\n            }\n         ],\n         \"clouds\": {\n            \"all\": 99\n         },\n         \"wind\": {\n            \"speed\": 4.47,\n            \"deg\": 71,\n            \"gust\": 4.76\n         },\n         \"visibility\": 10000,\n         \"pop\": 0.05,\n         \"sys\": {\n            \"pod\": \"n\"\n         },\n         \"dt_txt\": \"2023-05-30 09:00:00\"\n      }\n   ],\n   \"city\": {\n      \"id\": 5605242,\n      \"name\": \"Rexburg\",\n      \"coord\": {\n         \"lat\": 43.8254,\n         \"lon\": -111.7928\n      },\n      \"country\": \"US\",\n      \"population\": 25484,\n      \"timezone\": -21600,\n      \"sunrise\": 1685360996,\n      \"sunset\": 1685415573\n   }\n}\n"]}],"execution_count":0},{"cell_type":"code","source":["rdd = spark.sparkContext.parallelize([response.text])\ndf = spark.read.json(rdd)\ndisplay(df)"],"metadata":{"application/vnd.databricks.v1+cell":{"showTitle":false,"cellMetadata":{"rowLimit":10000,"byteLimit":2048000},"nuid":"c5f049d1-f2c8-417c-ac22-129eec2425d8","inputWidgets":{},"title":""}},"outputs":[{"output_type":"display_data","metadata":{"application/vnd.databricks.v1+output":{"overflow":false,"datasetInfos":[],"data":[[[[43.8254,-111.7928],"US",5605242,"Rexburg",25484,1685360996,1685415573,-21600],5,"200",[[[0],1685394000,"2023-05-29 21:00:00",[70.7,852,35,1015,1015,72.14,1.68,72.14,69.12],0.31,null,["d"],10000,[["clear sky","01d",800,"Clear"]],[217,14.92,11.14]],[[18],1685404800,"2023-05-30 00:00:00",[70.07,850,38,1012,1012,71.44,0.78,71.44,70.03],0.34,null,["d"],10000,[["few clouds","02d",801,"Clouds"]],[213,14.47,10.94]],[[56],1685415600,"2023-05-30 03:00:00",[60.89,850,59,1012,1012,62.2,2.76,62.2,57.24],0.71,[0.13],["n"],10000,[["light rain","10n",500,"Rain"]],[209,17.58,8.81]],[[68],1685426400,"2023-05-30 06:00:00",[48.79,849,84,1012,1012,50.13,0.0,50.13,50.13],0.42,null,["n"],10000,[["broken clouds","04n",803,"Clouds"]],[135,10.76,7.38]],[[99],1685437200,"2023-05-30 09:00:00",[46.87,848,82,1012,1012,48.56,0.0,48.56,48.56],0.05,null,["n"],10000,[["overcast clouds","04n",804,"Clouds"]],[71,4.76,4.47]]],0]],"plotOptions":{"displayType":"table","customPlotOptions":{},"pivotColumns":null,"pivotAggregation":null,"xColumns":null,"yColumns":null},"columnCustomDisplayInfos":{},"aggType":"","isJsonSchema":true,"removedWidgets":[],"aggSchema":[],"schema":[{"name":"city","type":"{\"type\":\"struct\",\"fields\":[{\"name\":\"coord\",\"type\":{\"type\":\"struct\",\"fields\":[{\"name\":\"lat\",\"type\":\"double\",\"nullable\":true,\"metadata\":{}},{\"name\":\"lon\",\"type\":\"double\",\"nullable\":true,\"metadata\":{}}]},\"nullable\":true,\"metadata\":{}},{\"name\":\"country\",\"type\":\"string\",\"nullable\":true,\"metadata\":{}},{\"name\":\"id\",\"type\":\"long\",\"nullable\":true,\"metadata\":{}},{\"name\":\"name\",\"type\":\"string\",\"nullable\":true,\"metadata\":{}},{\"name\":\"population\",\"type\":\"long\",\"nullable\":true,\"metadata\":{}},{\"name\":\"sunrise\",\"type\":\"long\",\"nullable\":true,\"metadata\":{}},{\"name\":\"sunset\",\"type\":\"long\",\"nullable\":true,\"metadata\":{}},{\"name\":\"timezone\",\"type\":\"long\",\"nullable\":true,\"metadata\":{}}]}","metadata":"{}"},{"name":"cnt","type":"\"long\"","metadata":"{}"},{"name":"cod","type":"\"string\"","metadata":"{}"},{"name":"list","type":"{\"type\":\"array\",\"elementType\":{\"type\":\"struct\",\"fields\":[{\"name\":\"clouds\",\"type\":{\"type\":\"struct\",\"fields\":[{\"name\":\"all\",\"type\":\"long\",\"nullable\":true,\"metadata\":{}}]},\"nullable\":true,\"metadata\":{}},{\"name\":\"dt\",\"type\":\"long\",\"nullable\":true,\"metadata\":{}},{\"name\":\"dt_txt\",\"type\":\"string\",\"nullable\":true,\"metadata\":{}},{\"name\":\"main\",\"type\":{\"type\":\"struct\",\"fields\":[{\"name\":\"feels_like\",\"type\":\"double\",\"nullable\":true,\"metadata\":{}},{\"name\":\"grnd_level\",\"type\":\"long\",\"nullable\":true,\"metadata\":{}},{\"name\":\"humidity\",\"type\":\"long\",\"nullable\":true,\"metadata\":{}},{\"name\":\"pressure\",\"type\":\"long\",\"nullable\":true,\"metadata\":{}},{\"name\":\"sea_level\",\"type\":\"long\",\"nullable\":true,\"metadata\":{}},{\"name\":\"temp\",\"type\":\"double\",\"nullable\":true,\"metadata\":{}},{\"name\":\"temp_kf\",\"type\":\"double\",\"nullable\":true,\"metadata\":{}},{\"name\":\"temp_max\",\"type\":\"double\",\"nullable\":true,\"metadata\":{}},{\"name\":\"temp_min\",\"type\":\"double\",\"nullable\":true,\"metadata\":{}}]},\"nullable\":true,\"metadata\":{}},{\"name\":\"pop\",\"type\":\"double\",\"nullable\":true,\"metadata\":{}},{\"name\":\"rain\",\"type\":{\"type\":\"struct\",\"fields\":[{\"name\":\"3h\",\"type\":\"double\",\"nullable\":true,\"metadata\":{}}]},\"nullable\":true,\"metadata\":{}},{\"name\":\"sys\",\"type\":{\"type\":\"struct\",\"fields\":[{\"name\":\"pod\",\"type\":\"string\",\"nullable\":true,\"metadata\":{}}]},\"nullable\":true,\"metadata\":{}},{\"name\":\"visibility\",\"type\":\"long\",\"nullable\":true,\"metadata\":{}},{\"name\":\"weather\",\"type\":{\"type\":\"array\",\"elementType\":{\"type\":\"struct\",\"fields\":[{\"name\":\"description\",\"type\":\"string\",\"nullable\":true,\"metadata\":{}},{\"name\":\"icon\",\"type\":\"string\",\"nullable\":true,\"metadata\":{}},{\"name\":\"id\",\"type\":\"long\",\"nullable\":true,\"metadata\":{}},{\"name\":\"main\",\"type\":\"string\",\"nullable\":true,\"metadata\":{}}]},\"containsNull\":true},\"nullable\":true,\"metadata\":{}},{\"name\":\"wind\",\"type\":{\"type\":\"struct\",\"fields\":[{\"name\":\"deg\",\"type\":\"long\",\"nullable\":true,\"metadata\":{}},{\"name\":\"gust\",\"type\":\"double\",\"nullable\":true,\"metadata\":{}},{\"name\":\"speed\",\"type\":\"double\",\"nullable\":true,\"metadata\":{}}]},\"nullable\":true,\"metadata\":{}}]},\"containsNull\":true}","metadata":"{}"},{"name":"message","type":"\"long\"","metadata":"{}"}],"aggError":"","aggData":[],"addedWidgets":{},"metadata":{},"dbfsResultPath":null,"type":"table","aggOverflow":false,"aggSeriesLimitReached":false,"arguments":{}}},"output_type":"display_data","data":{"text/html":["<style scoped>\n","  .table-result-container {\n","    max-height: 300px;\n","    overflow: auto;\n","  }\n","  table, th, td {\n","    border: 1px solid black;\n","    border-collapse: collapse;\n","  }\n","  th, td {\n","    padding: 5px;\n","  }\n","  th {\n","    text-align: left;\n","  }\n","</style><div class='table-result-container'><table class='table-result'><thead style='background-color: white'><tr><th>city</th><th>cnt</th><th>cod</th><th>list</th><th>message</th></tr></thead><tbody><tr><td>List(List(43.8254, -111.7928), US, 5605242, Rexburg, 25484, 1685360996, 1685415573, -21600)</td><td>5</td><td>200</td><td>List(List(List(0), 1685394000, 2023-05-29 21:00:00, List(70.7, 852, 35, 1015, 1015, 72.14, 1.68, 72.14, 69.12), 0.31, null, List(d), 10000, List(List(clear sky, 01d, 800, Clear)), List(217, 14.92, 11.14)), List(List(18), 1685404800, 2023-05-30 00:00:00, List(70.07, 850, 38, 1012, 1012, 71.44, 0.78, 71.44, 70.03), 0.34, null, List(d), 10000, List(List(few clouds, 02d, 801, Clouds)), List(213, 14.47, 10.94)), List(List(56), 1685415600, 2023-05-30 03:00:00, List(60.89, 850, 59, 1012, 1012, 62.2, 2.76, 62.2, 57.24), 0.71, List(0.13), List(n), 10000, List(List(light rain, 10n, 500, Rain)), List(209, 17.58, 8.81)), List(List(68), 1685426400, 2023-05-30 06:00:00, List(48.79, 849, 84, 1012, 1012, 50.13, 0.0, 50.13, 50.13), 0.42, null, List(n), 10000, List(List(broken clouds, 04n, 803, Clouds)), List(135, 10.76, 7.38)), List(List(99), 1685437200, 2023-05-30 09:00:00, List(46.87, 848, 82, 1012, 1012, 48.56, 0.0, 48.56, 48.56), 0.05, null, List(n), 10000, List(List(overcast clouds, 04n, 804, Clouds)), List(71, 4.76, 4.47)))</td><td>0</td></tr></tbody></table></div>"]}}],"execution_count":0},{"cell_type":"code","source":["df.printSchema()"],"metadata":{"application/vnd.databricks.v1+cell":{"showTitle":false,"cellMetadata":{"rowLimit":10000,"byteLimit":2048000},"nuid":"de7ca8e6-1e75-4461-bd45-310cabbc3bfb","inputWidgets":{},"title":""}},"outputs":[{"output_type":"stream","output_type":"stream","name":"stdout","text":["root\n |-- city: struct (nullable = true)\n |    |-- coord: struct (nullable = true)\n |    |    |-- lat: double (nullable = true)\n |    |    |-- lon: double (nullable = true)\n |    |-- country: string (nullable = true)\n |    |-- id: long (nullable = true)\n |    |-- name: string (nullable = true)\n |    |-- population: long (nullable = true)\n |    |-- sunrise: long (nullable = true)\n |    |-- sunset: long (nullable = true)\n |    |-- timezone: long (nullable = true)\n |-- cnt: long (nullable = true)\n |-- cod: string (nullable = true)\n |-- list: array (nullable = true)\n |    |-- element: struct (containsNull = true)\n |    |    |-- clouds: struct (nullable = true)\n |    |    |    |-- all: long (nullable = true)\n |    |    |-- dt: long (nullable = true)\n |    |    |-- dt_txt: string (nullable = true)\n |    |    |-- main: struct (nullable = true)\n |    |    |    |-- feels_like: double (nullable = true)\n |    |    |    |-- grnd_level: long (nullable = true)\n |    |    |    |-- humidity: long (nullable = true)\n |    |    |    |-- pressure: long (nullable = true)\n |    |    |    |-- sea_level: long (nullable = true)\n |    |    |    |-- temp: double (nullable = true)\n |    |    |    |-- temp_kf: double (nullable = true)\n |    |    |    |-- temp_max: double (nullable = true)\n |    |    |    |-- temp_min: double (nullable = true)\n |    |    |-- pop: double (nullable = true)\n |    |    |-- rain: struct (nullable = true)\n |    |    |    |-- 3h: double (nullable = true)\n |    |    |-- sys: struct (nullable = true)\n |    |    |    |-- pod: string (nullable = true)\n |    |    |-- visibility: long (nullable = true)\n |    |    |-- weather: array (nullable = true)\n |    |    |    |-- element: struct (containsNull = true)\n |    |    |    |    |-- description: string (nullable = true)\n |    |    |    |    |-- icon: string (nullable = true)\n |    |    |    |    |-- id: long (nullable = true)\n |    |    |    |    |-- main: string (nullable = true)\n |    |    |-- wind: struct (nullable = true)\n |    |    |    |-- deg: long (nullable = true)\n |    |    |    |-- gust: double (nullable = true)\n |    |    |    |-- speed: double (nullable = true)\n |-- message: long (nullable = true)\n\n"]}],"execution_count":0}],"metadata":{"application/vnd.databricks.v1+notebook":{"notebookName":"example_api","dashboards":[],"notebookMetadata":{"pythonIndentUnit":2},"language":"python","widgets":{}}},"nbformat":4,"nbformat_minor":0}

APIs/graphql_noauth.py

@@ -0,0 +1,30 @@
+# %%
+# https://towardsdatascience.com/connecting-to-a-graphql-api-using-python-246dda927840
+import requests
+import json
+import pandas as pd
+# %%
+query = """
+query {
+    characters {
+    results {
+      name
+      status
+      species
+      type
+      gender
+    }
+  }
+}
+"""
+# %%
+url = 'https://rickandmortyapi.com/graphql/'
+r = requests.post(url, json={'query': query})
+print(r.status_code)
+print(r.text)
+# %%
+json_data = json.loads(r.text)
+# %%
+df_data = json_data['data']['characters']['results']
+df = pd.DataFrame(df_data)
+# %%

APIs/graphql_noauth_activity.py

@@ -0,0 +1,84 @@
+# %%
+# https://towardsdatascience.com/connecting-to-a-graphql-api-using-python-246dda927840
+import requests
+import json
+import jsonlines
+import pandas as pd
+import pyarrow as pa
+import pyarrow.json as pj
+# %%
+query = """
+query {
+    characters {
+    results {
+      name
+      status
+      species
+      type
+      gender
+    }
+  }
+}
+"""
+
+query = """
+query {
+    characters {
+    results {
+      name
+      status
+      species
+      type
+      gender
+      image
+      episode{
+        episode
+        name
+        air_date
+      }
+    }
+  }
+}
+"""
+# %%
+url = 'https://rickandmortyapi.com/graphql/'
+r = requests.post(url, json={'query': query})
+print(r.status_code)
+print(r.text)
+# %%
+json_data = json.loads(r.text)
+# %%
+df_data = json_data['data']['characters']['results']
+df = pd.json_normalize(df_data)
+
+
+# %%  
+# https://blog.softhints.com/python-convert-json-to-json-lines/  
+with jsonlines.open('output.jsonl', mode='w') as writer:
+    writer.write_all(df_data)
+
+# %%
+datjl = pd.read_json("output.jsonl", lines=True)
+
+# %%
+pj.read_json("output.jsonl", parse_options = pj.ParseOptions(newlines_in_values=True))
+
+# %%
+# if we want to convert the format to JSON arrays.
+
+# df_data = json_data['data']['characters']['results']
+# for i, value in enumerate(df_data):
+#   print(i)
+#   idat = df_data[i].copy()
+#   number = list()
+#   name = list()
+#   airdate = list()
+#   for j, jvalue in enumerate(idat['episode']):
+#     iseason = idat['episode'][j]
+#     number.append(iseason['episode'])
+#     name.append(iseason['name'])
+#     airdate.append(iseason['air_date'])
+#   df_data[i]['episode'] = {
+#     'number':number,
+#     "name":name,
+#     "air_date":airdate}

APIs/openweather.md

@@ -0,0 +1,14 @@
+# [OpenWeather.org](https://openweathermap.org/)
+
+As academics we get free access to the [OpenWeather API](https://openweathermap.org/our-initiatives/student-initiative).
+
+We get the benefits of the _Developer Plan_ for [current weather and forecasts](https://openweathermap.org/price#current) and the _Medium Plan_ for the [historical weather collection](https://openweathermap.org/price#history).
+
+You have to fill out the request form to get access to the “Developer Plan” for Open Weather API for free as a student. If you don’t sign up, you get this error when trying to access it in your code:
+
+```
+{'cod': 401, 'message': 'Invalid API key. Please see https://openweathermap.org/faq#error401 for more info.'}
+```
+
+So just fill out the form at this site to get access: https://openweathermap.org/our-initiatives/student-initiative.   
+Then you should see the Developer plan for Weather and Medium plan for History on the Services tab: https://home.openweathermap.org/myservices.  

pages/compare_locations.py

@@ -0,0 +1,66 @@
+import streamlit as st
+import polars as pl
+from utils.weather_api import fetch_weather
+from datetime import datetime, timedelta
+import matplotlib.pyplot as plt
+
+def fetch_multiple_cities(start_date, end_date):
+    locations = {
+        "BYU Idaho": {"lat": 43.8145, "lon": -111.7833},
+        "BYU Hawaii": {"lat": 21.6419, "lon": -157.9267},
+        "BYU Provo": {"lat": 40.25, "lon": -111.65}
+    }
+    city_data = {}
+    for city, coords in locations.items():
+        try:
+            df = fetch_weather(coords, start_date, end_date)
+            city_data[city] = df.rename({"temperature_2m": city})
+        except Exception as e:
+            st.error(f"Failed to fetch data for {city}: {e}")
+
+    combined_df = city_data[list(city_data.keys())[0]].select("datetime")
+    for city, data in city_data.items():
+        combined_df = pl.concat([combined_df, data.select(city)], how="horizontal")
+    return combined_df
+
+def display_visualizations(df):
+    st.subheader("Daily High Temperatures Across Cities")
+    melted_df = df.melt(id_vars=["datetime"], variable_name="City", value_name="Temperature")
+    
+    plt.figure(figsize=(10, 6))
+    for city in df.columns[1:]:
+        plt.plot(df["datetime"], df[city], label=city)
+    plt.xlabel("Datetime")
+    plt.ylabel("Temperature (°F)")
+    plt.title("Temperature Comparison")
+    plt.legend()
+    st.pyplot(plt)
+
+def calculate_kpis(df):
+    kpis = {}
+    for city in df.columns[1:]:
+        highest = df[city].max()
+        lowest = df[city].min()
+        kpis[city] = (highest, lowest)
+    return kpis
+
+def main():
+    st.title("Compare Weather Data for All BYU Locations 🌍")
+    st.sidebar.header("User Input")
+    start_date = st.sidebar.date_input("Start Date", datetime.now() - timedelta(days=15))
+    end_date = st.sidebar.date_input("End Date", datetime.now())
+
+    if start_date and end_date:
+        st.write("Fetching weather data for all BYU locations...")
+        try:
+            combined_data = fetch_multiple_cities(start_date, end_date)
+            st.dataframe(combined_data)
+
+            st.subheader("Key Performance Indicators (KPIs)")
+            kpis = calculate_kpis(combined_data)
+            for city, (highest, lowest) in kpis.items():
+                st.write(f"**{city}:** Highest Temp: {highest:.2f} °F | Lowest Temp: {lowest:.2f} °F")
+
+            display_visualizations(combined_data)
+        except Exception as e:
+            st.error(f"Error: {e}")

pages/single_location.py

@@ -0,0 +1,25 @@
+import streamlit as st
+from utils.weather_api import fetch_weather
+from datetime import datetime, timedelta
+
+def main():
+    st.title("Weather Dashboard for Single BYU Location 🌤️")
+
+    locations = {
+        "BYU Idaho": {"lat": 43.8145, "lon": -111.7833},
+        "BYU Hawaii": {"lat": 21.6419, "lon": -157.9267},
+        "BYU Provo": {"lat": 40.25, "lon": -111.65}
+    }
+
+    st.sidebar.header("User Input")
+    selected_city = st.sidebar.selectbox("Select a BYU Location", list(locations.keys()))
+    start_date = st.sidebar.date_input("Start Date", datetime.now() - timedelta(days=15))
+    end_date = st.sidebar.date_input("End Date", datetime.now())
+
+    if start_date and end_date:
+        try:
+            weather_data = fetch_weather(locations[selected_city], start_date, end_date)
+            st.write(f"Weather Data for **{selected_city}**")
+            st.dataframe(weather_data)
+        except Exception as e:
+            st.error(f"Error fetching data: {e}")

pages/summary_metrics.py

@@ -0,0 +1,66 @@
+import streamlit as st
+import polars as pl
+from datetime import datetime, timedelta
+from utils.weather_api import fetch_weather 
+
+locations = {
+    "BYU Idaho": {"lat": 43.8145, "lon": -111.7833},
+    "BYU Hawaii": {"lat": 21.6419, "lon": -157.9267},
+    "BYU Provo": {"lat": 40.25, "lon": -111.65}
+}
+
+weather_variables = {
+    "Temperature (°F)": "temperature_2m",
+    "Wind Speed (mph)": "windspeed_10m",
+    "Relative Humidity (%)": "relative_humidity_2m"
+}
+
+def fetch_multiple_cities(start_date, end_date, variable, timezone_selection):
+    city_data = {}
+    for city, coords in locations.items():
+        df = fetch_weather(coords, start_date, end_date, variable, timezone_selection)
+        city_data[city] = df.rename({variable: city})
+    combined_df = city_data[list(city_data.keys())[0]].select("datetime")
+    for city, data in city_data.items():
+        combined_df = pl.concat([combined_df, data.select(city)], how="horizontal")
+    return combined_df
+
+def calculate_summary_metrics(df):
+    avg_values = df.select([pl.mean(city).alias(city) for city in df.columns if city != "datetime"])
+    max_values = df.select([pl.max(city).alias(city) for city in df.columns if city != "datetime"])
+    min_values = df.select([pl.min(city).alias(city) for city in df.columns if city != "datetime"])
+
+    avg_summary = avg_values.row(0)
+    max_summary = max_values.row(0)
+    min_summary = min_values.row(0)
+    return avg_summary, max_summary, min_summary
+
+def main():
+    st.title("Compare Weather Metrics Across BYU Locations 📊")
+    
+    st.sidebar.header("User Input")
+    start_date = st.sidebar.date_input("Start Date", datetime.now() - timedelta(days=15))
+    end_date = st.sidebar.date_input("End Date", datetime.now())
+    selected_variable_label = st.sidebar.selectbox("Select Weather Variable", weather_variables.keys())
+    timezone_selection = st.sidebar.radio("Select Time Zone", ["America/Denver", "Pacific/Honolulu"])
+    selected_variable = weather_variables[selected_variable_label]
+    
+    if start_date and end_date:
+        with st.spinner("Fetching weather data across all BYU locations..."):
+            combined_data = fetch_multiple_cities(start_date, end_date, selected_variable, timezone_selection)
+                
+        st.write(f"### {selected_variable_label} Data Across Locations")
+        st.dataframe(combined_data)
+        
+        avg_summary, max_summary, min_summary = calculate_summary_metrics(combined_data)
+
+        st.subheader("Key Performance Indicators (KPIs)")
+        kpi_cols = st.columns(len(locations))
+        for idx, city in enumerate(locations):
+            with kpi_cols[idx]:
+                st.metric(f"{city} - Avg", f"{avg_summary[idx]:.2f}")
+                st.metric(f"{city} - Max", f"{max_summary[idx]:.2f}")
+                st.metric(f"{city} - Min", f"{min_summary[idx]:.2f}")
+
+if __name__ == "__main__":
+    main()

pages/variable_selection.py

@@ -0,0 +1,86 @@
+import streamlit as st
+import polars as pl
+from datetime import datetime, timedelta
+from utils.weather_api import fetch_weather
+import matplotlib.pyplot as plt
+
+def main():
+    st.title("Compare Multiple Weather Variables Across BYU Locations 🌤️")
+    
+    locations = {
+        "BYU Idaho": {"lat": 43.8145, "lon": -111.7833},
+        "BYU Hawaii": {"lat": 21.6419, "lon": -157.9267},
+        "BYU Provo": {"lat": 40.25, "lon": -111.65}
+    }
+    
+    weather_variables = {
+        "Temperature (°F)": "temperature_2m",
+        "Wind Speed (mph)": "windspeed_10m",
+        "Relative Humidity (%)": "relative_humidity_2m"
+    }
+    
+    def fetch_all_variables(location, start_date, end_date, timezone_selection):
+        dataframes = {}
+        for label, variable in weather_variables.items():
+            try:
+                df = fetch_weather(location, start_date, end_date, variable, timezone_selection)
+                df = df.rename({variable: label})
+                dataframes[label] = df
+            except Exception as e:
+                st.error(f"Error fetching {label} data: {e}")
+        return dataframes
+    
+    st.sidebar.header("User Input")
+    start_date = st.sidebar.date_input("Start Date", datetime.now() - timedelta(days=15))
+    end_date = st.sidebar.date_input("End Date", datetime.now())
+    timezone_selection = st.sidebar.radio("Select Time Zone", ["America/Denver", "Pacific/Honolulu"])
+    
+    if start_date and end_date:
+        st.write("Fetching all weather variables for each BYU location...")
+        all_data = {}
+        for city, coords in locations.items():
+            st.write(f"Fetching data for **{city}**...")
+            all_data[city] = fetch_all_variables(coords, start_date, end_date, timezone_selection)
+        
+        st.subheader("Combined Weather Data Across Cities and Variables")
+        for city, dataframes in all_data.items():
+            st.write(f"**{city}**")
+            
+            datetime_df = dataframes[list(dataframes.keys())[0]].select("datetime")
+            data_without_datetime = [df.drop("datetime") for df in dataframes.values()]
+            combined_df = pl.concat([datetime_df] + data_without_datetime, how="horizontal")
+
+            st.dataframe(combined_df)
+            
+            st.subheader(f"Visualizing Weather Variables for {city}")
+            plt.figure(figsize=(10, 6))
+            for label in dataframes:
+                plt.plot(combined_df["datetime"], combined_df[label], label=label)
+            plt.xlabel("Datetime")
+            plt.ylabel("Values")
+            plt.title(f"Weather Variables for {city}")
+            plt.legend()
+            st.pyplot(plt)
+        
+        st.subheader("Boxplot of Weather Variables Across All Cities")
+        plt.figure(figsize=(12, 8))
+
+        for i, city in enumerate(all_data.keys(), start=1):
+            city_data = all_data[city]
+            
+            data_without_datetime = [df.drop("datetime") for df in city_data.values()]
+            combined_city_df = pl.concat(data_without_datetime, how="horizontal").to_pandas()
+            
+            plt.boxplot(combined_city_df.values,
+                        labels=combined_city_df.columns,
+                        positions=[i + idx * 0.3 for idx in range(len(weather_variables))],
+                        widths=0.25)
+
+        plt.title("Weather Variable Comparison")
+        plt.ylabel("Values")
+        plt.xticks(range(1, len(all_data.keys()) + 1), all_data.keys())
+        plt.grid(True)
+        st.pyplot(plt)
+
+if __name__ == "__main__":
+    main()

utils/weather_api.py

@@ -0,0 +1,32 @@
+import requests
+import polars as pl
+
+def fetch_weather(location, start_date, end_date, variable="temperature_2m", timezone="America/Denver"):
+    base_url = "https://api.open-meteo.com/v1/forecast"
+    params = {
+        "latitude": location['lat'],
+        "longitude": location['lon'],
+        "start_date": start_date,
+        "end_date": end_date,
+        "hourly": variable,
+        "timezone": timezone
+    }
+
+    response = requests.get(base_url, params=params)
+    if response.status_code == 200:
+        data = response.json()
+        hourly_data = data['hourly']
+        
+        if variable == "temperature_2m":
+            df = pl.DataFrame({
+                "datetime": hourly_data['time'],
+                variable: [temp * 9 / 5 + 32 for temp in hourly_data[variable]]
+            })
+        else:
+            df = pl.DataFrame({
+                "datetime": hourly_data['time'],
+                variable: hourly_data[variable]
+            })
+        return df
+    else:
+        raise Exception(f"API request failed: {response.status_code}")