first modification

.env_template

@@ -0,0 +1 @@
+YOUTUBE_API_KEY='YOUR YOUTUBE_API_KEY'

Dockerfile

@@ -0,0 +1,30 @@
+# Use an official Python runtime as a parent image
+FROM python:3.9-slim
+
+# Set the working directory in the container
+WORKDIR /app
+
+# Install system dependencies
+RUN apt-get update && apt-get install -y \
+    build-essential \
+    curl \
+    software-properties-common \
+    && rm -rf /var/lib/apt/lists/*
+
+# Copy the requirements file into the container
+COPY requirements.txt .
+
+# Install any needed packages specified in requirements.txt
+RUN pip install --no-cache-dir -r requirements.txt
+
+# Copy the application code into the container
+COPY app/ ./app/
+COPY config/ ./config/
+COPY data/ ./data/
+COPY grafana/ ./grafana/
+
+# Make port 8501 available to the world outside this container
+EXPOSE 8501
+
+# Run the Streamlit app when the container launches
+CMD ["streamlit", "run", "app/main.py", "--server.port=8501", "--server.address=0.0.0.0"]

README.md

@@ -1,2 +1,120 @@
-# rag-youtube-assistant
-A RAG assistant to fetch youtube transcript and build an AI chatbot on it
+# YouTube Assistant
+
+## Problem Description
+
+In the era of abundant video content on YouTube, users often struggle to efficiently extract specific information or insights from lengthy videos without watching them in their entirety. This challenge is particularly acute when dealing with educational content, tutorials, or informative videos where key points may be scattered throughout the video's duration.
+
+The YouTube Assistant project addresses this problem by providing a Retrieval-Augmented Generation (RAG) application that allows users to interact with and query video transcripts directly. This solution enables users to quickly access relevant information from YouTube videos without the need to watch them completely, saving time and improving the efficiency of information retrieval from video content.
+
+## Data
+
+The YouTube Assistant utilizes data pulled in real-time using the YouTube Data API v3. This data is then processed and stored in two databases:
+
+1. SQLite database: For structured data storage
+2. Elasticsearch vector database: For efficient similarity searches on embedded text
+
+### Data Schema
+
+The main columns in our data structure are:
+
+```json
+{
+    "content": {"type": "text"},
+    "video_id": {"type": "keyword"},
+    "segment_id": {"type": "keyword"},
+    "start_time": {"type": "float"},
+    "duration": {"type": "float"},
+    "title": {"type": "text"},
+    "author": {"type": "keyword"},
+    "upload_date": {"type": "date"},
+    "view_count": {"type": "integer"},
+    "like_count": {"type": "integer"},
+    "comment_count": {"type": "integer"},
+    "video_duration": {"type": "text"}
+}
+```
+
+This schema allows for comprehensive storage of video metadata alongside the transcript content, enabling rich querying and analysis capabilities.
+
+## Functionality
+
+The YouTube Assistant offers the following key features:
+
+1. **Real-time Data Extraction**: Utilizes the YouTube Data API v3 to fetch video data and transcripts on-demand.
+
+2. **Efficient Data Storage**: Stores structured data in SQLite and uses Elasticsearch for vector embeddings, allowing for fast retrieval and similarity searches.
+
+3. **Interactive Querying**: Provides a chat interface where users can ask questions about the video transcripts that have been downloaded or extracted in real-time.
+
+4. **Contextual Understanding**: Leverages RAG technology to understand the context of user queries and provide relevant information from the video transcripts.
+
+5. **Metadata Analysis**: Allows users to query not just the content of the videos but also metadata such as view counts, likes, and upload dates.
+
+6. **Time-stamped Responses**: Can provide information about specific segments of videos, including start times and durations.
+
+By combining these features, the YouTube Assistant empowers users to efficiently extract insights and information from YouTube videos without the need to watch them in full, significantly enhancing the way people interact with and learn from video content.
+
+## Project Structure
+
+The YouTube Assistant project is organized as follows:
+
+```
+youtube-rag-app/
+├── app/
+│   ├── main.py
+│   ├── ui.py
+│   ├── transcript_extractor.py
+│   ├── data_processor.py
+│   ├── elasticsearch_handler.py
+│   ├── database.py
+│   ├── rag.py
+│   ├── query_rewriter.py
+│   └── evaluation.py
+├── data/
+│   └── sqlite.db
+├── config/
+│   └── config.yaml
+├── requirements.txt
+├── Dockerfile
+└── docker-compose.yml
+```
+
+### Directory and File Descriptions:
+
+- `app/`: Contains the main application code
+  - `main.py`: Entry point of the application
+  - `ui.py`: Handles the user interface
+  - `transcript_extractor.py`: Manages YouTube transcript extraction
+  - `data_processor.py`: Processes and prepares data for storage and analysis
+  - `elasticsearch_handler.py`: Manages interactions with Elasticsearch
+  - `database.py`: Handles SQLite database operations
+  - `rag.py`: Implements the Retrieval-Augmented Generation logic
+  - `query_rewriter.py`: Refines and optimizes user queries
+  - `evaluation.py`: Contains evaluation metrics and functions
+- `data/`: Stores the SQLite database
+- `config/`: Contains configuration files
+- `requirements.txt`: Lists all Python dependencies
+- `Dockerfile`: Defines the Docker image for the application
+- `docker-compose.yml`: Orchestrates the application and its services
+
+## Getting Started
+
+git clone [email protected]:ganesh3/rag-youtube-assistant.git
+
+## Ingestion
+
+## Evaluation
+
+## Retrieval
+
+### RAG Flow
+
+## Monitoring
+
+
+## Usage Examples
+
+(Provide some example queries and interactions with the YouTube Assistant here.)
+
+## License
+GPL v3

app/data_processor.py

@@ -0,0 +1,145 @@
+from minsearch import Index
+from sentence_transformers import SentenceTransformer
+import numpy as np
+from sklearn.metrics.pairwise import cosine_similarity
+import re
+from elasticsearch import Elasticsearch
+import os
+
+def clean_text(text):
+    # Remove special characters and extra whitespace
+    text = re.sub(r'[^\w\s]', '', text)
+    text = re.sub(r'\s+', ' ', text).strip()
+    return text
+
+class DataProcessor:
+    def __init__(self, text_fields=["content", "title", "description"], 
+                 keyword_fields=["video_id", "start_time", "author", "upload_date"], 
+                 embedding_model="all-MiniLM-L6-v2"):
+        self.text_index = Index(text_fields=text_fields, keyword_fields=keyword_fields)
+        self.embedding_model = SentenceTransformer(embedding_model)
+        self.documents = []
+        self.embeddings = []
+        
+        # Use environment variables for Elasticsearch configuration
+        elasticsearch_host = os.getenv('ELASTICSEARCH_HOST', 'localhost')
+        elasticsearch_port = int(os.getenv('ELASTICSEARCH_PORT', 9200))
+        
+        # Initialize Elasticsearch client with explicit scheme
+        self.es = Elasticsearch([f'http://{elasticsearch_host}:{elasticsearch_port}'])
+
+    def process_transcript(self, video_id, transcript_data):
+        metadata = transcript_data['metadata']
+        transcript = transcript_data['transcript']
+
+        for i, segment in enumerate(transcript):
+            cleaned_text = clean_text(segment['text'])
+            doc = {
+                "video_id": video_id,
+                "content": cleaned_text,
+                "start_time": segment['start'],
+                "duration": segment['duration'],
+                "segment_id": f"{video_id}_{i}",
+                "title": metadata['title'],
+                "author": metadata['author'],
+                "upload_date": metadata['upload_date'],
+                "view_count": metadata['view_count'],
+                "like_count": metadata['like_count'],
+                "comment_count": metadata['comment_count'],
+                "video_duration": metadata['duration']
+            }
+            self.documents.append(doc)
+            self.embeddings.append(self.embedding_model.encode(cleaned_text + " " + metadata['title']))
+
+    def build_index(self, index_name):
+        self.text_index.fit(self.documents)
+        self.embeddings = np.array(self.embeddings)
+        
+        # Create Elasticsearch index
+        if not self.es.indices.exists(index=index_name):
+            self.es.indices.create(index=index_name, body={
+                "mappings": {
+                    "properties": {
+                        "embedding": {"type": "dense_vector", "dims": self.embeddings.shape[1]},
+                        "content": {"type": "text"},
+                        "video_id": {"type": "keyword"},
+                        "segment_id": {"type": "keyword"},
+                        "start_time": {"type": "float"},
+                        "duration": {"type": "float"},
+                        "title": {"type": "text"},
+                        "author": {"type": "keyword"},
+                        "upload_date": {"type": "date"},
+                        "view_count": {"type": "integer"},
+                        "like_count": {"type": "integer"},
+                        "comment_count": {"type": "integer"},
+                        "video_duration": {"type": "text"}
+                    }
+                }
+            })
+
+        # Index documents in Elasticsearch
+        for doc, embedding in zip(self.documents, self.embeddings):
+            doc['embedding'] = embedding.tolist()
+            self.es.index(index=index_name, body=doc, id=doc['segment_id'])
+
+    def search(self, query, filter_dict={}, boost_dict={}, num_results=10, method='hybrid', index_name=None):
+        if method == 'text':
+            return self.text_search(query, filter_dict, boost_dict, num_results)
+        elif method == 'embedding':
+            return self.embedding_search(query, num_results, index_name)
+        else:  # hybrid search
+            text_results = self.text_search(query, filter_dict, boost_dict, num_results)
+            embedding_results = self.embedding_search(query, num_results, index_name)
+            return self.combine_results(text_results, embedding_results, num_results)
+
+    def text_search(self, query, filter_dict={}, boost_dict={}, num_results=10):
+        return self.text_index.search(query, filter_dict, boost_dict, num_results)
+
+    def embedding_search(self, query, num_results=10, index_name=None):
+        if index_name:
+            # Use Elasticsearch for embedding search
+            query_vector = self.embedding_model.encode(query).tolist()
+            script_query = {
+                "script_score": {
+                    "query": {"match_all": {}},
+                    "script": {
+                        "source": "cosineSimilarity(params.query_vector, 'embedding') + 1.0",
+                        "params": {"query_vector": query_vector}
+                    }
+                }
+            }
+            response = self.es.search(
+                index=index_name,
+                body={
+                    "size": num_results,
+                    "query": script_query,
+                    "_source": {"excludes": ["embedding"]}
+                }
+            )
+            return [hit['_source'] for hit in response['hits']['hits']]
+        else:
+            # Use in-memory embedding search
+            query_embedding = self.embedding_model.encode(query)
+            similarities = cosine_similarity([query_embedding], self.embeddings)[0]
+            top_indices = np.argsort(similarities)[::-1][:num_results]
+            return [self.documents[i] for i in top_indices]
+
+    def combine_results(self, text_results, embedding_results, num_results):
+        combined = []
+        for i in range(max(len(text_results), len(embedding_results))):
+            if i < len(text_results):
+                combined.append(text_results[i])
+            if i < len(embedding_results):
+                combined.append(embedding_results[i])
+        
+        seen = set()
+        deduped = []
+        for doc in combined:
+            if doc['segment_id'] not in seen:
+                seen.add(doc['segment_id'])
+                deduped.append(doc)
+        
+        return deduped[:num_results]
+
+    def process_query(self, query):
+        return clean_text(query)

app/database.py

@@ -0,0 +1,103 @@
+import sqlite3
+import os
+
+class DatabaseHandler:
+    def __init__(self, db_path='data/sqlite.db'):
+        self.db_path = db_path
+        self.conn = None
+        self.create_tables()
+
+    def create_tables(self):
+        with sqlite3.connect(self.db_path) as conn:
+            cursor = conn.cursor()
+            cursor.execute('''
+                CREATE TABLE IF NOT EXISTS videos (
+                    id INTEGER PRIMARY KEY AUTOINCREMENT,
+                    youtube_id TEXT UNIQUE,
+                    title TEXT,
+                    channel_name TEXT,
+                    processed_date TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+                )
+            ''')
+            cursor.execute('''
+                CREATE TABLE IF NOT EXISTS user_feedback (
+                    id INTEGER PRIMARY KEY AUTOINCREMENT,
+                    video_id INTEGER,
+                    query TEXT,
+                    feedback INTEGER,
+                    timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
+                    FOREIGN KEY (video_id) REFERENCES videos (id)
+                )
+            ''')
+            cursor.execute('''
+                CREATE TABLE IF NOT EXISTS embedding_models (
+                    id INTEGER PRIMARY KEY AUTOINCREMENT,
+                    model_name TEXT UNIQUE,
+                    description TEXT
+                )
+            ''')
+            cursor.execute('''
+                CREATE TABLE IF NOT EXISTS elasticsearch_indices (
+                    id INTEGER PRIMARY KEY AUTOINCREMENT,
+                    video_id INTEGER,
+                    index_name TEXT,
+                    embedding_model_id INTEGER,
+                    FOREIGN KEY (video_id) REFERENCES videos (id),
+                    FOREIGN KEY (embedding_model_id) REFERENCES embedding_models (id)
+                )
+            ''')
+            conn.commit()
+
+    def add_video(self, youtube_id, title, channel_name):
+        with sqlite3.connect(self.db_path) as conn:
+            cursor = conn.cursor()
+            cursor.execute('''
+                INSERT OR IGNORE INTO videos (youtube_id, title, channel_name)
+                VALUES (?, ?, ?)
+            ''', (youtube_id, title, channel_name))
+            conn.commit()
+            return cursor.lastrowid
+
+    def add_user_feedback(self, video_id, query, feedback):
+        with sqlite3.connect(self.db_path) as conn:
+            cursor = conn.cursor()
+            cursor.execute('''
+                INSERT INTO user_feedback (video_id, query, feedback)
+                VALUES (?, ?, ?)
+            ''', (video_id, query, feedback))
+            conn.commit()
+
+    def add_embedding_model(self, model_name, description):
+        with sqlite3.connect(self.db_path) as conn:
+            cursor = conn.cursor()
+            cursor.execute('''
+                INSERT OR IGNORE INTO embedding_models (model_name, description)
+                VALUES (?, ?)
+            ''', (model_name, description))
+            conn.commit()
+            return cursor.lastrowid
+
+    def add_elasticsearch_index(self, video_id, index_name, embedding_model_id):
+        with sqlite3.connect(self.db_path) as conn:
+            cursor = conn.cursor()
+            cursor.execute('''
+                INSERT INTO elasticsearch_indices (video_id, index_name, embedding_model_id)
+                VALUES (?, ?, ?)
+            ''', (video_id, index_name, embedding_model_id))
+            conn.commit()
+
+    def get_video_by_youtube_id(self, youtube_id):
+        with sqlite3.connect(self.db_path) as conn:
+            cursor = conn.cursor()
+            cursor.execute('SELECT * FROM videos WHERE youtube_id = ?', (youtube_id,))
+            return cursor.fetchone()
+
+    def get_elasticsearch_index(self, video_id, embedding_model_id):
+        with sqlite3.connect(self.db_path) as conn:
+            cursor = conn.cursor()
+            cursor.execute('''
+                SELECT index_name FROM elasticsearch_indices
+                WHERE video_id = ? AND embedding_model_id = ?
+            ''', (video_id, embedding_model_id))
+            result = cursor.fetchone()
+            return result[0] if result else None

app/elasticsearch_handler.py

@@ -0,0 +1,37 @@
+from elasticsearch import Elasticsearch
+import uuid
+
+class ElasticsearchHandler:
+    def __init__(self, host='localhost', port=9200):
+        self.es = Elasticsearch([{'host': host, 'port': port}])
+
+    def create_index(self, index_name):
+        if not self.es.indices.exists(index=index_name):
+            self.es.indices.create(index=index_name)
+
+    def index_document(self, index_name, doc_id, text, embedding):
+        body = {
+            'text': text,
+            'embedding': embedding.tolist()
+        }
+        self.es.index(index=index_name, id=doc_id, body=body)
+
+    def search(self, index_name, query_vector, top_k=5):
+        script_query = {
+            "script_score": {
+                "query": {"match_all": {}},
+                "script": {
+                    "source": "cosineSimilarity(params.query_vector, 'embedding') + 1.0",
+                    "params": {"query_vector": query_vector.tolist()}
+                }
+            }
+        }
+        response = self.es.search(
+            index=index_name,
+            body={
+                "size": top_k,
+                "query": script_query,
+                "_source": {"includes": ["text"]}
+            }
+        )
+        return [hit["_source"]["text"] for hit in response["hits"]["hits"]]

app/evaluation.py

@@ -0,0 +1,48 @@
+from sklearn.metrics.pairwise import cosine_similarity
+import numpy as np
+
+class EvaluationSystem:
+    def __init__(self, data_processor, database_handler):
+        self.data_processor = data_processor
+        self.db_handler = database_handler
+
+    def relevance_scoring(self, query, retrieved_docs, top_k=5):
+        query_embedding = self.data_processor.process_query(query)
+        doc_embeddings = [self.data_processor.process_query(doc) for doc in retrieved_docs]
+        
+        similarities = cosine_similarity([query_embedding], doc_embeddings)[0]
+        return np.mean(sorted(similarities, reverse=True)[:top_k])
+
+    def answer_similarity(self, generated_answer, reference_answer):
+        gen_embedding = self.data_processor.process_query(generated_answer)
+        ref_embedding = self.data_processor.process_query(reference_answer)
+        return cosine_similarity([gen_embedding], [ref_embedding])[0][0]
+
+    def human_evaluation(self, video_id, query):
+        with self.db_handler.conn:
+            cursor = self.db_handler.conn.cursor()
+            cursor.execute('''
+                SELECT AVG(feedback) FROM user_feedback
+                WHERE video_id = ? AND query = ?
+            ''', (video_id, query))
+            result = cursor.fetchone()
+            return result[0] if result[0] is not None else 0
+
+    def evaluate_rag_performance(self, rag_system, test_queries, reference_answers, index_name):
+        relevance_scores = []
+        similarity_scores = []
+        human_scores = []
+
+        for query, reference in zip(test_queries, reference_answers):
+            retrieved_docs = rag_system.es_handler.search(index_name, rag_system.data_processor.process_query(query))
+            generated_answer = rag_system.query(index_name, query)
+
+            relevance_scores.append(self.relevance_scoring(query, retrieved_docs))
+            similarity_scores.append(self.answer_similarity(generated_answer, reference))
+            human_scores.append(self.human_evaluation(index_name, query))  # Assuming index_name can be used as video_id
+
+        return {
+            "avg_relevance_score": np.mean(relevance_scores),
+            "avg_similarity_score": np.mean(similarity_scores),
+            "avg_human_score": np.mean(human_scores)
+        }

app/generate_ground_truth.py

@@ -0,0 +1,67 @@
+import os
+import pandas as pd
+import json
+from youtube_transcript_api import YouTubeTranscriptApi
+from tqdm import tqdm
+import requests
+
+OLLAMA_HOST = os.getenv('OLLAMA_HOST', 'localhost')
+OLLAMA_PORT = os.getenv('OLLAMA_PORT', '11434')
+
+def get_transcript(video_id):
+    try:
+        transcript = YouTubeTranscriptApi.get_transcript(video_id)
+        return " ".join([entry['text'] for entry in transcript])
+    except Exception as e:
+        print(f"Error extracting transcript for video {video_id}: {str(e)}")
+        return None
+
+def generate_questions(transcript):
+    prompt_template = """
+    You are an AI assistant tasked with generating questions based on a YouTube video transcript.
+    Formulate 10 questions that a user might ask based on the provided transcript.
+    Make the questions specific to the content of the transcript.
+    The questions should be complete and not too short. Use as few words as possible from the transcript.
+
+    The transcript:
+
+    {transcript}
+
+    Provide the output in parsable JSON without using code blocks:
+
+    {{"questions": ["question1", "question2", ..., "question10"]}}
+    """.strip()
+
+    prompt = prompt_template.format(transcript=transcript)
+
+    response = requests.post(f'http://{OLLAMA_HOST}:{OLLAMA_PORT}/api/generate', json={
+        'model': 'phi3.5',
+        'prompt': prompt
+    })
+    
+    if response.status_code == 200:
+        return json.loads(response.json()['response'])
+    else:
+        print(f"Error: {response.status_code} - {response.text}")
+        return None
+
+def main():
+    video_id = "zjkBMFhNj_g"
+    transcript = get_transcript(video_id)
+    
+    if transcript:
+        questions = generate_questions(transcript)
+        
+        if questions:
+            df = pd.DataFrame([(video_id, q) for q in questions['questions']], columns=['video_id', 'question'])
+            
+            os.makedirs('data', exist_ok=True)
+            df.to_csv('data/ground-truth-retrieval.csv', index=False)
+            print("Ground truth data saved to data/ground-truth-retrieval.csv")
+        else:
+            print("Failed to generate questions.")
+    else:
+        print("Failed to generate ground truth data due to transcript retrieval error.")
+        
+if __name__ == "__main__":
+    main()

app/main.py

@@ -0,0 +1,287 @@
+import streamlit as st
+import pandas as pd
+from transcript_extractor import extract_video_id, get_transcript, get_channel_videos, process_videos
+from data_processor import DataProcessor
+from database import DatabaseHandler
+from rag import RAGSystem
+from query_rewriter import QueryRewriter
+from evaluation import EvaluationSystem
+from sentence_transformers import SentenceTransformer
+import os
+import json
+import requests
+from tqdm import tqdm
+import sqlite3
+
+# Initialize components
+@st.cache_resource
+def init_components():
+    db_handler = DatabaseHandler()
+    data_processor = DataProcessor()
+    rag_system = RAGSystem(data_processor)
+    query_rewriter = QueryRewriter()
+    evaluation_system = EvaluationSystem(data_processor, db_handler)
+    return db_handler, data_processor, rag_system, query_rewriter, evaluation_system
+
+db_handler, data_processor, rag_system, query_rewriter, evaluation_system = init_components()
+
+# Ground Truth Generation
+def generate_questions(transcript):
+    OLLAMA_HOST = os.getenv('OLLAMA_HOST', 'localhost')
+    OLLAMA_PORT = os.getenv('OLLAMA_PORT', '11434')
+    prompt_template = """
+    You are an AI assistant tasked with generating questions based on a YouTube video transcript.
+    Formulate 10 questions that a user might ask based on the provided transcript.
+    Make the questions specific to the content of the transcript.
+    The questions should be complete and not too short. Use as few words as possible from the transcript.
+
+    The transcript:
+
+    {transcript}
+
+    Provide the output in parsable JSON without using code blocks:
+
+    {{"questions": ["question1", "question2", ..., "question10"]}}
+    """.strip()
+
+    prompt = prompt_template.format(transcript=transcript)
+
+    try:
+        response = requests.post(f'http://{OLLAMA_HOST}:{OLLAMA_PORT}/api/generate', json={
+            'model': 'phi3.5',
+            'prompt': prompt
+        })
+        response.raise_for_status()
+        return json.loads(response.json()['response'])
+    except requests.RequestException as e:
+        st.error(f"Error generating questions: {str(e)}")
+        return None
+
+def generate_ground_truth(video_id):
+    transcript_data = get_transcript(video_id)
+    
+    if transcript_data and 'transcript' in transcript_data:
+        full_transcript = " ".join([entry['text'] for entry in transcript_data['transcript']])
+        questions = generate_questions(full_transcript)
+        
+        if questions and 'questions' in questions:
+            df = pd.DataFrame([(video_id, q) for q in questions['questions']], columns=['video_id', 'question'])
+            
+            os.makedirs('data', exist_ok=True)
+            df.to_csv('data/ground-truth-retrieval.csv', index=False)
+            st.success("Ground truth data generated and saved to data/ground-truth-retrieval.csv")
+            return df
+        else:
+            st.error("Failed to generate questions.")
+    else:
+        st.error("Failed to generate ground truth data due to transcript retrieval error.")
+    return None
+
+# RAG Evaluation
+def evaluate_rag(sample_size=200):
+    try:
+        ground_truth = pd.read_csv('data/ground-truth-retrieval.csv')
+    except FileNotFoundError:
+        st.error("Ground truth file not found. Please generate ground truth data first.")
+        return None
+
+    sample = ground_truth.sample(n=min(sample_size, len(ground_truth)), random_state=1)
+    evaluations = []
+    
+    prompt_template = """
+    You are an expert evaluator for a Youtube transcript assistant.
+    Your task is to analyze the relevance of the generated answer to the given question.
+    Based on the relevance of the generated answer, you will classify it
+    as "NON_RELEVANT", "PARTLY_RELEVANT", or "RELEVANT".
+
+    Here is the data for evaluation:
+
+    Question: {question}
+    Generated Answer: {answer_llm}
+
+    Please analyze the content and context of the generated answer in relation to the question
+    and provide your evaluation in parsable JSON without using code blocks:
+
+    {{
+      "Relevance": "NON_RELEVANT" | "PARTLY_RELEVANT" | "RELEVANT",
+      "Explanation": "[Provide a brief explanation for your evaluation]"
+    }}
+    """.strip()
+
+    progress_bar = st.progress(0)
+    for i, (_, row) in enumerate(sample.iterrows()):
+        question = row['question']
+        answer_llm = rag_system.query(question)
+        prompt = prompt_template.format(question=question, answer_llm=answer_llm)
+        evaluation = rag_system.query(prompt)  # Assuming rag_system can handle this type of query
+        try:
+            evaluation_json = json.loads(evaluation)
+            evaluations.append((row['video_id'], question, answer_llm, evaluation_json['Relevance'], evaluation_json['Explanation']))
+        except json.JSONDecodeError:
+            st.warning(f"Failed to parse evaluation for question: {question}")
+        progress_bar.progress((i + 1) / len(sample))
+
+    # Store RAG evaluations in the database
+    conn = sqlite3.connect('data/sqlite.db')
+    cursor = conn.cursor()
+    cursor.execute('''
+    CREATE TABLE IF NOT EXISTS rag_evaluations (
+        video_id TEXT,
+        question TEXT,
+        answer TEXT,
+        relevance TEXT,
+        explanation TEXT
+    )
+    ''')
+    cursor.executemany('''
+    INSERT INTO rag_evaluations (video_id, question, answer, relevance, explanation)
+    VALUES (?, ?, ?, ?, ?)
+    ''', evaluations)
+    conn.commit()
+    conn.close()
+
+    st.success("Evaluation complete. Results stored in the database.")
+    return evaluations
+
+def main():
+    st.title("YouTube Transcript RAG System")
+
+    tab1, tab2, tab3 = st.tabs(["RAG System", "Ground Truth Generation", "Evaluation"])
+
+    with tab1:
+        st.header("RAG System")
+        # Input section
+        input_type = st.radio("Select input type:", ["Video URL", "Channel URL", "YouTube ID"])
+        input_value = st.text_input("Enter the URL or ID:")
+        embedding_model = st.selectbox("Select embedding model:", ["all-MiniLM-L6-v2", "all-mpnet-base-v2"])
+
+        if st.button("Process"):
+            with st.spinner("Processing..."):
+                data_processor.embedding_model = SentenceTransformer(embedding_model)
+                if input_type == "Video URL":
+                    video_id = extract_video_id(input_value)
+                    if video_id:
+                        process_single_video(video_id, embedding_model)
+                    else:
+                        st.error("Failed to extract video ID from the URL")
+                elif input_type == "Channel URL":
+                    channel_videos = get_channel_videos(input_value)
+                    if channel_videos:
+                        process_multiple_videos([video['video_id'] for video in channel_videos], embedding_model)
+                    else:
+                        st.error("Failed to retrieve videos from the channel")
+                else:
+                    process_single_video(input_value, embedding_model)
+
+        # Query section
+        st.subheader("Query the RAG System")
+        query = st.text_input("Enter your query:")
+        rewrite_method = st.radio("Query rewriting method:", ["None", "Chain of Thought", "ReAct"])
+        search_method = st.radio("Search method:", ["Hybrid", "Text-only", "Embedding-only"])
+
+        if st.button("Search"):
+            with st.spinner("Searching..."):
+                if rewrite_method == "Chain of Thought":
+                    query = query_rewriter.rewrite_cot(query)
+                elif rewrite_method == "ReAct":
+                    query = query_rewriter.rewrite_react(query)
+
+                search_method_map = {"Hybrid": "hybrid", "Text-only": "text", "Embedding-only": "embedding"}
+                response = rag_system.query(query, search_method=search_method_map[search_method])
+                st.write("Response:", response)
+
+            # Feedback
+            feedback = st.radio("Provide feedback:", ["+1", "-1"])
+            if st.button("Submit Feedback"):
+                db_handler.add_user_feedback("all_videos", query, 1 if feedback == "+1" else -1)
+                st.success("Feedback submitted successfully!")
+
+    with tab2:
+        st.header("Ground Truth Generation")
+        video_id = st.text_input("Enter YouTube Video ID for ground truth generation:")
+        if st.button("Generate Ground Truth"):
+            with st.spinner("Generating ground truth..."):
+                ground_truth_df = generate_ground_truth(video_id)
+                if ground_truth_df is not None:
+                    st.dataframe(ground_truth_df)
+                    csv = ground_truth_df.to_csv(index=False)
+                    st.download_button(
+                        label="Download Ground Truth CSV",
+                        data=csv,
+                        file_name="ground_truth.csv",
+                        mime="text/csv",
+                    )
+
+    with tab3:
+        st.header("RAG Evaluation")
+        sample_size = st.number_input("Enter sample size for evaluation:", min_value=1, max_value=1000, value=200)
+        if st.button("Run Evaluation"):
+            with st.spinner("Running evaluation..."):
+                evaluation_results = evaluate_rag(sample_size)
+                if evaluation_results:
+                    st.write("Evaluation Results:")
+                    st.dataframe(pd.DataFrame(evaluation_results, columns=['Video ID', 'Question', 'Answer', 'Relevance', 'Explanation']))
+
+@st.cache_data
+def process_single_video(video_id, embedding_model):
+    # Check if the video has already been processed with the current embedding model
+    existing_index = db_handler.get_elasticsearch_index(video_id, embedding_model)
+    if existing_index:
+        st.info(f"Video {video_id} has already been processed with {embedding_model}. Using existing index: {existing_index}")
+        return existing_index
+
+    transcript_data = get_transcript(video_id)
+    if transcript_data:
+        # Store video metadata in the database
+        video_data = {
+            'video_id': video_id,
+            'title': transcript_data['metadata'].get('title', 'Unknown Title'),
+            'author': transcript_data['metadata'].get('author', 'Unknown Author'),
+            'upload_date': transcript_data['metadata'].get('upload_date', 'Unknown Date'),
+            'view_count': int(transcript_data['metadata'].get('view_count', 0)),
+            'like_count': int(transcript_data['metadata'].get('like_count', 0)),
+            'comment_count': int(transcript_data['metadata'].get('comment_count', 0)),
+            'video_duration': transcript_data['metadata'].get('duration', 'Unknown Duration')
+        }
+        db_handler.add_video(video_data)
+
+        # Store transcript segments in the database
+        for i, segment in enumerate(transcript_data['transcript']):
+            segment_data = {
+                'segment_id': f"{video_id}_{i}",
+                'video_id': video_id,
+                'content': segment.get('text', ''),
+                'start_time': segment.get('start', 0),
+                'duration': segment.get('duration', 0)
+            }
+            db_handler.add_transcript_segment(segment_data)
+
+        # Process transcript for RAG system
+        data_processor.process_transcript(video_id, transcript_data)
+        
+        # Create Elasticsearch index
+        index_name = f"video_{video_id}_{embedding_model}"
+        data_processor.build_index(index_name)
+        
+        # Store Elasticsearch index information
+        db_handler.add_elasticsearch_index(video_id, index_name, embedding_model)
+        
+        st.success(f"Processed and indexed transcript for video {video_id}")
+        st.write("Metadata:", transcript_data['metadata'])
+        return index_name
+    else:
+        st.error(f"Failed to retrieve transcript for video {video_id}")
+        return None
+
+@st.cache_data
+def process_multiple_videos(video_ids, embedding_model):
+    indices = []
+    for video_id in video_ids:
+        index = process_single_video(video_id, embedding_model)
+        if index:
+            indices.append(index)
+    st.success(f"Processed and indexed transcripts for {len(indices)} videos")
+    return indices
+
+if __name__ == "__main__":
+    main()

app/minsearch.py

@@ -0,0 +1,96 @@
+import pandas as pd
+
+from sklearn.feature_extraction.text import TfidfVectorizer
+from sklearn.metrics.pairwise import cosine_similarity
+
+import numpy as np
+
+
+class Index:
+    """
+    A simple search index using TF-IDF and cosine similarity for text fields and exact matching for keyword fields.
+
+    Attributes:
+        text_fields (list): List of text field names to index.
+        keyword_fields (list): List of keyword field names to index.
+        vectorizers (dict): Dictionary of TfidfVectorizer instances for each text field.
+        keyword_df (pd.DataFrame): DataFrame containing keyword field data.
+        text_matrices (dict): Dictionary of TF-IDF matrices for each text field.
+        docs (list): List of documents indexed.
+    """
+
+    def __init__(self, text_fields, keyword_fields, vectorizer_params={}):
+        """
+        Initializes the Index with specified text and keyword fields.
+
+        Args:
+            text_fields (list): List of text field names to index.
+            keyword_fields (list): List of keyword field names to index.
+            vectorizer_params (dict): Optional parameters to pass to TfidfVectorizer.
+        """
+        self.text_fields = text_fields
+        self.keyword_fields = keyword_fields
+
+        self.vectorizers = {field: TfidfVectorizer(**vectorizer_params) for field in text_fields}
+        self.keyword_df = None
+        self.text_matrices = {}
+        self.docs = []
+
+    def fit(self, docs):
+        """
+        Fits the index with the provided documents.
+
+        Args:
+            docs (list of dict): List of documents to index. Each document is a dictionary.
+        """
+        self.docs = docs
+        keyword_data = {field: [] for field in self.keyword_fields}
+
+        for field in self.text_fields:
+            texts = [doc.get(field, '') for doc in docs]
+            self.text_matrices[field] = self.vectorizers[field].fit_transform(texts)
+
+        for doc in docs:
+            for field in self.keyword_fields:
+                keyword_data[field].append(doc.get(field, ''))
+
+        self.keyword_df = pd.DataFrame(keyword_data)
+
+        return self
+
+    def search(self, query, filter_dict={}, boost_dict={}, num_results=10):
+        """
+        Searches the index with the given query, filters, and boost parameters.
+
+        Args:
+            query (str): The search query string.
+            filter_dict (dict): Dictionary of keyword fields to filter by. Keys are field names and values are the values to filter by.
+            boost_dict (dict): Dictionary of boost scores for text fields. Keys are field names and values are the boost scores.
+            num_results (int): The number of top results to return. Defaults to 10.
+
+        Returns:
+            list of dict: List of documents matching the search criteria, ranked by relevance.
+        """
+        query_vecs = {field: self.vectorizers[field].transform([query]) for field in self.text_fields}
+        scores = np.zeros(len(self.docs))
+
+        # Compute cosine similarity for each text field and apply boost
+        for field, query_vec in query_vecs.items():
+            sim = cosine_similarity(query_vec, self.text_matrices[field]).flatten()
+            boost = boost_dict.get(field, 1)
+            scores += sim * boost
+
+        # Apply keyword filters
+        for field, value in filter_dict.items():
+            if field in self.keyword_fields:
+                mask = self.keyword_df[field] == value
+                scores = scores * mask.to_numpy()
+
+        # Use argpartition to get top num_results indices
+        top_indices = np.argpartition(scores, -num_results)[-num_results:]
+        top_indices = top_indices[np.argsort(-scores[top_indices])]
+
+        # Filter out zero-score results
+        top_docs = [self.docs[i] for i in top_indices if scores[i] > 0]
+
+        return top_docs

app/query_rewriter.py

@@ -0,0 +1,33 @@
+import ollama
+
+class QueryRewriter:
+    def __init__(self):
+        self.model = "phi"  # Using Phi-3.5 model
+
+    def rewrite_cot(self, query):
+        prompt = f"""
+        Rewrite the following query using Chain-of-Thought reasoning:
+        Query: {query}
+        
+        Rewritten query:
+        """
+        response = ollama.generate(model=self.model, prompt=prompt)
+        return response['response'].strip()
+
+    def rewrite_react(self, query):
+        prompt = f"""
+        Rewrite the following query using the ReAct framework (Reasoning and Acting):
+        Query: {query}
+        
+        Thought 1:
+        Action 1:
+        Observation 1:
+        
+        Thought 2:
+        Action 2:
+        Observation 2:
+        
+        Final rewritten query:
+        """
+        response = ollama.generate(model=self.model, prompt=prompt)
+        return response['response'].strip()

app/rag.py

@@ -0,0 +1,31 @@
+import ollama
+
+class RAGSystem:
+    def __init__(self, data_processor):
+        self.data_processor = data_processor
+        self.model = "phi3.5"  # Using Phi-3.5 model
+
+    def query(self, user_query, top_k=3, search_method='hybrid'):
+        # Retrieve relevant documents using the specified search method
+        relevant_docs = self.data_processor.search(user_query, num_results=top_k, method=search_method)
+        
+        # Construct the prompt
+        context = "\n".join([doc['content'] for doc in relevant_docs])
+        prompt = f"Context: {context}\n\nQuestion: {user_query}\n\nAnswer:"
+        
+        # Generate response using Ollama
+        response = ollama.generate(model=self.model, prompt=prompt)
+        
+        return response['response']
+
+    def rerank_documents(self, documents, query):
+        # Implement a simple re-ranking strategy
+        # This could be improved with more sophisticated methods
+        reranked = sorted(documents, key=lambda doc: self.calculate_relevance(doc['content'], query), reverse=True)
+        return reranked
+
+    def calculate_relevance(self, document, query):
+        # Simple relevance calculation based on word overlap
+        doc_words = set(document.lower().split())
+        query_words = set(query.lower().split())
+        return len(doc_words.intersection(query_words)) / len(query_words)

app/rag_evaluation.py

@@ -0,0 +1,193 @@
+import pandas as pd
+import numpy as np
+from tqdm import tqdm
+import json
+import requests
+import sqlite3
+from minsearch import Index
+
+# Database connection
+conn = sqlite3.connect('data/sqlite.db')
+cursor = conn.cursor()
+
+# Load ground truth data from CSV
+def load_ground_truth():
+    return pd.read_csv('data/ground-truth-retrieval.csv')
+
+ground_truth = load_ground_truth()
+
+# Load transcript data
+def load_transcripts():
+    cursor.execute("SELECT * FROM transcript_segments")
+    rows = cursor.fetchall()
+    return pd.DataFrame(rows, columns=['segment_id', 'video_id', 'content', 'start_time', 'duration'])
+
+transcripts = load_transcripts()
+
+# Create index
+index = Index(
+    text_fields=['content'],
+    keyword_fields=['video_id', 'segment_id']
+)
+index.fit(transcripts.to_dict('records'))
+
+# RAG flow
+def search(query):
+    boost = {}
+    results = index.search(
+        query=query,
+        filter_dict={},
+        boost_dict=boost,
+        num_results=10
+    )
+    return results
+
+prompt_template = """
+You're an AI assistant for YouTube video transcripts. Answer the QUESTION based on the CONTEXT from our transcript database.
+Use only the facts from the CONTEXT when answering the QUESTION.
+
+QUESTION: {question}
+
+CONTEXT:
+{context}
+""".strip()
+
+def build_prompt(query, search_results):
+    context = "\n\n".join([f"Segment {i+1}: {result['content']}" for i, result in enumerate(search_results)])
+    prompt = prompt_template.format(question=query, context=context).strip()
+    return prompt
+
+def llm(prompt):
+    response = requests.post('http://localhost:11434/api/generate', json={
+        'model': 'phi',
+        'prompt': prompt
+    })
+    if response.status_code == 200:
+        return response.json()['response']
+    else:
+        print(f"Error: {response.status_code} - {response.text}")
+        return None
+
+def rag(query):
+    search_results = search(query)
+    prompt = build_prompt(query, search_results)
+    answer = llm(prompt)
+    return answer
+
+# Evaluation metrics
+def hit_rate(relevance_total):
+    return sum(any(line) for line in relevance_total) / len(relevance_total)
+
+def mrr(relevance_total):
+    scores = []
+    for line in relevance_total:
+        for rank, relevant in enumerate(line, 1):
+            if relevant:
+                scores.append(1 / rank)
+                break
+        else:
+            scores.append(0)
+    return sum(scores) / len(scores)
+
+def evaluate(ground_truth, search_function):
+    relevance_total = []
+    for _, row in tqdm(ground_truth.iterrows(), total=len(ground_truth)):
+        video_id = row['video_id']
+        results = search_function(row['question'])
+        relevance = [d['video_id'] == video_id for d in results]
+        relevance_total.append(relevance)
+    return {
+        'hit_rate': hit_rate(relevance_total),
+        'mrr': mrr(relevance_total),
+    }
+
+# Parameter optimization
+param_ranges = {
+    'content': (0.0, 3.0),
+}
+
+def simple_optimize(param_ranges, objective_function, n_iterations=10):
+    best_params = None
+    best_score = float('-inf')
+    for _ in range(n_iterations):
+        current_params = {param: np.random.uniform(min_val, max_val) 
+                          for param, (min_val, max_val) in param_ranges.items()}
+        current_score = objective_function(current_params)
+        if current_score > best_score:
+            best_score = current_score
+            best_params = current_params
+    return best_params, best_score
+
+def objective(boost_params):
+    def search_function(q):
+        return search(q, boost_params)
+    results = evaluate(ground_truth, search_function)
+    return results['mrr']
+
+# RAG evaluation
+prompt2_template = """
+You are an expert evaluator for a Youtube transcript assistant.
+Your task is to analyze the relevance of the generated answer to the given question.
+Based on the relevance of the generated answer, you will classify it
+as "NON_RELEVANT", "PARTLY_RELEVANT", or "RELEVANT".
+
+Here is the data for evaluation:
+
+Question: {question}
+Generated Answer: {answer_llm}
+
+Please analyze the content and context of the generated answer in relation to the question
+and provide your evaluation in parsable JSON without using code blocks:
+
+{{
+  "Relevance": "NON_RELEVANT" | "PARTLY_RELEVANT" | "RELEVANT",
+  "Explanation": "[Provide a brief explanation for your evaluation]"
+}}
+""".strip()
+
+def evaluate_rag(sample_size=200):
+    sample = ground_truth.sample(n=sample_size, random_state=1)
+    evaluations = []
+    for _, row in tqdm(sample.iterrows(), total=len(sample)):
+        question = row['question']
+        answer_llm = rag(question)
+        prompt = prompt2_template.format(question=question, answer_llm=answer_llm)
+        evaluation = llm(prompt)
+        evaluation = json.loads(evaluation)
+        evaluations.append((row['video_id'], question, answer_llm, evaluation['Relevance'], evaluation['Explanation']))
+    return evaluations
+
+# Main execution
+if __name__ == "__main__":
+    print("Evaluating search performance...")
+    search_performance = evaluate(ground_truth, lambda q: search(q['question']))
+    print(f"Search performance: {search_performance}")
+
+    print("\nOptimizing search parameters...")
+    best_params, best_score = simple_optimize(param_ranges, objective, n_iterations=20)
+    print(f"Best parameters: {best_params}")
+    print(f"Best score: {best_score}")
+
+    print("\nEvaluating RAG performance...")
+    rag_evaluations = evaluate_rag(sample_size=200)
+    
+    # Store RAG evaluations in the database
+    cursor.execute('''
+    CREATE TABLE IF NOT EXISTS rag_evaluations (
+        video_id TEXT,
+        question TEXT,
+        answer TEXT,
+        relevance TEXT,
+        explanation TEXT
+    )
+    ''')
+    cursor.executemany('''
+    INSERT INTO rag_evaluations (video_id, question, answer, relevance, explanation)
+    VALUES (?, ?, ?, ?, ?)
+    ''', rag_evaluations)
+    conn.commit()
+
+    print("Evaluation complete. Results stored in the database.")
+
+    # Close the database connection
+    conn.close()

app/transcript_extractor.py

@@ -0,0 +1,85 @@
+from youtube_transcript_api import YouTubeTranscriptApi
+from googleapiclient.discovery import build
+from googleapiclient.errors import HttpError
+import re
+import os
+
+# Replace with your actual API key
+API_KEY = os.environ.get('YOUTUBE_API_KEY', 'YOUR_API_KEY_HERE')
+
+youtube = build('youtube', 'v3', developerKey=API_KEY)
+
+def extract_video_id(url):
+    video_id_match = re.search(r"(?:v=|\/)([0-9A-Za-z_-]{11}).*", url)
+    if video_id_match:
+        return video_id_match.group(1)
+    return None
+
+def get_video_metadata(video_id):
+    try:
+        request = youtube.videos().list(
+            part="snippet,contentDetails,statistics",
+            id=video_id
+        )
+        response = request.execute()
+
+        if 'items' in response and len(response['items']) > 0:
+            video = response['items'][0]
+            snippet = video['snippet']
+            return {
+                'title': snippet['title'],
+                'author': snippet['channelTitle'],
+                'upload_date': snippet['publishedAt'],
+                'view_count': video['statistics']['viewCount'],
+                'like_count': video['statistics'].get('likeCount', 'N/A'),
+                'comment_count': video['statistics'].get('commentCount', 'N/A'),
+                'duration': video['contentDetails']['duration']
+            }
+        else:
+            return None
+    except HttpError as e:
+        print(f"An HTTP error {e.resp.status} occurred: {e.content}")
+        return None
+
+def get_transcript(video_id):
+    try:
+        transcript = YouTubeTranscriptApi.get_transcript(video_id)
+        metadata = get_video_metadata(video_id)
+        return {
+            'transcript': transcript,
+            'metadata': metadata
+        }
+    except Exception as e:
+        print(f"Error extracting transcript for video {video_id}: {str(e)}")
+        return None
+
+def get_channel_videos(channel_id):
+    try:
+        request = youtube.search().list(
+            part="id,snippet",
+            channelId=channel_id,
+            type="video",
+            maxResults=50  # Adjust as needed
+        )
+        response = request.execute()
+
+        videos = []
+        for item in response['items']:
+            videos.append({
+                'video_id': item['id']['videoId'],
+                'title': item['snippet']['title'],
+                'description': item['snippet']['description'],
+                'published_at': item['snippet']['publishedAt']
+            })
+        return videos
+    except HttpError as e:
+        print(f"An HTTP error {e.resp.status} occurred: {e.content}")
+        return []
+
+def process_videos(video_ids):
+    transcripts = {}
+    for video_id in video_ids:
+        transcript_data = get_transcript(video_id)
+        if transcript_data:
+            transcripts[video_id] = transcript_data
+    return transcripts

docker-compose.yaml

@@ -0,0 +1,42 @@
+version: '3.8'
+
+services:
+  app:
+    build: .
+    ports:
+      - "8501:8501"
+    depends_on:
+      - elasticsearch
+    environment:
+      - ELASTICSEARCH_HOST=elasticsearch
+      - ELASTICSEARCH_PORT=9200
+      - YOUTUBE_API_KEY=${YOUTUBE_API_KEY}
+    env_file:
+      - .env
+    volumes:
+      - ./data:/app/data
+      - ./config:/app/config
+
+  elasticsearch:
+    image: docker.elastic.co/elasticsearch/elasticsearch:7.14.0
+    environment:
+      - discovery.type=single-node
+      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
+    ports:
+      - "9200:9200"
+    volumes:
+      - esdata:/usr/share/elasticsearch/data
+
+  grafana:
+    image: grafana/grafana:latest
+    ports:
+      - "3000:3000"
+    volumes:
+      - grafana-storage:/var/lib/grafana
+      - ./config/grafana:/etc/grafana/provisioning
+    depends_on:
+      - elasticsearch
+
+volumes:
+  esdata:
+  grafana-storage:

grafana/provisioning/dashboards/rag_evaluation.json

@@ -0,0 +1,129 @@
+{
+    "annotations": {
+      "list": [
+        {
+          "builtIn": 1,
+          "datasource": "-- Grafana --",
+          "enable": true,
+          "hide": true,
+          "iconColor": "rgba(0, 211, 255, 1)",
+          "name": "Annotations & Alerts",
+          "type": "dashboard"
+        }
+      ]
+    },
+    "editable": true,
+    "gnetId": null,
+    "graphTooltip": 0,
+    "id": 1,
+    "links": [],
+    "panels": [
+      {
+        "aliasColors": {},
+        "bars": false,
+        "dashLength": 10,
+        "dashes": false,
+        "datasource": "SQLite",
+        "fieldConfig": {
+          "defaults": {},
+          "overrides": []
+        },
+        "fill": 1,
+        "fillGradient": 0,
+        "gridPos": {
+          "h": 9,
+          "w": 12,
+          "x": 0,
+          "y": 0
+        },
+        "hiddenSeries": false,
+        "id": 2,
+        "legend": {
+          "avg": false,
+          "current": false,
+          "max": false,
+          "min": false,
+          "show": true,
+          "total": false,
+          "values": false
+        },
+        "lines": true,
+        "linewidth": 1,
+        "nullPointMode": "null",
+        "options": {
+          "alertThreshold": true
+        },
+        "percentage": false,
+        "pluginVersion": "7.5.7",
+        "pointradius": 2,
+        "points": false,
+        "renderer": "flot",
+        "seriesOverrides": [],
+        "spaceLength": 10,
+        "stack": false,
+        "steppedLine": false,
+        "targets": [
+          {
+            "queryType": "table",
+            "refId": "A",
+            "sql": "SELECT relevance, COUNT(*) as count FROM rag_evaluations GROUP BY relevance"
+          }
+        ],
+        "thresholds": [],
+        "timeFrom": null,
+        "timeRegions": [],
+        "timeShift": null,
+        "title": "RAG Evaluation Results",
+        "tooltip": {
+          "shared": true,
+          "sort": 0,
+          "value_type": "individual"
+        },
+        "type": "graph",
+        "xaxis": {
+          "buckets": null,
+          "mode": "categories",
+          "name": null,
+          "show": true,
+          "values": []
+        },
+        "yaxes": [
+          {
+            "format": "short",
+            "label": null,
+            "logBase": 1,
+            "max": null,
+            "min": null,
+            "show": true
+          },
+          {
+            "format": "short",
+            "label": null,
+            "logBase": 1,
+            "max": null,
+            "min": null,
+            "show": true
+          }
+        ],
+        "yaxis": {
+          "align": false,
+          "alignLevel": null
+        }
+      }
+    ],
+    "schemaVersion": 27,
+    "style": "dark",
+    "tags": [],
+    "templating": {
+      "list": []
+    },
+    "time": {
+      "from": "now-6h",
+      "to": "now"
+    },
+    "timepicker": {},
+    "timezone": "",
+    "title": "RAG Evaluation Dashboard",
+    "uid": "rag_evaluation",
+    "version": 1
+  }

grafana/provisioning/datasources/sqlite.yaml

@@ -0,0 +1,7 @@
+apiVersion: 1
+
+datasources:
+  - name: SQLite
+    type: sqlite
+    url: /app/data/sqlite.db
+    isDefault: true

llmrag/Scripts/Activate.ps1

@@ -0,0 +1,472 @@
+<#
+.Synopsis
+Activate a Python virtual environment for the current PowerShell session.
+
+.Description
+Pushes the python executable for a virtual environment to the front of the
+$Env:PATH environment variable and sets the prompt to signify that you are
+in a Python virtual environment. Makes use of the command line switches as
+well as the `pyvenv.cfg` file values present in the virtual environment.
+
+.Parameter VenvDir
+Path to the directory that contains the virtual environment to activate. The
+default value for this is the parent of the directory that the Activate.ps1
+script is located within.
+
+.Parameter Prompt
+The prompt prefix to display when this virtual environment is activated. By
+default, this prompt is the name of the virtual environment folder (VenvDir)
+surrounded by parentheses and followed by a single space (ie. '(.venv) ').
+
+.Example
+Activate.ps1
+Activates the Python virtual environment that contains the Activate.ps1 script.
+
+.Example
+Activate.ps1 -Verbose
+Activates the Python virtual environment that contains the Activate.ps1 script,
+and shows extra information about the activation as it executes.
+
+.Example
+Activate.ps1 -VenvDir C:\Users\MyUser\Common\.venv
+Activates the Python virtual environment located in the specified location.
+
+.Example
+Activate.ps1 -Prompt "MyPython"
+Activates the Python virtual environment that contains the Activate.ps1 script,
+and prefixes the current prompt with the specified string (surrounded in
+parentheses) while the virtual environment is active.
+
+.Notes
+On Windows, it may be required to enable this Activate.ps1 script by setting the
+execution policy for the user. You can do this by issuing the following PowerShell
+command:
+
+PS C:\> Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
+
+For more information on Execution Policies: 
+https://go.microsoft.com/fwlink/?LinkID=135170
+
+#>
+Param(
+    [Parameter(Mandatory = $false)]
+    [String]
+    $VenvDir,
+    [Parameter(Mandatory = $false)]
+    [String]
+    $Prompt
+)
+
+<# Function declarations --------------------------------------------------- #>
+
+<#
+.Synopsis
+Remove all shell session elements added by the Activate script, including the
+addition of the virtual environment's Python executable from the beginning of
+the PATH variable.
+
+.Parameter NonDestructive
+If present, do not remove this function from the global namespace for the
+session.
+
+#>
+function global:deactivate ([switch]$NonDestructive) {
+    # Revert to original values
+
+    # The prior prompt:
+    if (Test-Path -Path Function:_OLD_VIRTUAL_PROMPT) {
+        Copy-Item -Path Function:_OLD_VIRTUAL_PROMPT -Destination Function:prompt
+        Remove-Item -Path Function:_OLD_VIRTUAL_PROMPT
+    }
+
+    # The prior PYTHONHOME:
+    if (Test-Path -Path Env:_OLD_VIRTUAL_PYTHONHOME) {
+        Copy-Item -Path Env:_OLD_VIRTUAL_PYTHONHOME -Destination Env:PYTHONHOME
+        Remove-Item -Path Env:_OLD_VIRTUAL_PYTHONHOME
+    }
+
+    # The prior PATH:
+    if (Test-Path -Path Env:_OLD_VIRTUAL_PATH) {
+        Copy-Item -Path Env:_OLD_VIRTUAL_PATH -Destination Env:PATH
+        Remove-Item -Path Env:_OLD_VIRTUAL_PATH
+    }
+
+    # Just remove the VIRTUAL_ENV altogether:
+    if (Test-Path -Path Env:VIRTUAL_ENV) {
+        Remove-Item -Path env:VIRTUAL_ENV
+    }
+
+    # Just remove VIRTUAL_ENV_PROMPT altogether.
+    if (Test-Path -Path Env:VIRTUAL_ENV_PROMPT) {
+        Remove-Item -Path env:VIRTUAL_ENV_PROMPT
+    }
+
+    # Just remove the _PYTHON_VENV_PROMPT_PREFIX altogether:
+    if (Get-Variable -Name "_PYTHON_VENV_PROMPT_PREFIX" -ErrorAction SilentlyContinue) {
+        Remove-Variable -Name _PYTHON_VENV_PROMPT_PREFIX -Scope Global -Force
+    }
+
+    # Leave deactivate function in the global namespace if requested:
+    if (-not $NonDestructive) {
+        Remove-Item -Path function:deactivate
+    }
+}
+
+<#
+.Description
+Get-PyVenvConfig parses the values from the pyvenv.cfg file located in the
+given folder, and returns them in a map.
+
+For each line in the pyvenv.cfg file, if that line can be parsed into exactly
+two strings separated by `=` (with any amount of whitespace surrounding the =)
+then it is considered a `key = value` line. The left hand string is the key,
+the right hand is the value.
+
+If the value starts with a `'` or a `"` then the first and last character is
+stripped from the value before being captured.
+
+.Parameter ConfigDir
+Path to the directory that contains the `pyvenv.cfg` file.
+#>
+function Get-PyVenvConfig(
+    [String]
+    $ConfigDir
+) {
+    Write-Verbose "Given ConfigDir=$ConfigDir, obtain values in pyvenv.cfg"
+
+    # Ensure the file exists, and issue a warning if it doesn't (but still allow the function to continue).
+    $pyvenvConfigPath = Join-Path -Resolve -Path $ConfigDir -ChildPath 'pyvenv.cfg' -ErrorAction Continue
+
+    # An empty map will be returned if no config file is found.
+    $pyvenvConfig = @{ }
+
+    if ($pyvenvConfigPath) {
+
+        Write-Verbose "File exists, parse `key = value` lines"
+        $pyvenvConfigContent = Get-Content -Path $pyvenvConfigPath
+
+        $pyvenvConfigContent | ForEach-Object {
+            $keyval = $PSItem -split "\s*=\s*", 2
+            if ($keyval[0] -and $keyval[1]) {
+                $val = $keyval[1]
+
+                # Remove extraneous quotations around a string value.
+                if ("'""".Contains($val.Substring(0, 1))) {
+                    $val = $val.Substring(1, $val.Length - 2)
+                }
+
+                $pyvenvConfig[$keyval[0]] = $val
+                Write-Verbose "Adding Key: '$($keyval[0])'='$val'"
+            }
+        }
+    }
+    return $pyvenvConfig
+}
+
+
+<# Begin Activate script --------------------------------------------------- #>
+
+# Determine the containing directory of this script
+$VenvExecPath = Split-Path -Parent $MyInvocation.MyCommand.Definition
+$VenvExecDir = Get-Item -Path $VenvExecPath
+
+Write-Verbose "Activation script is located in path: '$VenvExecPath'"
+Write-Verbose "VenvExecDir Fullname: '$($VenvExecDir.FullName)"
+Write-Verbose "VenvExecDir Name: '$($VenvExecDir.Name)"
+
+# Set values required in priority: CmdLine, ConfigFile, Default
+# First, get the location of the virtual environment, it might not be
+# VenvExecDir if specified on the command line.
+if ($VenvDir) {
+    Write-Verbose "VenvDir given as parameter, using '$VenvDir' to determine values"
+}
+else {
+    Write-Verbose "VenvDir not given as a parameter, using parent directory name as VenvDir."
+    $VenvDir = $VenvExecDir.Parent.FullName.TrimEnd("\\/")
+    Write-Verbose "VenvDir=$VenvDir"
+}
+
+# Next, read the `pyvenv.cfg` file to determine any required value such
+# as `prompt`.
+$pyvenvCfg = Get-PyVenvConfig -ConfigDir $VenvDir
+
+# Next, set the prompt from the command line, or the config file, or
+# just use the name of the virtual environment folder.
+if ($Prompt) {
+    Write-Verbose "Prompt specified as argument, using '$Prompt'"
+}
+else {
+    Write-Verbose "Prompt not specified as argument to script, checking pyvenv.cfg value"
+    if ($pyvenvCfg -and $pyvenvCfg['prompt']) {
+        Write-Verbose "  Setting based on value in pyvenv.cfg='$($pyvenvCfg['prompt'])'"
+        $Prompt = $pyvenvCfg['prompt'];
+    }
+    else {
+        Write-Verbose "  Setting prompt based on parent's directory's name. (Is the directory name passed to venv module when creating the virtual environment)"
+        Write-Verbose "  Got leaf-name of $VenvDir='$(Split-Path -Path $venvDir -Leaf)'"
+        $Prompt = Split-Path -Path $venvDir -Leaf
+    }
+}
+
+Write-Verbose "Prompt = '$Prompt'"
+Write-Verbose "VenvDir='$VenvDir'"
+
+# Deactivate any currently active virtual environment, but leave the
+# deactivate function in place.
+deactivate -nondestructive
+
+# Now set the environment variable VIRTUAL_ENV, used by many tools to determine
+# that there is an activated venv.
+$env:VIRTUAL_ENV = $VenvDir
+
+if (-not $Env:VIRTUAL_ENV_DISABLE_PROMPT) {
+
+    Write-Verbose "Setting prompt to '$Prompt'"
+
+    # Set the prompt to include the env name
+    # Make sure _OLD_VIRTUAL_PROMPT is global
+    function global:_OLD_VIRTUAL_PROMPT { "" }
+    Copy-Item -Path function:prompt -Destination function:_OLD_VIRTUAL_PROMPT
+    New-Variable -Name _PYTHON_VENV_PROMPT_PREFIX -Description "Python virtual environment prompt prefix" -Scope Global -Option ReadOnly -Visibility Public -Value $Prompt
+
+    function global:prompt {
+        Write-Host -NoNewline -ForegroundColor Green "($_PYTHON_VENV_PROMPT_PREFIX) "
+        _OLD_VIRTUAL_PROMPT
+    }
+    $env:VIRTUAL_ENV_PROMPT = $Prompt
+}
+
+# Clear PYTHONHOME
+if (Test-Path -Path Env:PYTHONHOME) {
+    Copy-Item -Path Env:PYTHONHOME -Destination Env:_OLD_VIRTUAL_PYTHONHOME
+    Remove-Item -Path Env:PYTHONHOME
+}
+
+# Add the venv to the PATH
+Copy-Item -Path Env:PATH -Destination Env:_OLD_VIRTUAL_PATH
+$Env:PATH = "$VenvExecDir$([System.IO.Path]::PathSeparator)$Env:PATH"
+
+# SIG # Begin signature block
+# MIIpigYJKoZIhvcNAQcCoIIpezCCKXcCAQExDzANBglghkgBZQMEAgEFADB5Bgor
+# BgEEAYI3AgEEoGswaTA0BgorBgEEAYI3AgEeMCYCAwEAAAQQH8w7YFlLCE63JNLG
+# KX7zUQIBAAIBAAIBAAIBAAIBADAxMA0GCWCGSAFlAwQCAQUABCBnL745ElCYk8vk
+# dBtMuQhLeWJ3ZGfzKW4DHCYzAn+QB6CCDi8wggawMIIEmKADAgECAhAIrUCyYNKc
+# TJ9ezam9k67ZMA0GCSqGSIb3DQEBDAUAMGIxCzAJBgNVBAYTAlVTMRUwEwYDVQQK
+# EwxEaWdpQ2VydCBJbmMxGTAXBgNVBAsTEHd3dy5kaWdpY2VydC5jb20xITAfBgNV
+# BAMTGERpZ2lDZXJ0IFRydXN0ZWQgUm9vdCBHNDAeFw0yMTA0MjkwMDAwMDBaFw0z
+# NjA0MjgyMzU5NTlaMGkxCzAJBgNVBAYTAlVTMRcwFQYDVQQKEw5EaWdpQ2VydCwg
+# SW5jLjFBMD8GA1UEAxM4RGlnaUNlcnQgVHJ1c3RlZCBHNCBDb2RlIFNpZ25pbmcg
+# UlNBNDA5NiBTSEEzODQgMjAyMSBDQTEwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAw
+# ggIKAoICAQDVtC9C0CiteLdd1TlZG7GIQvUzjOs9gZdwxbvEhSYwn6SOaNhc9es0
+# JAfhS0/TeEP0F9ce2vnS1WcaUk8OoVf8iJnBkcyBAz5NcCRks43iCH00fUyAVxJr
+# Q5qZ8sU7H/Lvy0daE6ZMswEgJfMQ04uy+wjwiuCdCcBlp/qYgEk1hz1RGeiQIXhF
+# LqGfLOEYwhrMxe6TSXBCMo/7xuoc82VokaJNTIIRSFJo3hC9FFdd6BgTZcV/sk+F
+# LEikVoQ11vkunKoAFdE3/hoGlMJ8yOobMubKwvSnowMOdKWvObarYBLj6Na59zHh
+# 3K3kGKDYwSNHR7OhD26jq22YBoMbt2pnLdK9RBqSEIGPsDsJ18ebMlrC/2pgVItJ
+# wZPt4bRc4G/rJvmM1bL5OBDm6s6R9b7T+2+TYTRcvJNFKIM2KmYoX7BzzosmJQay
+# g9Rc9hUZTO1i4F4z8ujo7AqnsAMrkbI2eb73rQgedaZlzLvjSFDzd5Ea/ttQokbI
+# YViY9XwCFjyDKK05huzUtw1T0PhH5nUwjewwk3YUpltLXXRhTT8SkXbev1jLchAp
+# QfDVxW0mdmgRQRNYmtwmKwH0iU1Z23jPgUo+QEdfyYFQc4UQIyFZYIpkVMHMIRro
+# OBl8ZhzNeDhFMJlP/2NPTLuqDQhTQXxYPUez+rbsjDIJAsxsPAxWEQIDAQABo4IB
+# WTCCAVUwEgYDVR0TAQH/BAgwBgEB/wIBADAdBgNVHQ4EFgQUaDfg67Y7+F8Rhvv+
+# YXsIiGX0TkIwHwYDVR0jBBgwFoAU7NfjgtJxXWRM3y5nP+e6mK4cD08wDgYDVR0P
+# AQH/BAQDAgGGMBMGA1UdJQQMMAoGCCsGAQUFBwMDMHcGCCsGAQUFBwEBBGswaTAk
+# BggrBgEFBQcwAYYYaHR0cDovL29jc3AuZGlnaWNlcnQuY29tMEEGCCsGAQUFBzAC
+# hjVodHRwOi8vY2FjZXJ0cy5kaWdpY2VydC5jb20vRGlnaUNlcnRUcnVzdGVkUm9v
+# dEc0LmNydDBDBgNVHR8EPDA6MDigNqA0hjJodHRwOi8vY3JsMy5kaWdpY2VydC5j
+# b20vRGlnaUNlcnRUcnVzdGVkUm9vdEc0LmNybDAcBgNVHSAEFTATMAcGBWeBDAED
+# MAgGBmeBDAEEATANBgkqhkiG9w0BAQwFAAOCAgEAOiNEPY0Idu6PvDqZ01bgAhql
+# +Eg08yy25nRm95RysQDKr2wwJxMSnpBEn0v9nqN8JtU3vDpdSG2V1T9J9Ce7FoFF
+# UP2cvbaF4HZ+N3HLIvdaqpDP9ZNq4+sg0dVQeYiaiorBtr2hSBh+3NiAGhEZGM1h
+# mYFW9snjdufE5BtfQ/g+lP92OT2e1JnPSt0o618moZVYSNUa/tcnP/2Q0XaG3Ryw
+# YFzzDaju4ImhvTnhOE7abrs2nfvlIVNaw8rpavGiPttDuDPITzgUkpn13c5Ubdld
+# AhQfQDN8A+KVssIhdXNSy0bYxDQcoqVLjc1vdjcshT8azibpGL6QB7BDf5WIIIJw
+# 8MzK7/0pNVwfiThV9zeKiwmhywvpMRr/LhlcOXHhvpynCgbWJme3kuZOX956rEnP
+# LqR0kq3bPKSchh/jwVYbKyP/j7XqiHtwa+aguv06P0WmxOgWkVKLQcBIhEuWTatE
+# QOON8BUozu3xGFYHKi8QxAwIZDwzj64ojDzLj4gLDb879M4ee47vtevLt/B3E+bn
+# KD+sEq6lLyJsQfmCXBVmzGwOysWGw/YmMwwHS6DTBwJqakAwSEs0qFEgu60bhQji
+# WQ1tygVQK+pKHJ6l/aCnHwZ05/LWUpD9r4VIIflXO7ScA+2GRfS0YW6/aOImYIbq
+# yK+p/pQd52MbOoZWeE4wggd3MIIFX6ADAgECAhAHHxQbizANJfMU6yMM0NHdMA0G
+# CSqGSIb3DQEBCwUAMGkxCzAJBgNVBAYTAlVTMRcwFQYDVQQKEw5EaWdpQ2VydCwg
+# SW5jLjFBMD8GA1UEAxM4RGlnaUNlcnQgVHJ1c3RlZCBHNCBDb2RlIFNpZ25pbmcg
+# UlNBNDA5NiBTSEEzODQgMjAyMSBDQTEwHhcNMjIwMTE3MDAwMDAwWhcNMjUwMTE1
+# MjM1OTU5WjB8MQswCQYDVQQGEwJVUzEPMA0GA1UECBMGT3JlZ29uMRIwEAYDVQQH
+# EwlCZWF2ZXJ0b24xIzAhBgNVBAoTGlB5dGhvbiBTb2Z0d2FyZSBGb3VuZGF0aW9u
+# MSMwIQYDVQQDExpQeXRob24gU29mdHdhcmUgRm91bmRhdGlvbjCCAiIwDQYJKoZI
+# hvcNAQEBBQADggIPADCCAgoCggIBAKgc0BTT+iKbtK6f2mr9pNMUTcAJxKdsuOiS
+# YgDFfwhjQy89koM7uP+QV/gwx8MzEt3c9tLJvDccVWQ8H7mVsk/K+X+IufBLCgUi
+# 0GGAZUegEAeRlSXxxhYScr818ma8EvGIZdiSOhqjYc4KnfgfIS4RLtZSrDFG2tN1
+# 6yS8skFa3IHyvWdbD9PvZ4iYNAS4pjYDRjT/9uzPZ4Pan+53xZIcDgjiTwOh8VGu
+# ppxcia6a7xCyKoOAGjvCyQsj5223v1/Ig7Dp9mGI+nh1E3IwmyTIIuVHyK6Lqu35
+# 2diDY+iCMpk9ZanmSjmB+GMVs+H/gOiofjjtf6oz0ki3rb7sQ8fTnonIL9dyGTJ0
+# ZFYKeb6BLA66d2GALwxZhLe5WH4Np9HcyXHACkppsE6ynYjTOd7+jN1PRJahN1oE
+# RzTzEiV6nCO1M3U1HbPTGyq52IMFSBM2/07WTJSbOeXjvYR7aUxK9/ZkJiacl2iZ
+# I7IWe7JKhHohqKuceQNyOzxTakLcRkzynvIrk33R9YVqtB4L6wtFxhUjvDnQg16x
+# ot2KVPdfyPAWd81wtZADmrUtsZ9qG79x1hBdyOl4vUtVPECuyhCxaw+faVjumapP
+# Unwo8ygflJJ74J+BYxf6UuD7m8yzsfXWkdv52DjL74TxzuFTLHPyARWCSCAbzn3Z
+# Ily+qIqDAgMBAAGjggIGMIICAjAfBgNVHSMEGDAWgBRoN+Drtjv4XxGG+/5hewiI
+# ZfROQjAdBgNVHQ4EFgQUt/1Teh2XDuUj2WW3siYWJgkZHA8wDgYDVR0PAQH/BAQD
+# AgeAMBMGA1UdJQQMMAoGCCsGAQUFBwMDMIG1BgNVHR8Ega0wgaowU6BRoE+GTWh0
+# dHA6Ly9jcmwzLmRpZ2ljZXJ0LmNvbS9EaWdpQ2VydFRydXN0ZWRHNENvZGVTaWdu
+# aW5nUlNBNDA5NlNIQTM4NDIwMjFDQTEuY3JsMFOgUaBPhk1odHRwOi8vY3JsNC5k
+# aWdpY2VydC5jb20vRGlnaUNlcnRUcnVzdGVkRzRDb2RlU2lnbmluZ1JTQTQwOTZT
+# SEEzODQyMDIxQ0ExLmNybDA+BgNVHSAENzA1MDMGBmeBDAEEATApMCcGCCsGAQUF
+# BwIBFhtodHRwOi8vd3d3LmRpZ2ljZXJ0LmNvbS9DUFMwgZQGCCsGAQUFBwEBBIGH
+# MIGEMCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC5kaWdpY2VydC5jb20wXAYIKwYB
+# BQUHMAKGUGh0dHA6Ly9jYWNlcnRzLmRpZ2ljZXJ0LmNvbS9EaWdpQ2VydFRydXN0
+# ZWRHNENvZGVTaWduaW5nUlNBNDA5NlNIQTM4NDIwMjFDQTEuY3J0MAwGA1UdEwEB
+# /wQCMAAwDQYJKoZIhvcNAQELBQADggIBABxv4AeV/5ltkELHSC63fXAFYS5tadcW
+# TiNc2rskrNLrfH1Ns0vgSZFoQxYBFKI159E8oQQ1SKbTEubZ/B9kmHPhprHya08+
+# VVzxC88pOEvz68nA82oEM09584aILqYmj8Pj7h/kmZNzuEL7WiwFa/U1hX+XiWfL
+# IJQsAHBla0i7QRF2de8/VSF0XXFa2kBQ6aiTsiLyKPNbaNtbcucaUdn6vVUS5izW
+# OXM95BSkFSKdE45Oq3FForNJXjBvSCpwcP36WklaHL+aHu1upIhCTUkzTHMh8b86
+# WmjRUqbrnvdyR2ydI5l1OqcMBjkpPpIV6wcc+KY/RH2xvVuuoHjlUjwq2bHiNoX+
+# W1scCpnA8YTs2d50jDHUgwUo+ciwpffH0Riq132NFmrH3r67VaN3TuBxjI8SIZM5
+# 8WEDkbeoriDk3hxU8ZWV7b8AW6oyVBGfM06UgkfMb58h+tJPrFx8VI/WLq1dTqMf
+# ZOm5cuclMnUHs2uqrRNtnV8UfidPBL4ZHkTcClQbCoz0UbLhkiDvIS00Dn+BBcxw
+# /TKqVL4Oaz3bkMSsM46LciTeucHY9ExRVt3zy7i149sd+F4QozPqn7FrSVHXmem3
+# r7bjyHTxOgqxRCVa18Vtx7P/8bYSBeS+WHCKcliFCecspusCDSlnRUjZwyPdP0VH
+# xaZg2unjHY3rMYIasTCCGq0CAQEwfTBpMQswCQYDVQQGEwJVUzEXMBUGA1UEChMO
+# RGlnaUNlcnQsIEluYy4xQTA/BgNVBAMTOERpZ2lDZXJ0IFRydXN0ZWQgRzQgQ29k
+# ZSBTaWduaW5nIFJTQTQwOTYgU0hBMzg0IDIwMjEgQ0ExAhAHHxQbizANJfMU6yMM
+# 0NHdMA0GCWCGSAFlAwQCAQUAoIHEMBkGCSqGSIb3DQEJAzEMBgorBgEEAYI3AgEE
+# MBwGCisGAQQBgjcCAQsxDjAMBgorBgEEAYI3AgEVMC8GCSqGSIb3DQEJBDEiBCBn
+# AZ6P7YvTwq0fbF62o7E75R0LxsW5OtyYiFESQckLhjBYBgorBgEEAYI3AgEMMUow
+# SKBGgEQAQgB1AGkAbAB0ADoAIABSAGUAbABlAGEAcwBlAF8AdgAzAC4AMQAxAC4A
+# MABfADIAMAAyADIAMQAwADIANAAuADAAMTANBgkqhkiG9w0BAQEFAASCAgAu2uG5
+# zPAAKY4N8BVMzMPRSoTqq2HAcX+oqvto72DGzHLKlfAuuyf59saf7TQZQ04Ao1ni
+# EvpzZ8C4Wv7yu8RyPwJQThIuFQuhMgB+Zscl+YDnAo5+GFTBpevgcG2n2ClHAPuT
+# 7aXe3+5wChDpMqyusrBYws+8R6tg8rKFyRhQndxIJkIMlZhoh1qI3tRypW6e2r5l
+# Uf4pPDkNBBySzjNOupTyv1/d2Y31Ise8xLrLbuMLYxtir/5A0z6GlUueoecpe9TS
+# uEqz2bI+HZbGC6xK2BU4vW8s7qefVTmPFAf3JiCjZZ46qFAg9jnWCRzAA/3jOtu6
+# V345rFhCRJxPKz4M96B5mUCnMU0BB4cHJFKZfezd5phtExi1///WcnKNkpNTto+d
+# etpWbJ87DibBro3ZhDPh9FpHW2jxy2IQBZo02Udbwfd7aoKhRf7MCLqZUIziPjRS
+# FcA1hyOzYk4XfHK1qW3Wpflduz86UGDbURWP3XhXQNaSScJGOhVylZbiBWcjFKlD
+# E/sl+bDyafUy0jLur6/Vl4H2xCgXbJlEazr04QfizW9N9x2G6sDkdbQd4k3kSEJt
+# UOufbrdjDY1MRd/NlnjVGY+zslEDN9QJQuKq00SJagicDJ+vIzg6J7YjnRfDGLAi
+# RJb9rXxuQyEoSTdtxQgnPNkb6vCNQz80bjHmoqGCFz4wghc6BgorBgEEAYI3AwMB
+# MYIXKjCCFyYGCSqGSIb3DQEHAqCCFxcwghcTAgEDMQ8wDQYJYIZIAWUDBAIBBQAw
+# eAYLKoZIhvcNAQkQAQSgaQRnMGUCAQEGCWCGSAGG/WwHATAxMA0GCWCGSAFlAwQC
+# AQUABCCJnxONky4RAgM+R4O2F+soqJ9cjrZDLL3JqXN+msPWngIRAPgphjs42egI
+# Fn/RXf6+TgkYDzIwMjIxMDI0MTgzMzM4WqCCEwcwggbAMIIEqKADAgECAhAMTWly
+# S5T6PCpKPSkHgD1aMA0GCSqGSIb3DQEBCwUAMGMxCzAJBgNVBAYTAlVTMRcwFQYD
+# VQQKEw5EaWdpQ2VydCwgSW5jLjE7MDkGA1UEAxMyRGlnaUNlcnQgVHJ1c3RlZCBH
+# NCBSU0E0MDk2IFNIQTI1NiBUaW1lU3RhbXBpbmcgQ0EwHhcNMjIwOTIxMDAwMDAw
+# WhcNMzMxMTIxMjM1OTU5WjBGMQswCQYDVQQGEwJVUzERMA8GA1UEChMIRGlnaUNl
+# cnQxJDAiBgNVBAMTG0RpZ2lDZXJ0IFRpbWVzdGFtcCAyMDIyIC0gMjCCAiIwDQYJ
+# KoZIhvcNAQEBBQADggIPADCCAgoCggIBAM/spSY6xqnya7uNwQ2a26HoFIV0Mxom
+# rNAcVR4eNm28klUMYfSdCXc9FZYIL2tkpP0GgxbXkZI4HDEClvtysZc6Va8z7GGK
+# 6aYo25BjXL2JU+A6LYyHQq4mpOS7eHi5ehbhVsbAumRTuyoW51BIu4hpDIjG8b7g
+# L307scpTjUCDHufLckkoHkyAHoVW54Xt8mG8qjoHffarbuVm3eJc9S/tjdRNlYRo
+# 44DLannR0hCRRinrPibytIzNTLlmyLuqUDgN5YyUXRlav/V7QG5vFqianJVHhoV5
+# PgxeZowaCiS+nKrSnLb3T254xCg/oxwPUAY3ugjZNaa1Htp4WB056PhMkRCWfk3h
+# 3cKtpX74LRsf7CtGGKMZ9jn39cFPcS6JAxGiS7uYv/pP5Hs27wZE5FX/NurlfDHn
+# 88JSxOYWe1p+pSVz28BqmSEtY+VZ9U0vkB8nt9KrFOU4ZodRCGv7U0M50GT6Vs/g
+# 9ArmFG1keLuY/ZTDcyHzL8IuINeBrNPxB9ThvdldS24xlCmL5kGkZZTAWOXlLimQ
+# prdhZPrZIGwYUWC6poEPCSVT8b876asHDmoHOWIZydaFfxPZjXnPYsXs4Xu5zGcT
+# B5rBeO3GiMiwbjJ5xwtZg43G7vUsfHuOy2SJ8bHEuOdTXl9V0n0ZKVkDTvpd6kVz
+# HIR+187i1Dp3AgMBAAGjggGLMIIBhzAOBgNVHQ8BAf8EBAMCB4AwDAYDVR0TAQH/
+# BAIwADAWBgNVHSUBAf8EDDAKBggrBgEFBQcDCDAgBgNVHSAEGTAXMAgGBmeBDAEE
+# AjALBglghkgBhv1sBwEwHwYDVR0jBBgwFoAUuhbZbU2FL3MpdpovdYxqII+eyG8w
+# HQYDVR0OBBYEFGKK3tBh/I8xFO2XC809KpQU31KcMFoGA1UdHwRTMFEwT6BNoEuG
+# SWh0dHA6Ly9jcmwzLmRpZ2ljZXJ0LmNvbS9EaWdpQ2VydFRydXN0ZWRHNFJTQTQw
+# OTZTSEEyNTZUaW1lU3RhbXBpbmdDQS5jcmwwgZAGCCsGAQUFBwEBBIGDMIGAMCQG
+# CCsGAQUFBzABhhhodHRwOi8vb2NzcC5kaWdpY2VydC5jb20wWAYIKwYBBQUHMAKG
+# TGh0dHA6Ly9jYWNlcnRzLmRpZ2ljZXJ0LmNvbS9EaWdpQ2VydFRydXN0ZWRHNFJT
+# QTQwOTZTSEEyNTZUaW1lU3RhbXBpbmdDQS5jcnQwDQYJKoZIhvcNAQELBQADggIB
+# AFWqKhrzRvN4Vzcw/HXjT9aFI/H8+ZU5myXm93KKmMN31GT8Ffs2wklRLHiIY1UJ
+# RjkA/GnUypsp+6M/wMkAmxMdsJiJ3HjyzXyFzVOdr2LiYWajFCpFh0qYQitQ/Bu1
+# nggwCfrkLdcJiXn5CeaIzn0buGqim8FTYAnoo7id160fHLjsmEHw9g6A++T/350Q
+# p+sAul9Kjxo6UrTqvwlJFTU2WZoPVNKyG39+XgmtdlSKdG3K0gVnK3br/5iyJpU4
+# GYhEFOUKWaJr5yI+RCHSPxzAm+18SLLYkgyRTzxmlK9dAlPrnuKe5NMfhgFknADC
+# 6Vp0dQ094XmIvxwBl8kZI4DXNlpflhaxYwzGRkA7zl011Fk+Q5oYrsPJy8P7mxNf
+# arXH4PMFw1nfJ2Ir3kHJU7n/NBBn9iYymHv+XEKUgZSCnawKi8ZLFUrTmJBFYDOA
+# 4CPe+AOk9kVH5c64A0JH6EE2cXet/aLol3ROLtoeHYxayB6a1cLwxiKoT5u92Bya
+# UcQvmvZfpyeXupYuhVfAYOd4Vn9q78KVmksRAsiCnMkaBXy6cbVOepls9Oie1FqY
+# yJ+/jbsYXEP10Cro4mLueATbvdH7WwqocH7wl4R44wgDXUcsY6glOJcB0j862uXl
+# 9uab3H4szP8XTE0AotjWAQ64i+7m4HJViSwnGWH2dwGMMIIGrjCCBJagAwIBAgIQ
+# BzY3tyRUfNhHrP0oZipeWzANBgkqhkiG9w0BAQsFADBiMQswCQYDVQQGEwJVUzEV
+# MBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3d3cuZGlnaWNlcnQuY29t
+# MSEwHwYDVQQDExhEaWdpQ2VydCBUcnVzdGVkIFJvb3QgRzQwHhcNMjIwMzIzMDAw
+# MDAwWhcNMzcwMzIyMjM1OTU5WjBjMQswCQYDVQQGEwJVUzEXMBUGA1UEChMORGln
+# aUNlcnQsIEluYy4xOzA5BgNVBAMTMkRpZ2lDZXJ0IFRydXN0ZWQgRzQgUlNBNDA5
+# NiBTSEEyNTYgVGltZVN0YW1waW5nIENBMIICIjANBgkqhkiG9w0BAQEFAAOCAg8A
+# MIICCgKCAgEAxoY1BkmzwT1ySVFVxyUDxPKRN6mXUaHW0oPRnkyibaCwzIP5WvYR
+# oUQVQl+kiPNo+n3znIkLf50fng8zH1ATCyZzlm34V6gCff1DtITaEfFzsbPuK4CE
+# iiIY3+vaPcQXf6sZKz5C3GeO6lE98NZW1OcoLevTsbV15x8GZY2UKdPZ7Gnf2ZCH
+# RgB720RBidx8ald68Dd5n12sy+iEZLRS8nZH92GDGd1ftFQLIWhuNyG7QKxfst5K
+# fc71ORJn7w6lY2zkpsUdzTYNXNXmG6jBZHRAp8ByxbpOH7G1WE15/tePc5OsLDni
+# pUjW8LAxE6lXKZYnLvWHpo9OdhVVJnCYJn+gGkcgQ+NDY4B7dW4nJZCYOjgRs/b2
+# nuY7W+yB3iIU2YIqx5K/oN7jPqJz+ucfWmyU8lKVEStYdEAoq3NDzt9KoRxrOMUp
+# 88qqlnNCaJ+2RrOdOqPVA+C/8KI8ykLcGEh/FDTP0kyr75s9/g64ZCr6dSgkQe1C
+# vwWcZklSUPRR8zZJTYsg0ixXNXkrqPNFYLwjjVj33GHek/45wPmyMKVM1+mYSlg+
+# 0wOI/rOP015LdhJRk8mMDDtbiiKowSYI+RQQEgN9XyO7ZONj4KbhPvbCdLI/Hgl2
+# 7KtdRnXiYKNYCQEoAA6EVO7O6V3IXjASvUaetdN2udIOa5kM0jO0zbECAwEAAaOC
+# AV0wggFZMBIGA1UdEwEB/wQIMAYBAf8CAQAwHQYDVR0OBBYEFLoW2W1NhS9zKXaa
+# L3WMaiCPnshvMB8GA1UdIwQYMBaAFOzX44LScV1kTN8uZz/nupiuHA9PMA4GA1Ud
+# DwEB/wQEAwIBhjATBgNVHSUEDDAKBggrBgEFBQcDCDB3BggrBgEFBQcBAQRrMGkw
+# JAYIKwYBBQUHMAGGGGh0dHA6Ly9vY3NwLmRpZ2ljZXJ0LmNvbTBBBggrBgEFBQcw
+# AoY1aHR0cDovL2NhY2VydHMuZGlnaWNlcnQuY29tL0RpZ2lDZXJ0VHJ1c3RlZFJv
+# b3RHNC5jcnQwQwYDVR0fBDwwOjA4oDagNIYyaHR0cDovL2NybDMuZGlnaWNlcnQu
+# Y29tL0RpZ2lDZXJ0VHJ1c3RlZFJvb3RHNC5jcmwwIAYDVR0gBBkwFzAIBgZngQwB
+# BAIwCwYJYIZIAYb9bAcBMA0GCSqGSIb3DQEBCwUAA4ICAQB9WY7Ak7ZvmKlEIgF+
+# ZtbYIULhsBguEE0TzzBTzr8Y+8dQXeJLKftwig2qKWn8acHPHQfpPmDI2AvlXFvX
+# bYf6hCAlNDFnzbYSlm/EUExiHQwIgqgWvalWzxVzjQEiJc6VaT9Hd/tydBTX/6tP
+# iix6q4XNQ1/tYLaqT5Fmniye4Iqs5f2MvGQmh2ySvZ180HAKfO+ovHVPulr3qRCy
+# Xen/KFSJ8NWKcXZl2szwcqMj+sAngkSumScbqyQeJsG33irr9p6xeZmBo1aGqwpF
+# yd/EjaDnmPv7pp1yr8THwcFqcdnGE4AJxLafzYeHJLtPo0m5d2aR8XKc6UsCUqc3
+# fpNTrDsdCEkPlM05et3/JWOZJyw9P2un8WbDQc1PtkCbISFA0LcTJM3cHXg65J6t
+# 5TRxktcma+Q4c6umAU+9Pzt4rUyt+8SVe+0KXzM5h0F4ejjpnOHdI/0dKNPH+ejx
+# mF/7K9h+8kaddSweJywm228Vex4Ziza4k9Tm8heZWcpw8De/mADfIBZPJ/tgZxah
+# ZrrdVcA6KYawmKAr7ZVBtzrVFZgxtGIJDwq9gdkT/r+k0fNX2bwE+oLeMt8EifAA
+# zV3C+dAjfwAL5HYCJtnwZXZCpimHCUcr5n8apIUP/JiW9lVUKx+A+sDyDivl1vup
+# L0QVSucTDh3bNzgaoSv27dZ8/DCCBY0wggR1oAMCAQICEA6bGI750C3n79tQ4ghA
+# GFowDQYJKoZIhvcNAQEMBQAwZTELMAkGA1UEBhMCVVMxFTATBgNVBAoTDERpZ2lD
+# ZXJ0IEluYzEZMBcGA1UECxMQd3d3LmRpZ2ljZXJ0LmNvbTEkMCIGA1UEAxMbRGln
+# aUNlcnQgQXNzdXJlZCBJRCBSb290IENBMB4XDTIyMDgwMTAwMDAwMFoXDTMxMTEw
+# OTIzNTk1OVowYjELMAkGA1UEBhMCVVMxFTATBgNVBAoTDERpZ2lDZXJ0IEluYzEZ
+# MBcGA1UECxMQd3d3LmRpZ2ljZXJ0LmNvbTEhMB8GA1UEAxMYRGlnaUNlcnQgVHJ1
+# c3RlZCBSb290IEc0MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAv+aQ
+# c2jeu+RdSjwwIjBpM+zCpyUuySE98orYWcLhKac9WKt2ms2uexuEDcQwH/MbpDgW
+# 61bGl20dq7J58soR0uRf1gU8Ug9SH8aeFaV+vp+pVxZZVXKvaJNwwrK6dZlqczKU
+# 0RBEEC7fgvMHhOZ0O21x4i0MG+4g1ckgHWMpLc7sXk7Ik/ghYZs06wXGXuxbGrzr
+# yc/NrDRAX7F6Zu53yEioZldXn1RYjgwrt0+nMNlW7sp7XeOtyU9e5TXnMcvak17c
+# jo+A2raRmECQecN4x7axxLVqGDgDEI3Y1DekLgV9iPWCPhCRcKtVgkEy19sEcypu
+# kQF8IUzUvK4bA3VdeGbZOjFEmjNAvwjXWkmkwuapoGfdpCe8oU85tRFYF/ckXEaP
+# ZPfBaYh2mHY9WV1CdoeJl2l6SPDgohIbZpp0yt5LHucOY67m1O+SkjqePdwA5EUl
+# ibaaRBkrfsCUtNJhbesz2cXfSwQAzH0clcOP9yGyshG3u3/y1YxwLEFgqrFjGESV
+# GnZifvaAsPvoZKYz0YkH4b235kOkGLimdwHhD5QMIR2yVCkliWzlDlJRR3S+Jqy2
+# QXXeeqxfjT/JvNNBERJb5RBQ6zHFynIWIgnffEx1P2PsIV/EIFFrb7GrhotPwtZF
+# X50g/KEexcCPorF+CiaZ9eRpL5gdLfXZqbId5RsCAwEAAaOCATowggE2MA8GA1Ud
+# EwEB/wQFMAMBAf8wHQYDVR0OBBYEFOzX44LScV1kTN8uZz/nupiuHA9PMB8GA1Ud
+# IwQYMBaAFEXroq/0ksuCMS1Ri6enIZ3zbcgPMA4GA1UdDwEB/wQEAwIBhjB5Bggr
+# BgEFBQcBAQRtMGswJAYIKwYBBQUHMAGGGGh0dHA6Ly9vY3NwLmRpZ2ljZXJ0LmNv
+# bTBDBggrBgEFBQcwAoY3aHR0cDovL2NhY2VydHMuZGlnaWNlcnQuY29tL0RpZ2lD
+# ZXJ0QXNzdXJlZElEUm9vdENBLmNydDBFBgNVHR8EPjA8MDqgOKA2hjRodHRwOi8v
+# Y3JsMy5kaWdpY2VydC5jb20vRGlnaUNlcnRBc3N1cmVkSURSb290Q0EuY3JsMBEG
+# A1UdIAQKMAgwBgYEVR0gADANBgkqhkiG9w0BAQwFAAOCAQEAcKC/Q1xV5zhfoKN0
+# Gz22Ftf3v1cHvZqsoYcs7IVeqRq7IviHGmlUIu2kiHdtvRoU9BNKei8ttzjv9P+A
+# ufih9/Jy3iS8UgPITtAq3votVs/59PesMHqai7Je1M/RQ0SbQyHrlnKhSLSZy51P
+# pwYDE3cnRNTnf+hZqPC/Lwum6fI0POz3A8eHqNJMQBk1RmppVLC4oVaO7KTVPeix
+# 3P0c2PR3WlxUjG/voVA9/HYJaISfb8rbII01YBwCA8sgsKxYoA5AY8WYIsGyWfVV
+# a88nq2x2zm8jLfR+cWojayL/ErhULSd+2DrZ8LaHlv1b0VysGMNNn3O3AamfV6pe
+# KOK5lDGCA3YwggNyAgEBMHcwYzELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDkRpZ2lD
+# ZXJ0LCBJbmMuMTswOQYDVQQDEzJEaWdpQ2VydCBUcnVzdGVkIEc0IFJTQTQwOTYg
+# U0hBMjU2IFRpbWVTdGFtcGluZyBDQQIQDE1pckuU+jwqSj0pB4A9WjANBglghkgB
+# ZQMEAgEFAKCB0TAaBgkqhkiG9w0BCQMxDQYLKoZIhvcNAQkQAQQwHAYJKoZIhvcN
+# AQkFMQ8XDTIyMTAyNDE4MzMzOFowKwYLKoZIhvcNAQkQAgwxHDAaMBgwFgQU84ci
+# TYYzgpI1qZS8vY+W6f4cfHMwLwYJKoZIhvcNAQkEMSIEILoHmtH34MMtLSezOEUS
+# 8z6MwtqV/PFPq/sNVq5aJnKMMDcGCyqGSIb3DQEJEAIvMSgwJjAkMCIEIMf04b4y
+# KIkgq+ImOr4axPxP5ngcLWTQTIB1V6Ajtbb6MA0GCSqGSIb3DQEBAQUABIICAEtb
+# WINxaVTjBdclvuFwJT/uHWvlOdcKzc1o+toRkFb1OA7shEdXFvjNU549TilTs8qQ
+# bly8CbFcz3JzLVLrNKO7lr4GXd2iyJV5sv/XU4ED866fznOnFWtZJvxKGOdqN0W7
+# 01pw7mIJ8+2aRqpow1ppPzju7VagRQ8fKmtj9Sg5N8Ja3+AehpjwM/PYzctan/1m
+# ytIK/HCw5k/MeGmPVBs/fqbN0DT4KGrJ7YMySdYZMs0U9V7Ak7PelZLgw8BkNi1Y
+# Rb9i+7/t9AaBlVYMy/6+gzdsnarnlSzV8/6Est8w4Ie7sBxx3Tpsokopb+oPF///
+# 2cA3jMNToO9YfsqvgpTEkWwjWanC2cd26K8ikw0uu0klmaxNvYpP459/QU3JMyFj
+# I4ReTxVXLZrQlzCDUUdLmLSeV1AugCOYOHM2RAv4r+3qxk0jBCfA8RRK+prLNjXE
+# af1QEbeRRNr0418MtnBdIzxHnW8yffWfHmtDNJoyPqggkRU3Mb8Myu8QPD3ZiCPj
+# F+HsKUntyCV64hr9BNLmkpbw+kUvGtC0/7sZF9Gyp/DKnnbQu8vSR+CaZQqVQxJo
+# UeI7m44utNTSSZCJ9JV7bnniwqztrP/r2PTAxkUywoCzif6R863qJ/uQA0QQjq8t
+# +aR822g6YVyJsLYQKbpEgshG2QwzGHun5HkvawJ8
+# SIG # End signature block

llmrag/Scripts/activate

@@ -0,0 +1,69 @@
+# This file must be used with "source bin/activate" *from bash*
+# you cannot run it directly
+
+deactivate () {
+    # reset old environment variables
+    if [ -n "${_OLD_VIRTUAL_PATH:-}" ] ; then
+        PATH="${_OLD_VIRTUAL_PATH:-}"
+        export PATH
+        unset _OLD_VIRTUAL_PATH
+    fi
+    if [ -n "${_OLD_VIRTUAL_PYTHONHOME:-}" ] ; then
+        PYTHONHOME="${_OLD_VIRTUAL_PYTHONHOME:-}"
+        export PYTHONHOME
+        unset _OLD_VIRTUAL_PYTHONHOME
+    fi
+
+    # This should detect bash and zsh, which have a hash command that must
+    # be called to get it to forget past commands.  Without forgetting
+    # past commands the $PATH changes we made may not be respected
+    if [ -n "${BASH:-}" -o -n "${ZSH_VERSION:-}" ] ; then
+        hash -r 2> /dev/null
+    fi
+
+    if [ -n "${_OLD_VIRTUAL_PS1:-}" ] ; then
+        PS1="${_OLD_VIRTUAL_PS1:-}"
+        export PS1
+        unset _OLD_VIRTUAL_PS1
+    fi
+
+    unset VIRTUAL_ENV
+    unset VIRTUAL_ENV_PROMPT
+    if [ ! "${1:-}" = "nondestructive" ] ; then
+    # Self destruct!
+        unset -f deactivate
+    fi
+}
+
+# unset irrelevant variables
+deactivate nondestructive
+
+VIRTUAL_ENV="D:\llm-chatbot\rag-youtube-assistant\llmrag"
+export VIRTUAL_ENV
+
+_OLD_VIRTUAL_PATH="$PATH"
+PATH="$VIRTUAL_ENV/Scripts:$PATH"
+export PATH
+
+# unset PYTHONHOME if set
+# this will fail if PYTHONHOME is set to the empty string (which is bad anyway)
+# could use `if (set -u; : $PYTHONHOME) ;` in bash
+if [ -n "${PYTHONHOME:-}" ] ; then
+    _OLD_VIRTUAL_PYTHONHOME="${PYTHONHOME:-}"
+    unset PYTHONHOME
+fi
+
+if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT:-}" ] ; then
+    _OLD_VIRTUAL_PS1="${PS1:-}"
+    PS1="(llmrag) ${PS1:-}"
+    export PS1
+    VIRTUAL_ENV_PROMPT="(llmrag) "
+    export VIRTUAL_ENV_PROMPT
+fi
+
+# This should detect bash and zsh, which have a hash command that must
+# be called to get it to forget past commands.  Without forgetting
+# past commands the $PATH changes we made may not be respected
+if [ -n "${BASH:-}" -o -n "${ZSH_VERSION:-}" ] ; then
+    hash -r 2> /dev/null
+fi

llmrag/Scripts/activate.bat

@@ -0,0 +1,34 @@
+@echo off
+
+rem This file is UTF-8 encoded, so we need to update the current code page while executing it
+for /f "tokens=2 delims=:." %%a in ('"%SystemRoot%\System32\chcp.com"') do (
+    set _OLD_CODEPAGE=%%a
+)
+if defined _OLD_CODEPAGE (
+    "%SystemRoot%\System32\chcp.com" 65001 > nul
+)
+
+set VIRTUAL_ENV=D:\llm-chatbot\rag-youtube-assistant\llmrag
+
+if not defined PROMPT set PROMPT=$P$G
+
+if defined _OLD_VIRTUAL_PROMPT set PROMPT=%_OLD_VIRTUAL_PROMPT%
+if defined _OLD_VIRTUAL_PYTHONHOME set PYTHONHOME=%_OLD_VIRTUAL_PYTHONHOME%
+
+set _OLD_VIRTUAL_PROMPT=%PROMPT%
+set PROMPT=(llmrag) %PROMPT%
+
+if defined PYTHONHOME set _OLD_VIRTUAL_PYTHONHOME=%PYTHONHOME%
+set PYTHONHOME=
+
+if defined _OLD_VIRTUAL_PATH set PATH=%_OLD_VIRTUAL_PATH%
+if not defined _OLD_VIRTUAL_PATH set _OLD_VIRTUAL_PATH=%PATH%
+
+set PATH=%VIRTUAL_ENV%\Scripts;%PATH%
+set VIRTUAL_ENV_PROMPT=(llmrag) 
+
+:END
+if defined _OLD_CODEPAGE (
+    "%SystemRoot%\System32\chcp.com" %_OLD_CODEPAGE% > nul
+    set _OLD_CODEPAGE=
+)

llmrag/Scripts/deactivate.bat

@@ -0,0 +1,22 @@
+@echo off
+
+if defined _OLD_VIRTUAL_PROMPT (
+    set "PROMPT=%_OLD_VIRTUAL_PROMPT%"
+)
+set _OLD_VIRTUAL_PROMPT=
+
+if defined _OLD_VIRTUAL_PYTHONHOME (
+    set "PYTHONHOME=%_OLD_VIRTUAL_PYTHONHOME%"
+    set _OLD_VIRTUAL_PYTHONHOME=
+)
+
+if defined _OLD_VIRTUAL_PATH (
+    set "PATH=%_OLD_VIRTUAL_PATH%"
+)
+
+set _OLD_VIRTUAL_PATH=
+
+set VIRTUAL_ENV=
+set VIRTUAL_ENV_PROMPT=
+
+:END

llmrag/pyvenv.cfg

@@ -0,0 +1,5 @@
+home = C:\Python311
+include-system-site-packages = false
+version = 3.11.0
+executable = C:\Python311\python.exe
+command = C:\Python311\python.exe -m venv D:\llm-chatbot\rag-youtube-assistant\llmrag

requirements.txt

@@ -0,0 +1,14 @@
+streamlit
+youtube_transcript_api
+sentence-transformers
+google-api-python-client
+google-auth-httplib2
+google-auth-oauthlib
+pandas
+numpy
+scikit-learn
+elasticsearch
+ollama
+requests
+matplotlib
+tqdm

run-docker-compose-windows.ps1

@@ -0,0 +1,23 @@
+# Define the path to the .env file
+$envPath = ".\.env"
+
+# Check if the .env file exists
+if (Test-Path $envPath) {
+    # Read the .env file
+    $envContent = Get-Content $envPath
+
+    # Parse the environment variables
+    foreach ($line in $envContent) {
+        if ($line -match '^([^=]+)=(.*)$') {
+            $name = $matches[1]
+            $value = $matches[2]
+            [Environment]::SetEnvironmentVariable($name, $value, "Process")
+        }
+    }
+
+    # Run docker-compose
+    docker-compose up --build
+}
+else {
+    Write-Error "The .env file was not found at $envPath"
+}

run-docker-compose.sh

@@ -0,0 +1,19 @@
+#!/bin/bash
+
+# Start Ollama
+ollama serve &
+
+# Wait for Ollama to start
+sleep 10
+
+# Run Phi model to ensure it's loaded
+ollama run phi "hello" &
+
+# Generate ground truth
+python generate_ground_truth.py
+
+# Run RAG evaluation
+python rag_evaluation.py
+
+# Start the Streamlit app
+streamlit run main.py