Added logging for debugging

flashcards/flashcards_03fb423e-2087-4598-9ebb-b99a20db0b93.csv

@@ -1,11 +0,0 @@
-Front,Back
-What is the first step in the retriever creation process?,The first step is loading a fine-tuned embedding model from the hub using the HuggingFaceEmbeddings class. The model used is named 'JulsdL/e2erag-arctic-m' and it is configured to run on a CUDA device.
-Which class is used to load the embedding model in the retriever creation process?,The HuggingFaceEmbeddings class is used to load the embedding model.
-What is the name of the embedding model used in the retriever creation process?,The embedding model used is named 'JulsdL/e2erag-arctic-m'.
-On which device is the embedding model 'JulsdL/e2erag-arctic-m' configured to run?,The embedding model is configured to run on a CUDA device.
-What is the purpose of setting up a VectorStore in the retriever creation process?,The VectorStore is set up to power the dense vector search and is populated with documents that have been split into chunks and embedded using the embedding model.
-Which tool is used to set up the VectorStore in the retriever creation process?,Meta's FAISS (Facebook AI Similarity Search) is used to set up the VectorStore.
-How are documents prepared for the VectorStore in the retriever creation process?,Documents are split into chunks and embedded using the previously loaded embedding model before being added to the VectorStore.
-What is the final step in the retriever creation process?,The final step is converting the VectorStore into a retriever that can fetch relevant documents or chunks based on query embeddings.
-"In the context of retriever creation, what is the purpose of converting the VectorStore to a retriever?",Converting the VectorStore to a retriever allows it to be used for fetching relevant documents or chunks based on the query embeddings.
-How does a retriever enhance the model's ability in a Retrieval-Augmented Generation (RAG) setup?,"A retriever enhances the model's ability by providing contextually relevant responses based on the input queries, improving the quality and relevance of the generated content."

flashcards/flashcards_30a9e627-3d47-45fc-81db-a991269edb24.csv

@@ -1,11 +0,0 @@
-Front,Back
-What command is used to clone a repository from GitHub?,!git clone https://github.com/arcee-ai/DALM
-How do you install a package using pip and upgrade it if necessary?,!pip install --upgrade -q -e .
-"Which command is used to install the latest versions of langchain, langchain-core, langchain-community, and sentence_transformers?",!pip install -qU langchain langchain-core langchain-community sentence_transformers
-How do you install the pymupdf and faiss-cpu libraries using pip?,!pip install -qU pymupdf faiss-cpu
-What is the import statement for pandas?,import pandas as pd
-How do you import HuggingFaceEmbeddings from the langchain_community library?,from langchain_community.embeddings import HuggingFaceEmbeddings
-What is the import statement for FAISS from the langchain_community library?,from langchain_community.vectorstores import FAISS
-Which import statement is used for SimpleDirectoryReader from llama_index.core?,from llama_index.core import SimpleDirectoryReader
-How do you import SimpleNodeParser from the llama_index.core.node_parser module?,from llama_index.core.node_parser import SimpleNodeParser
-What is the import statement for MetadataMode from llama_index.core.schema?,from llama_index.core.schema import MetadataMode

flashcards/flashcards_37136597-b97c-4889-9dc1-c3b5e10084c1.csv

@@ -1,11 +0,0 @@
-Front,Back
-What is the first step in the retriever creation process?,The first step in the retriever creation process is loading the embedding model.
-Which model is used as the embedding model in the retriever creation process?,The embedding model used is 'JulsdL/e2erag-arctic-m'.
-Which class and module are used to load the embedding model?,The embedding model is loaded using the HuggingFaceEmbeddings class from the langchain_community.embeddings module.
-On which device is the embedding model configured to run?,The embedding model is configured to run on a CUDA device.
-What is the purpose of the VectorStore in the retriever creation process?,The VectorStore is created to power dense vector searches.
-Which technology is used to set up the VectorStore?,Meta's FAISS (Facebook AI Similarity Search) is used to set up the VectorStore.
-How are the documents prepared for creating the VectorStore?,The documents are split into chunks before creating the VectorStore.
-Which method is called to create the VectorStore from the documents and embedding model?,The method FAISS.from_documents is called to create the VectorStore.
-How is the VectorStore converted into a retriever?,The VectorStore is converted into a retriever by invoking the as_retriever() method on the vector_store object.
-What are the benefits of the retriever created through this process?,"The retriever leverages the power of dense embeddings and efficient search capabilities provided by FAISS, making it effective for retrieval-augmented tasks."

flashcards/flashcards_4dbaa0d1-6363-4584-9691-824540220571.csv

@@ -1,11 +0,0 @@
-Front,Back
-What is the first step in creating a retriever for a document?,The first step is loading the documents using a loader like `PyMuPDFLoader` to load documents from a PDF file.
-Which function is used to load documents from a PDF file?,The function `PyMuPDFLoader` is used to load documents from a PDF file.
-How are documents split into chunks for processing?,"Documents are split into chunks using the `RecursiveCharacterTextSplitter`, which divides the documents based on predefined rules like token limits and split characters."
-What is the purpose of chunking documents?,Chunking documents into smaller parts helps in managing and processing them more efficiently for embedding and retrieval.
-Which library is used for loading a pre-trained embedding model?,The `HuggingFaceEmbeddings` library is used for loading a pre-trained embedding model.
-How do you specify the device for running the embedding model?,"You specify the device (e.g., 'cuda') in the `model_kwargs` parameter when loading the embedding model."
-What is Meta's FAISS used for in the retriever creation process?,Meta's FAISS is used to power dense vector search by creating a vector store from the document chunks and embedding model.
-How do you convert a vector store into a retriever?,You convert a vector store into a retriever using the `as_retriever` method.
-What is the role of the retriever in the retrieval-augmented generation (RAG) system?,"The retriever fetches relevant document chunks based on query vectors, providing contextually relevant responses."
-Which embedding model is used in the example provided for retriever creation?,The embedding model 'JulsdL/e2erag-arctic-m' from Hugging Face is used in the example.

flashcards/flashcards_52137f3d-8a8d-4891-9216-ab3bbc6ee66a.csv

@@ -1,11 +0,0 @@
-Front,Back
-What library is used to load the fine-tuned embedding model for retriever creation?,The HuggingFaceEmbeddings class is used to load the fine-tuned embedding model.
-What model is loaded for embedding in the retriever creation process?,The model 'JulsdL/e2erag-arctic-m' is loaded for embedding.
-On which device is the embedding model set to run?,The embedding model is set to run on a CUDA device.
-Which class is used to set up the vector store in retriever creation?,The FAISS class from Meta is used to set up the vector store.
-What is the purpose of the vector store in the retriever creation process?,The vector store powers the dense vector search by storing documents that have been split into chunks and embedded.
-How is the vector store created in the retriever creation process?,The vector store is created from documents that have been split into chunks and embedded using the loaded embedding model.
-How is the vector store converted into a retriever?,The vector store is converted into a retriever using the 'as_retriever()' method.
-What is a retriever used for in the context of retriever creation?,A retriever is used to retrieve context based on a query for a language model.
-What is the benefit of creating a retriever in a RAG setup?,Creating a retriever enhances the capabilities of language models by providing them with relevant context for generating responses.
-What does 'RAG' stand for in the context of retriever creation?,'RAG' stands for Retrieval-Augmented Generation.

flashcards/flashcards_545f9f54-f8b0-4549-b27d-66b7303a017e.csv

@@ -1,11 +0,0 @@
-Front,Back
-What tool is used to load documents from a PDF file in the retriever creation process?,The PyMuPDFLoader is used to load documents from a PDF file.
-What is the purpose of the RecursiveCharacterTextSplitter in the retriever creation process?,The RecursiveCharacterTextSplitter is used to divide the documents into manageable chunks based on specific rules such as token limits and preferred split characters.
-Which library is used to create a dense vector store in the retriever creation process?,Meta's FAISS library is used to create a dense vector store from the document chunks and the embedding model.
-How do you load a pre-trained embedding model from Hugging Face to run on a CUDA device?,Use the HuggingFaceEmbeddings class with the model name and specify the device as CUDA in the model_kwargs parameter.
-What is the role of the embedding model in the retriever creation process?,The embedding model is used to convert text into vectors that can be stored in the vector store for efficient retrieval.
-How do you convert a vector store into a retriever?,Use the as_retriever method on the vector store to convert it into a retriever.
-What is the first step in the retriever creation process?,The first step is loading the documents using PyMuPDFLoader.
-Which model name is used in the provided example for the embedding model?,"The model name used is ""JulsdL/e2erag-arctic-m""."
-What is the purpose of the vector store in the retriever creation process?,"The vector store holds the dense vector representations of document chunks, allowing for efficient retrieval based on query embeddings."
-Which Python import is necessary to use FAISS for creating a vector store?,You need to import FAISS from langchain_community.vectorstores.

flashcards/flashcards_af3650ed-34b5-4040-b672-036e1cf3b8e3.csv

@@ -1,11 +0,0 @@
-Front,Back
-What is the first step in creating a retriever?,The first step in creating a retriever is loading a fine-tuned embedding model from the hub using the HuggingFaceEmbeddings class.
-Which class is used to load the embedding model?,The HuggingFaceEmbeddings class is used to load the embedding model.
-How is the model 'JulsdL/e2erag-arctic-m' loaded onto the GPU?,"The model 'JulsdL/e2erag-arctic-m' is loaded onto the GPU by setting the model_kwargs parameter to {""device"": ""cuda""} in the HuggingFaceEmbeddings class."
-What library is used to set up the vector store?,Meta's FAISS library is used to set up the vector store.
-What does the vector store manage?,The vector store manages the dense vectors generated by the embedding model.
-How is the vector store converted into a retriever?,The vector store is converted into a retriever using the as_retriever() method.
-What is the purpose of the retriever in this context?,The purpose of the retriever is to efficiently fetch relevant document vectors based on the query vectors.
-What is the role of the embedding model in the retriever creation process?,"The embedding model generates dense vectors for the documents, which are then managed by the vector store and used by the retriever to find relevant information."
-Which parameter in the HuggingFaceEmbeddings class specifies the device to be used?,"The model_kwargs parameter specifies the device to be used, such as ""cuda"" for GPU."
-What does the FAISS library stand for?,FAISS stands for Facebook AI Similarity Search.

notebook_tutor/chainlit_frontend.py

@@ -75,24 +75,25 @@ async def main(message: cl.Message):
         flashcard_filename="",
     )
 
-    print(f"Initial state: {state}")
+    print("\033[93m" + f"Initial state: {state}" + "\033[0m")
 
     # Process the message through the LangGraph chain
     for s in tutor_chain.stream(state, {"recursion_limit": 10}):
-        print(f"State after processing: {s}")
+        print("\033[93m" + f"State after processing: {s}" + "\033[0m")
 
         # Extract messages from the state
         if "__end__" not in s:
             agent_state = next(iter(s.values()))
             if "messages" in agent_state:
                 response = agent_state["messages"][-1].content
-                print(f"Response: {response}")
+                print("\033[93m" + f"Response: {response}" + "\033[0m")
                 await cl.Message(content=response).send()
             else:
                 print("Error: No messages found in agent state.")
         else:
             # Extract the final state
             final_state = next(iter(s.values()))
+            print("\033[93m" + f"Final state: {final_state}" + "\033[0m")
 
             # Check if the quiz was created and send it to the frontend
             if final_state.get("quiz_created"):
@@ -109,20 +110,20 @@ async def main(message: cl.Message):
                 flashcards_message = final_state["messages"][-1].content
                 await cl.Message(content=flashcards_message).send()
 
-                # Create a full path to the file
+                # Create a relative path to the file
                 flashcard_filename = final_state["flashcard_filename"]
-                print(f"Flashcard filename: {flashcard_filename}")
-                flashcard_path = os.path.abspath(flashcard_filename)
-                print(f"Flashcard path: {flashcard_path}")
+                print("\033[93m" + f"Flashcard filename: {flashcard_filename}" + "\033[0m")
+                flashcard_path = os.path.join(".files", flashcard_filename)
+                print("\033[93m" + f"Flashcard path: {flashcard_path}" + "\033[0m")
 
                 # Use the File class to send the file
-                file_element = cl.File(name=os.path.basename(flashcard_filename), path=flashcard_path)
-                print(f"Sending flashcards file: {file_element}")
+                file_element = cl.File(name=os.path.basename(flashcard_path), path=flashcard_path)
+                print("\033[93m" + f"Sending flashcards file: {file_element}" + "\033[0m")
                 await cl.Message(
                     content="Here are your flashcards:",
                     elements=[file_element]
                 ).send()
 
-            print("Reached END state.")
+            print("\033[93m" + "Reached END state." + "\033[0m")
 
             break

notebook_tutor/tools.py

@@ -22,8 +22,8 @@ class FlashcardTool(BaseTool):
     ) -> str:
         """Use the tool to create flashcards."""
         filename = f"flashcards_{uuid.uuid4()}.csv"
-        save_path = os.path.join('flashcards', filename)  # Save in 'flashcards' directory
-        os.makedirs(os.path.dirname(save_path), exist_ok=True)
+        save_path = os.path.join('.files', filename)
+        # os.makedirs(os.path.dirname(save_path), exist_ok=True) # Create directory if it doesn't exist
         with open(save_path, 'w', newline='') as csvfile:
             fieldnames = ['Front', 'Back']
             writer = csv.DictWriter(csvfile, fieldnames=fieldnames)