Duplicate from ashrma/Chat-with-Docs

Co-authored-by: Anoop Sharma <[email protected]>

.gitattributes

@@ -0,0 +1,34 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,51 @@
+---
+title: Chat with Docs
+emoji: 🦙
+license: mit
+sdk: streamlit
+python_version: 3.9
+app_file: app.py
+colorFrom: pink
+colorTo: blue
+pinned: false
+duplicated_from: ashrma/Chat-with-Docs
+---
+# Chat-with-Docs
+
+![image](https://user-images.githubusercontent.com/26565263/236671146-5fc5d5f0-4acb-40c7-9d9a-dc072efd8078.png)
+
+Chat with your Docs and gain better insights. Powered by `LlamaIndex` and `Streamlit` is used for UI. 
+Handles `CSV/PDFs/Txt/Doc`. CSV file is catered via [PandasAI](https://llamahub.ai/l/pandas_ai) loader and rest of the docs are handled via 
+`GPTVectorStoreIndex`.
+
+Clone the repo or copy the `.py ` file in your local machine. 
+
+## Install required Dependencies
+```
+pip install -r requirements.txt
+```
+
+## Create a folder in the root dir and name it as `documents`
+
+## Run the application
+`streamlit run chat_with_docs.py`
+
+## How to Contribute
+Feel free to open any Issue or PR request. This small application can help anyone to interact with their docs more smartly in just 2-3 steps.
+
+## Roadmap
+- [ ] Add support for choosing in between GPT-3/GPT-3.5/GPT-4 or HuggingFace model for creating vectors and generating rich responses.
+- [x] Blog explaining the entire application in detail.
+- [ ] Add Docker support.
+- [ ] Deploy the project on Streamlit or DataButton platform.
+- [ ] Add support to handle multiple files at once.
+
+## Snapshots
+- Upload a CSV file. Get better insights by just asking question, Render graphs based on the Data
+![image](https://user-images.githubusercontent.com/26565263/236671237-8517eecd-59f5-4961-8e33-772a26e92962.png)
+![image](https://user-images.githubusercontent.com/26565263/236671280-e5e9da7a-dd32-4af2-bd79-42545ad67d07.png)
+![image](https://user-images.githubusercontent.com/26565263/236671344-31967a79-2601-4cf2-bb2e-12a9eaf9429d.png)
+
+- In Doc section, Upload PDFs/Txt/Docs to chat with your docs directly. No need to press `CTRL+F` to search for anything in the Docs
+![image](https://user-images.githubusercontent.com/26565263/236671378-650d387f-57ad-4738-9bd0-15229f7e2e1d.png)
+![image](https://user-images.githubusercontent.com/26565263/236671580-0b032941-6c89-430a-a42c-f68655d39f71.png)

app.py

@@ -0,0 +1,131 @@
+from llama_index import GPTVectorStoreIndex, SimpleDirectoryReader
+from llama_index import download_loader
+from pandasai.llm.openai import OpenAI
+from matplotlib import pyplot as plt
+import streamlit as st
+import pandas as pd
+import os
+
+
+documents_folder = "./documents"
+
+# Load PandasAI loader, Which is a wrapper over PandasAI library
+PandasAIReader = download_loader("PandasAIReader")
+
+st.title("Welcome to `ChatwithDocs`")
+st.header("Interact with Documents such as `PDFs/CSV/Docs` using the power of LLMs\nPowered by `LlamaIndex🦙` \nCheckout the [GITHUB Repo Here](https://github.com/anoopshrma/Chat-with-Docs) and Leave a star⭐")
+
+
+def get_csv_result(df, query):
+  reader = PandasAIReader(llm=csv_llm)
+  response = reader.run_pandas_ai(
+    df, 
+    query, 
+    is_conversational_answer=False
+    )
+  return response
+
+def save_file(doc):        
+    fn = os.path.basename(doc.name)
+    # open read and write the file into the server
+    open(documents_folder+'/'+fn, 'wb').write(doc.read())
+    # Check for the current filename, If new filename
+    # clear the previous cached vectors and update the filename 
+    # with current name     
+    if st.session_state.get('file_name'):
+        if st.session_state.file_name != fn:
+            st.cache_resource.clear()
+            st.session_state['file_name'] = fn
+    else:
+        st.session_state['file_name'] = fn
+
+    return fn
+
+def remove_file(file_path):
+    # Remove the file from the Document folder once 
+    # vectors are created
+    if os.path.isfile(documents_folder+'/'+file_path):
+        os.remove(documents_folder+'/'+file_path)
+
+    
+
+@st.cache_resource
+def create_index():
+    # Create vectors for the file stored under Document folder. 
+    # NOTE: You can create vectors for multiple files at once.
+    documents = SimpleDirectoryReader(documents_folder).load_data()
+    index = GPTVectorStoreIndex.from_documents(documents)
+    return index
+
+
+
+def query_doc(vector_index, query):
+    # Applies Similarity Algo, Finds the nearest match and 
+    # take the match and user query to OpenAI for rich response
+    query_engine = vector_index.as_query_engine()
+    response = query_engine.query(query)
+    return response
+
+
+api_key = st.text_input("Enter your OpenAI API key here:", type="password")
+if api_key:
+    os.environ['OPENAI_API_KEY'] = api_key
+    csv_llm = OpenAI(api_token=api_key)
+
+
+tab1, tab2= st.tabs(["CSV", "PDFs/Docs"])
+
+with tab1:
+   
+   st.write("Chat with CSV files using PandasAI loader with LlamaIndex")
+   input_csv = st.file_uploader("Upload your CSV file", type=['csv'])
+   
+   if input_csv is not None:      
+    st.info("CSV Uploaded Successfully")
+    df = pd.read_csv(input_csv)
+    st.dataframe(df, use_container_width=True)
+    
+
+   st.write("---")
+   
+   input_text = st.text_area("Ask your query")
+   
+   if input_text is not None:
+    if st.button("Send"):
+      st.info("Your query: "+ input_text)
+      with st.spinner('Processing your query...'):
+        response = get_csv_result(df, input_text)
+      if plt.get_fignums():
+          st.pyplot(plt.gcf())
+      else:
+        st.success(response)
+
+    
+with tab2:
+   st.write("Chat with PDFs/Docs")
+   input_doc = st.file_uploader("Upload your Docs")
+   
+   if input_doc is not None: 
+       st.info("Doc Uploaded Successfully")
+       file_name = save_file(input_doc)
+       index = create_index()
+       remove_file(file_name)
+       
+   
+   st.write("---")
+   input_text = st.text_area("Ask your question")
+   
+   if input_text is not None:
+    if st.button("Ask"):
+        st.info("Your query: \n" +input_text)
+        with st.spinner("Processing your query.."):
+            response = query_doc(index, input_text)
+            print(response)
+        
+        st.success(response)
+
+        st.write("---")
+        # Shows the source documents context which 
+        # has been used to prepare the response
+        st.write("Source Documents")
+        st.write(response.get_formatted_sources())

documents/sample.txt

@@ -0,0 +1,176 @@
+UNITED STATES
+SECURITIES AND EXCHANGE COMMISSION
+Washington, D.C. 20549
+____________________________________________ 
+FORM 10-K
+____________________________________________ 
+(Mark One)
+
+☒ ANNUAL REPORT PURSUANT TO SECTION 13 OR 15(d) OF THE SECURITIES EXCHANGE ACT OF 1934
+For the fiscal year ended December 31, 2019
+OR
+☐ TRANSITION REPORT PURSUANT TO SECTION 13 OR 15(d) OF THE SECURITIES EXCHANGE ACT OF 1934
+For the transition period from_____ to _____            
+Commission File Number: 001-38902
+____________________________________________ 
+UBER TECHNOLOGIES, INC.
+(Exact name of registrant as specified in its charter)
+____________________________________________ 
+Delaware
+45-2647441
+(State or other jurisdiction of incorporation or organization)
+(I.R.S. Employer Identification No.)
+1455 Market Street, 4th Floor
+San Francisco, California 94103
+(Address of principal executive offices, including zip code)
+(415) 612-8582
+(Registrant’s telephone number, including area code)
+ ____________________________________________
+Securities registered pursuant to Section 12(b) of the Act:
+Title of each class
+ 
+Trading Symbol(s)
+ 
+Name of each exchange on which registered
+Common Stock, par value $0.00001 per share
+ 
+UBER
+ 
+New York Stock Exchange
+Securities registered pursuant to Section 12(g) of the Act: None
+Indicate by check mark whether the registrant is a well-known seasoned issuer, as defined in Rule 405 of the Securities Act. Yes  ☐ No ☒
+Indicate by check mark whether the registrant is not required to file reports pursuant to Section 13 or Section 15(d) of the Act. Yes  ☐ No  ☒
+Indicate by check mark whether the registrant (1) has filed all reports required to be filed by Section 13 or 15(d) of the Securities Exchange Act of 1934 during the preceding 12 months (or for such shorter period that the registrant was required to file such reports), and (2) has been subject to such filing requirements for the past 90 days. Yes  ☒ No ☐
+Indicate by check mark whether the registrant has submitted electronically every Interactive Data File required to be submitted pursuant to Rule 405 of Regulation S-T (§232.405 of this chapter) during the preceding 12 months (or for such shorter period that the registrant was required to submit such files). Yes  ☒ No ☐
+Indicate by check mark whether the registrant is a large accelerated filer, an accelerated filer, a non-accelerated filer, a smaller reporting company, or an emerging growth company. See the definitions of “large accelerated filer,” “accelerated filer,” “smaller reporting company,” and “emerging growth company” in Rule 12b-2 of the Exchange Act.
+
+
+Large accelerated filer
+☐
+ 
+ 
+Accelerated filer
+☐
+Non-accelerated filer
+☒
+ 
+ 
+Smaller reporting company
+☐
+ 
+ 
+ 
+ 
+Emerging growth company
+☐
+If an emerging growth company, indicate by check mark if the registrant has elected not to use the extended transition period for complying with any new or revised financial accounting standards provided pursuant to Section 13(a) of the Exchange Act.
+☐
+Indicate by check mark whether the registrant is a shell company (as defined in Rule 12b-2 of the Exchange Act). Yes ☐ No  ☒
+The aggregate market value of the voting and non-voting common equity held by non-affiliates of the registrant as of June 28, 2019, the last business day of the registrant's most recently completed second fiscal quarter, was approximately $59.7 billion based upon the closing price reported for such date on the New York Stock Exchange.
+The number of shares of the registrant's common stock outstanding as of February 19, 2020 was 1,723,775,076.
+DOCUMENTS INCORPORATED BY REFERENCE
+Portions of the registrant’s Definitive Proxy Statement relating to the Annual Meeting of Stockholders are incorporated by reference into Part III of this Annual Report on Form 10-K where indicated. Such Definitive Proxy Statement will be filed with the Securities and Exchange Commission within 120 days after the end of the registrant’s fiscal year ended December 31, 2019.
+
+
+UBER TECHNOLOGIES, INC.
+TABLE OF CONTENTS
+ 
+ 
+Pages
+ 
+Special Note Regarding Forward-Looking Statements
+2
+ 
+ 
+ 
+PART I
+ 
+ 
+Item 1.
+Business
+4
+Item 1A.
+Risk Factors
+9
+Item 1B.
+Unresolved Staff Comments
+44
+Item 2.
+Properties
+44
+Item 3.
+Legal Proceedings
+44
+Item 4.
+Mine Safety Disclosures
+45
+ 
+ 
+ 
+PART II
+ 
+ 
+Item 5.
+Market for Registrant’s Common Equity, Related Stockholder Matters and Issuer Purchases of Equity Securities
+45
+Item 6.
+Selected Financial Data
+46
+Item 7.
+Management’s Discussion and Analysis of Financial Condition and Results of Operations
+48
+Item 7A.
+Quantitative and Qualitative Disclosures About Market Risk
+75
+Item 8.
+Financial Statements and Supplementary Data
+76
+Item 9.
+Changes in and Disagreements with Accountants on Accounting and Financial Disclosure
+139
+Item 9A.
+Controls and Procedures
+139
+Item 9B.
+Other Information
+140
+ 
+ 
+ 
+PART III
+ 
+ 
+Item 10.
+Directors, Executive Officers and Corporate Governance
+140
+Item 11.
+Executive Compensation
+140
+Item 12.
+Security Ownership of Certain Beneficial Owners and Management and Related Stockholder Matters
+140
+Item 13.
+Certain Relationships and Related Transactions, and Director Independence
+140
+Item 14.
+Principal Accounting Fees and Services
+140
+ 
+ 
+ 
+PART IV
+ 
+ 
+Item 15.
+Exhibits, Financial Statement Schedules
+140
+Item 16.
+Form 10-K Summary
+140
+ 
+Exhibit Index
+141
+ 
+Signatures
+143
+

requirements.txt

@@ -0,0 +1,5 @@
+llama-index
+streamlit
+pandas
+pandasai
+PyPDF2