init

LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

app.py

@@ -0,0 +1,244 @@
+import streamlit as st
+import pandas as pd
+from langchain.prompts import FewShotPromptTemplate, PromptTemplate
+from langchain.llms import OpenAIChat
+from langchain.callbacks import get_openai_callback
+from langchain.chains import LLMChain
+
+
+# Function to generate empty dataframe
+def create_empty_dataframe(rows, columns):
+    data = {column: [""] * rows for column in columns}
+    return pd.DataFrame(data)
+
+
+# Sidebar
+st.sidebar.title("Configuration")
+selected_model = st.sidebar.selectbox(
+    "Select Model",
+    [
+        "chat-gpt-3.5-turbo",
+    ],  # "gpt3-text-davinci-003"
+)
+selected_dataset = st.sidebar.selectbox(
+    "Select Dataset", ["Regulatory Changes", "Dealer Risk", "Custom"]
+)
+openai_api_key = st.sidebar.text_input("OpenAI API Key", type="password")
+
+
+prompt_prefix = """You are a news recommender and your job is to look at an article and decide if I would find it useful or not.
+{dataset_specific_prompt}
+I'm going to pass in a few headlines and publishers. Can you just respond with "Relevant" or "Not Relevant" for the headlines I send you?"""
+
+if selected_dataset == "Regulatory Changes":
+    dataset_specific_prompt = """I work in a bank/credit card company and I'm looking for news articles that deal with changes in the regulatory environment. 
+It could be articles about regulatory authorities like CFPB or OCC announcing changes in regulation or a new focus areas. 
+Or articles like the president or senate or government announcing new regulations. 
+Or peer financial institutions getting fined for violations. 
+To reiterate I'm especially looking for articles in the finance, fin-tech, banking and credit card sector. 
+**I don't need think pieces or opinion pieces or editorials that talk about the consequences of a regulation but just the ones that talk about objective changes in the regulatory environment that might impact my work**"""
+    examples = [
+        {
+            "headline": "CFPB Launches Inquiry Into the Business Practices of Data Brokers",
+            "publisher": "CFPB",
+            "answer": "Relevant",
+        },
+        {
+            "headline": "OCC Issues Prohibition Order, Fines Former Wells Fargo Executive $17 Million in Settlement",
+            "publisher": "NYT",
+            "answer": "Relevant",
+        },
+        {
+            "headline": "CFPB Announces Appointments of New Advisory Committee Members",
+            "publisher": "CFPB",
+            "answer": "Not Relevant",
+        },
+    ]
+    prediction_examples = [
+        {
+            "headline": "Ray Dalio Commentary: What I Think About the Silicon Valley Bank Situation",
+            "publisher": "Yahoo Entertainment",
+        },
+        {
+            "headline": "Financial Regulators, Black History and Epistemic Capital",
+            "publisher": "Harvard School of Engineering",
+        },
+        {
+            "headline": "Agencies Issue Joint Statement on Crypto-Asset Risks to Banking Organizations",
+            "publisher": "OCC",
+        },
+        {
+            "headline": "Innovating during a regulatory wave",
+            "publisher": "Venture beat",
+        },
+        {
+            "headline": "CFPB Proposes Rule to Rein in Excessive Credit Card Late Fees",
+            "publisher": "CFPB",
+        },
+    ]
+
+elif selected_dataset == "Dealer Risk":
+    dataset_specific_prompt = """I work in a auto-loan company and I'm looking for news articles that talk about with automobile dealerships being in the news for committing fraud and crime.
+I am not interested in just about any crime or fraud but only that which is related to dealerships. 
+I am NOT concerned about articles that where the dealership is the victim of a crime. 
+I am only interested where the dealerships are the perpetrators of said crime or accusations.
+"""
+    examples = [
+        {
+            "headline": "Kentucky Police arrest suspect following $600,000 multiple car heist that took less than 45 seconds",
+            "publisher": "Business Insider",
+            "answer": "Not Relevant",
+        },
+        {
+            "headline": "Car Dealership workers kidnapped, tortured in Armed Robbery",
+            "publisher": "ABC7 Chicago",
+            "answer": "Not Relevant",
+        },
+        {
+            "headline": "Auto Dealership Owner Sentenced to 10 Years in Prison for $1.5 Million Fraud Scheme",
+            "publisher": "Long Island Business News",
+            "answer": "Relevant",
+        },
+    ]
+    prediction_examples = [
+        {
+            "headline": "New York City Pharmacist charged with running healthcare fraud scheme",
+            "publisher": "amnycom",
+        },
+        {
+            "headline": "California man arrested for 3.6 million paycheck protection program fraud",
+            "publisher": "DoJ",
+        },
+        {
+            "headline": "Former Moline Businessman sentenced to 41 months in connection with scam to rollback odometeres at used car dealerships",
+            "publisher": "DoJ",
+        },
+        {
+            "headline": "Kentucky car dealer convicted in truck warranty fraud scheme",
+            "publisher": "apnews.com",
+        },
+        {
+            "headline": "Owner of car dealership pleads guilty to defraduing financing company",
+            "publisher": "Automotive News",
+        },
+    ]
+
+elif selected_dataset == "Custom":
+    dataset_specific_prompt = """I work in a XYZ company and I'm looking for news articles that deal with XYZ"""
+    examples = [{k: "" for k in ["headline", "publisher", "answer"]} for i in range(3)]
+    prediction_examples = [{k: "" for k in ["headline", "publisher"]} for i in range(5)]
+
+
+prompt_prefix = prompt_prefix.format(
+    dataset_specific_prompt=dataset_specific_prompt
+).replace("\n", " ")
+
+
+# Main panel
+st.title("News Topic Subscription")
+
+st.header("Dataset")
+st.write(f"Prompt data for {selected_dataset}:")
+
+prompt_data = st.text_area(
+    "Prompt Prefix", prompt_prefix, placeholder="Enter dataset prompt here", height=300
+)
+
+st.header("Few Shot Examples")
+
+columns = ["headline", "publisher", "answer"]
+data = pd.DataFrame({k: v for k, v in example.items()} for example in examples)
+data.columns = columns
+# data["Relevance"] = [""] * 3
+
+for index, row in data.iterrows():
+    row["headline"] = st.text_input(
+        f"Headline", row["headline"], key=f"headline_{index}"
+    )
+    row["publisher"] = st.text_input(
+        f"Publisher", row["publisher"], key=f"publisher_{index}"
+    )
+    choice_index = (
+        ["Relevant", "Not Relevant"].index(row["answer"]) if row["answer"] != "" else 0
+    )
+    row["answer"] = st.radio(
+        f"Relevance {index + 1}",
+        index=choice_index,
+        options=["Relevant", "Not Relevant"],
+        key=f"answer_{index}",
+    )
+    st.markdown("---")
+
+st.header("Predictions")
+
+predictions_data = pd.DataFrame(
+    {k: v for k, v in example.items()} for example in prediction_examples
+)
+predictions_data["answer"] = [""] * 5
+prediction_labels = []
+
+for index, row in predictions_data.iterrows():
+    row["headline"] = st.text_input(
+        f"headline", row["headline"], key=f"prediction_headline_{index}"
+    )
+    row["publisher"] = st.text_input(
+        f"publisher", row["publisher"], key=f"prediction_publisher_{index}"
+    )
+    prediction_labels.append(st.empty())
+    st.markdown("---")
+
+
+llm = OpenAIChat(temperature=0.0, openai_api_key=openai_api_key)
+example_prompt = PromptTemplate(
+    input_variables=["headline", "publisher", "answer"],
+    template="Headline: {headline}\nPublisher: {publisher}\nAnswer: {answer}",
+)
+
+
+def update_relevance(predictions_data):
+
+    prompt = FewShotPromptTemplate(
+        prefix=prompt_data,
+        examples=data.to_dict("records"),
+        example_prompt=example_prompt,
+        suffix="Headline: {headline}\nPublisher: {publisher}\nAnswer: ",
+        input_variables=["headline", "publisher"],
+    )
+    print(prompt.format(headline="", publisher=""))
+    print(data)
+    news_predictor_chain = LLMChain(prompt=prompt, llm=llm, verbose=True)
+    token_counts = []
+
+    for index, row in predictions_data.iterrows():
+        if row["headline"] != "":
+            with get_openai_callback() as cb:
+                predictions_data.loc[index, "answer"] = news_predictor_chain.run(
+                    {
+                        "headline": row["headline"],
+                        "publisher": row["publisher"]
+                        if row["publisher"] != ""
+                        else "Unknown",
+                    }
+                )
+                token_counts.append(cb.total_tokens)
+
+        else:
+            predictions_data.loc[index, "answer"] = ""
+    st.text(
+        f"{sum(token_counts)} tokens used in this run for {len(token_counts)} articles. This costed ${sum(token_counts) * 0.002/1000 } @$0.002/1K tokens."
+    )
+    st.text(
+        f"For predicting 1000 articles, it would cost ${sum(token_counts)/len(token_counts) * 0.002 } at the current rate. "
+    )
+    return predictions_data
+
+
+if st.button("Run Model", type="primary"):
+    predictions_data = update_relevance(predictions_data)
+    for index, row in predictions_data.iterrows():
+        if row["answer"] != "":
+            with prediction_labels[index].container():
+                if row["answer"] == "Relevant":
+                    st.markdown("This article is **Relevant** :thumbsup:")
+                else:
+                    st.markdown("This article is **Not Relevant** :thumbsdown:")

int.ipynb

@@ -0,0 +1,515 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7114fd1c-671c-4616-a4fc-19ec40a8423f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.prompts import FewShotPromptTemplate\n",
+    "from langchain.llms import OpenAIChat\n",
+    "from langchain.callbacks import get_openai_callback\n",
+    "from langchain.chains import LLMChain\n",
+    "import requests\n",
+    "import datetime\n",
+    "\n",
+    "\n",
+    "\n",
+    "%reload_ext lab_black"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "295d30ed-ba52-43ab-8df0-4c7b8f9b663e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# url = 'https://newsapi.org/v2/sources'\n",
+    "# params = {\n",
+    "#     'apiKey': NEWS_API_KEY\n",
+    "# }\n",
+    "\n",
+    "# response = requests.get(url, params=params)\n",
+    "\n",
+    "# sources = response.json()['sources']\n",
+    "\n",
+    "# for source in sources:\n",
+    "#     print(source['id'], '-', source['name'])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "122e6596-286b-4fbf-b29f-f1e8db9434f6",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "finance_regulatory_bodies = [\n",
+    "    \"Consumer Financial Protection Bureau\",\n",
+    "    \"CFPB\",\n",
+    "    \"Federal Reserve System\",\n",
+    "    \"The Fed\",\n",
+    "    \"Securities and Exchange Commission\",\n",
+    "    \"SEC\",\n",
+    "    \"Commodity Futures Trading Commission\",\n",
+    "    \"CFTC\",\n",
+    "    \"National Credit Union Administration\",\n",
+    "    \"NCUA\",\n",
+    "    \"Office of the Comptroller of the Currency\",\n",
+    "    \"OCC\",\n",
+    "    \"Financial Industry Regulatory Authority\",\n",
+    "    \"FINRA\",\n",
+    "    \"Financial Stability Oversight Council\",\n",
+    "    \"FSOC\",\n",
+    "    \"Internal Revenue Service\",\n",
+    "    \"IRS\",\n",
+    "]\n",
+    "\n",
+    "finance_regulatory_bodies = \" OR \".join(\n",
+    "    [f\"'{s.lower()}'\" for s in finance_regulatory_bodies]\n",
+    ")\n",
+    "\n",
+    "query = f\"(regulation OR compliance OR legislation) AND (cfpb OR finra OR OCC)\"\n",
+    "# \"law OR regulation OR policy OR compliance OR legislation OR government oversight\"\n",
+    "# categories = \"business,technology,general\"    # \"category\": categories,\n",
+    "from_date = datetime.datetime.now() - datetime.timedelta(days=7)\n",
+    "to_date = datetime.datetime.now()\n",
+    "\n",
+    "url = \"https://newsapi.org/v2/everything\"\n",
+    "params = {\n",
+    "    \"q\": query,\n",
+    "    \"from\": from_date,\n",
+    "    \"to\": to_date,\n",
+    "    \"language\": \"en\",\n",
+    "    \"sortBy\": \"publishedAt\",\n",
+    "    \"pageSize\": 100,\n",
+    "    \"page\": 1,\n",
+    "    \"apiKey\": NEWS_API_KEY,\n",
+    "}\n",
+    "\n",
+    "response = requests.get(url, params=params)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6fb7e1b8-642a-456b-a161-91f6993edfc8",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "articles = response.json()[\"articles\"]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6f4784f7-61d4-4d8f-8b55-29c1373c2cb7",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "How about some more federal laws to fix banking? - Americanthinker.com - 2023-03-18T05:00:00Z\n",
+      "Bragar Eagel & Squire, P.C. Reminds Investors That Class Action Lawsuits Have Been Filed Against Block, Atlassian, PLDT, and Global Payments and Encourages Investors to Contact the Firm - GlobeNewswire - 2023-03-18T01:00:00Z\n",
+      "Innovating during a regulatory wave - VentureBeat - 2023-03-17T20:07:00Z\n",
+      "Consumer Watchdog Wants to Know Who's Watching Your Internet Behavior - Investopedia - 2023-03-17T15:26:26Z\n",
+      "Regulators Shut Down Banks, Raising Questions About Neutrality - Forbes - 2023-03-17T11:30:00Z\n",
+      "Moomoo's Continuous Innovation Journey on AWS - PRNewswire - 2023-03-17T11:00:00Z\n",
+      "Regulatory Failure 101: What the Collapse of Silicon Valley Bank Reveals - ProPublica - 2023-03-17T11:00:00Z\n",
+      "So Where Were the Regulators? - The Atlantic - 2023-03-17T11:00:00Z\n",
+      "Buy-backs of shares in CoinShares International Limited - Marketscreener.com - 2023-03-17T08:01:01Z\n",
+      "Buy-backs of shares in CoinShares International Limited - Yahoo Entertainment - 2023-03-17T08:00:00Z\n",
+      "Buy-backs of shares in CoinShares International Limited - GlobeNewswire - 2023-03-17T08:00:00Z\n",
+      "Bank Runs, Busts, and Bailouts Are Back. What It Means for Regulations and the Industry. - Barron's - 2023-03-17T06:52:39Z\n",
+      "AdvisorVault Partners With Compliant Workspace to Help FINRA Firms Achieve 17a-4 Compliance on Microsoft 365 - Yahoo Entertainment - 2023-03-17T01:06:00Z\n",
+      "Silicon Valley Bank ex-CEO backed Big Tech lobbying groups that targeted Dodd Frank, sought corporate tax cuts - CNBC - 2023-03-16T17:02:22Z\n",
+      "Blockchain Association seeks info from Fed, FDIC, and OCC on 'de-banking' crypto firms - Cointelegraph - 2023-03-16T16:45:00Z\n",
+      "Baldwin Mader Law Group Wins $6.1 Million Arbitration Award Against Fisker, Inc. - Yahoo Entertainment - 2023-03-16T15:45:00Z\n",
+      "What is Trading Software Development? - Thefrisky.com - 2023-03-16T14:30:58Z\n",
+      "INX ANNOUNCES INTEGRATION WITH POLYGON - Yahoo Entertainment - 2023-03-16T13:48:00Z\n",
+      "AI CEO on GPT-4: This Can Get “Super-Dangerous Very Quickly” - Futurism - 2023-03-16T13:43:43Z\n",
+      "INX ANNOUNCES INTEGRATION WITH POLYGON - PR Newswire UK - 2023-03-16T13:31:00Z\n",
+      "INX ANNOUNCES INTEGRATION WITH POLYGON - Marketscreener.com - 2023-03-16T13:14:03Z\n",
+      "INX ANNOUNCES INTEGRATION WITH POLYGON - Yahoo Entertainment - 2023-03-16T13:13:00Z\n",
+      "INX ANNOUNCES INTEGRATION WITH POLYGON - PRNewswire - 2023-03-16T13:13:00Z\n",
+      "CFPB Launches Long Overdue Probe Of Unaccountable Data Broker Market - Techdirt - 2023-03-16T12:36:58Z\n",
+      "Hashdex Celebrates Six-Month Anniversary of World’s First Bitcoin Futures ETF Registered Solely Under ‘33 Act by Ringing NYSE Opening Bell - GlobeNewswire - 2023-03-16T12:35:00Z\n",
+      "GigaStar, a Startup Bringing YouTube Creators and Fans Together as Partners, Completes a $4.8 Million Seed Round - PRNewswire - 2023-03-16T11:00:00Z\n",
+      "Ray Dalio Commentary: What I Think About the Silicon Valley Bank Situation - Yahoo Entertainment - 2023-03-15T22:41:59Z\n",
+      "Sterling Bancorp, Inc. Enters Into Plea Agreement with DOJ; Resolves DOJ Investigation of the Company Relating to the Advantage Loan Program; Revises Fourth Quarter and Full Year 2022 Unaudited Financial Results - Marketscreener.com - 2023-03-15T21:13:02Z\n",
+      "Retail Traders Left Hanging As Brokers Halt Transactions On Silicon Valley Bank Just As Bets Are Set To Pay Off - Forbes - 2023-03-15T19:27:06Z\n",
+      "How the 2008 financial crisis fuels today's populist politics - PBS - 2023-03-15T17:58:06Z\n",
+      "Patti Brennan Ranks # 5 in Barron s Top 1,200 Financial Advisors State by State - GlobeNewswire - 2023-03-15T17:25:00Z\n",
+      "Patti Brennan Ranks # 5 in Barron s Top 1,200 Financial Advisors State by State - Yahoo Entertainment - 2023-03-15T17:25:00Z\n",
+      "GoLogiq to Acquire Institutional Investment Advisory Firm, CPG Research & Advisory, in $15 Million Merger - GlobeNewswire - 2023-03-15T17:00:00Z\n",
+      "GoLogiq to Acquire Institutional Investment Advisory Firm, CPG Research & Advisory, in $15 Million Merger - Yahoo Entertainment - 2023-03-15T17:00:00Z\n",
+      "Webinar: Navigating Marketing Compliance Challenges in 2023 - Performline.com - 2023-03-15T16:24:05Z\n",
+      "Financial Regulators, Black History and Epistemic Capital - Harvard School of Engineering and Applied Sciences - 2023-03-15T13:30:43Z\n",
+      "How the last banking tumult fuels today's populist politics - seattlepi.com - 2023-03-15T12:20:48Z\n",
+      "How the last banking tumult fuels today's populist politics - ABC News - 2023-03-15T10:32:20Z\n",
+      "How the last banking tumult fuels today's populist politics - ABC News - 2023-03-15T10:32:12Z\n",
+      "How the last banking tumult fuels today's populist politics - Independent - 2023-03-15T10:00:45Z\n",
+      "How the last banking tumult fuels today's populist politics - Yahoo Entertainment - 2023-03-15T10:00:44Z\n",
+      "Central Pacific Financial : CPF Balance Sheet Update - Marketscreener.com - 2023-03-15T00:00:07Z\n",
+      "Bowman, The Innovation Imperative: Modernizing Traditional Banking - Federalreserve.gov - 2023-03-14T21:20:00Z\n",
+      "SVB collapse creates rift among Democrats over 2018 banking law - Yahoo Entertainment - 2023-03-14T20:53:04Z\n",
+      "Enterprise Health Sponsors ACOEM Ambassador Program - Yahoo Entertainment - 2023-03-14T15:42:00Z\n",
+      "Community Reinvestment Act (CRA) Expert Sarah Brons Joins Asurity's RiskExec Team - PRNewswire - 2023-03-14T15:02:00Z\n",
+      "Tom Oscherwitz Joins Informed.IQ as VP of Legal and Regulatory Advisor - PRNewswire - 2023-03-14T13:40:00Z\n",
+      "Biden Highlights Dodd-Frank Repeal in Calls for Stiffer Bank Regulations - Investopedia - 2023-03-14T11:33:00Z\n",
+      "Republicans Who Pushed for Financial Deregulation Blame Silicon Valley Bank Collapse on “Woke Agenda” - The New Republic - 2023-03-13T22:06:50Z\n",
+      "USDC's 'Black Swan' Depegging Could Have Been Avoided With Proper Regulatory Framework - CoinDesk - 2023-03-13T21:59:35Z\n",
+      "How Silicon Valley Bank skirted Washington's toughest banking rules - Yahoo Entertainment - 2023-03-13T21:55:01Z\n",
+      "It’s Not Just Fraud That Chilled Crypto Regulation - CoinDesk - 2023-03-13T20:56:08Z\n",
+      "SVB Moral: Idiot Businesses Get a Bailout—and Indebted College Kids Get Lawsuits - The New Republic - 2023-03-13T20:23:18Z\n",
+      "Fed announces regulatory review of Silicon Valley Bank failure - Yahoo Entertainment - 2023-03-13T20:22:21Z\n",
+      "INX Announces the Listing of Hashrate Asset Group's (HAG) Bitcoin Mining Security Token - Yahoo Entertainment - 2023-03-13T15:13:00Z\n",
+      "INX Announces the Listing of Hashrate Asset Group's (HAG) Bitcoin Mining Security Token - PR Newswire UK - 2023-03-13T14:54:00Z\n",
+      "INX Announces the Listing of Hashrate Asset Group's (HAG) Bitcoin Mining Security Token - Marketscreener.com - 2023-03-13T14:39:04Z\n",
+      "INX Announces the Listing of Hashrate Asset Group's (HAG) Bitcoin Mining Security Token - Yahoo Entertainment - 2023-03-13T14:38:00Z\n",
+      "INX Announces the Listing of Hashrate Asset Group's (HAG) Bitcoin Mining Security Token - PRNewswire - 2023-03-13T14:38:00Z\n",
+      "The Impact Of Evolving Cloud Regulations On Financial Services - Forbes - 2023-03-13T12:45:00Z\n",
+      "Bragar Eagel & Squire, P.C. Reminds Investors That Class Action Lawsuits Have Been Filed Against Global Payments, Caribou, Inspirato, and Kornit and Encourages Investors to Contact the Firm - GlobeNewswire - 2023-03-13T01:00:00Z\n"
+     ]
+    }
+   ],
+   "source": [
+    "for article in articles:\n",
+    "    print(article[\"title\"], \"-\", article[\"source\"][\"name\"], \"-\", article[\"publishedAt\"])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b3758b70-c1f8-41dd-8a2b-5d000e851fed",
+   "metadata": {},
+   "source": [
+    "# Langchain"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a21af9b3-6376-4226-81df-377d774f4a99",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "llm = OpenAIChat(temperature=0.0)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "87936389-312c-49de-a88d-40a7ad82a06d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Also, I don't need any article that talks about regulation but rather something that discusses a regulatory change that might impact my work"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "aae3283e-b042-4618-9070-073303985527",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "prefix = \"\"\"You are a news recommender and your job is to look at an article and decide if I would find it useful or not.\n",
+    "I work in a bank/credit card company and I'm looking for news articles that deal with changes in the regulatory environment. \n",
+    "It could be articles about regulatory authorities like CFPB or OCC announcing changes in regulation or a new focus areas. \n",
+    "Or articles like the president or senate or government announcing new regulations. \n",
+    "Or peer financial institutions getting fined for violations. \n",
+    "To reiterate I'm especially looking for articles in the finance, fin-tech, banking and credit card sector. \n",
+    "**I don't need think pieces or opinion pieces or editorials that talk about the consequences of a regulation but just the ones that talk about objective changes in the regulatory environment that might impact my work**\n",
+    "I'm going to pass in a few headlines. Can you just respond with \"relevant\" or \"not relevant\" for the headlines I send you?\"\"\"\n",
+    "\n",
+    "examples = [\n",
+    "    {\n",
+    "        \"headline\": \"CFPB Launches Inquiry Into the Business Practices of Data Brokers\",\n",
+    "        \"publisher\": \"CFPB\",\n",
+    "        \"answer\": \"relevant\",\n",
+    "    },\n",
+    "    {\n",
+    "        \"headline\": \"OCC Issues Prohibition Order, Fines Former Wells Fargo Executive $17 Million in Settlement\",\n",
+    "        \"publisher\": \"NYT\",\n",
+    "        \"answer\": \"relevant\",\n",
+    "    },\n",
+    "    {\n",
+    "        \"headline\": \"Innovating during a regulatory wave\",\n",
+    "        \"publisher\": \"Venture beat\",\n",
+    "        \"answer\": \"not relevant\",\n",
+    "    },\n",
+    "]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "07b77a00-3f61-4d4f-9ff5-fbe330e7e6b2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "example_prompt = PromptTemplate(\n",
+    "    input_variables=[\"headline\", \"publisher\", \"answer\"],\n",
+    "    template=\"Headline: {headline}\\nPublisher: {publisher}\\nAnswer: {answer}\",\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "da1d7f5d-bd1b-489f-ac80-804ccc241fa4",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "prompt = FewShotPromptTemplate(\n",
+    "    prefix=prefix,\n",
+    "    examples=examples,\n",
+    "    example_prompt=example_prompt,\n",
+    "    suffix=\"Headline: {headline}\\nPublisher: {publisher}\\nAnswer: \",\n",
+    "    input_variables=[\"headline\", \"publisher\"],\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "51216122-1424-4f85-9d72-a094f644f90c",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "You are a news recommender and your job is to look at an article and decide if I would find it useful or not.\n",
+      "I work in a bank/credit card company and I'm looking for news articles that deal with changes in the regulatory environment. \n",
+      "It could be articles about regulatory authorities like CFPB or OCC announcing changes in regulation or a new focus areas. \n",
+      "Or articles like the president or senate or government announcing new regulations. \n",
+      "Or peer financial institutions getting fined for violations. \n",
+      "To reiterate I'm especially looking for articles in the finance, fin-tech, banking and credit card sector. \n",
+      "**I don't need think pieces or opinion pieces or editorials that talk about the consequences of a regulation but just the ones that talk about objective changes in the regulatory environment that might impact my work**\n",
+      "I'm going to pass in a few headlines. Can you just respond with \"relevant\" or \"not relevant\" for the headlines I send you?\n",
+      "\n",
+      "\n",
+      "\n",
+      "Headline: CFPB Launches Inquiry Into the Business Practices of Data Brokers\n",
+      "Publisher: CFPB\n",
+      "Answer: relevant\n",
+      "\n",
+      "Headline: OCC Issues Prohibition Order, Fines Former Wells Fargo Executive $17 Million in Settlement\n",
+      "Publisher: NYT\n",
+      "Answer: relevant\n",
+      "\n",
+      "Headline: Innovating during a regulatory wave\n",
+      "Publisher: Venture beat\n",
+      "Answer: not relevant\n",
+      "\n",
+      "Headline: CFPB Launches Inquiry Into the Business Practices of Data Brokers\n",
+      "Publisher: CFPB\n",
+      "Answer: \n"
+     ]
+    }
+   ],
+   "source": [
+    "print(\n",
+    "    prompt.format(\n",
+    "        **{\n",
+    "            \"headline\": \"CFPB Launches Inquiry Into the Business Practices of Data Brokers\",\n",
+    "            \"publisher\": \"CFPB\",\n",
+    "        }\n",
+    "    )\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d1bcdf3a-c41c-4815-a802-7903e6ce43b2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "news_predictor_chain = LLMChain(prompt=prompt, llm=llm)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c1dc07e5-e2fb-4a5b-8078-de83a3c8e1d4",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'headline': 'Ray Dalio Commentary: What I Think About the Silicon Valley Bank Situation',\n",
+       " 'publisher': 'Yahoo Entertainment',\n",
+       " 'text': 'not relevant'}"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "news_predictor_chain.run(\n",
+    "    {\n",
+    "        \"headline\": \"Ray Dalio Commentary: What I Think About the Silicon Valley Bank Situation\",\n",
+    "        \"publisher\": \"Yahoo Entertainment\",\n",
+    "    }\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "91d5dfc2-08ed-417e-abc6-cd961ac62a79",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'headline': 'Financial Regulators, Black History and Epistemic Capital',\n",
+       " 'publisher': 'Harvard School of Engineering',\n",
+       " 'text': 'not relevant'}"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "news_predictor_chain.run(\n",
+    "    {\n",
+    "        \"headline\": \"Financial Regulators, Black History and Epistemic Capital\",\n",
+    "        \"publisher\": \"Harvard School of Engineering\",\n",
+    "    }\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b2b868b0-1d7e-4074-b55d-fab0e8b12bf6",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'headline': 'Agencies Issue Joint Statement on Crypto-Asset Risks to Banking Organizations',\n",
+       " 'publisher': 'OCC',\n",
+       " 'text': 'relevant'}"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "news_predictor_chain(\n",
+    "    {\n",
+    "        \"headline\": \"Agencies Issue Joint Statement on Crypto-Asset Risks to Banking Organizations\",\n",
+    "        \"publisher\": \"OCC\",\n",
+    "    }\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "8d18d0b9-274d-4ae9-8c0e-1dc1e6f42add",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'headline': 'Acting Comptroller of the Currency Testifies on Regulatory Priorities',\n",
+       " 'publisher': 'The Washington Post',\n",
+       " 'text': 'relevant'}"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "news_predictor_chain.run(\n",
+    "    {\n",
+    "        \"headline\": \"Acting Comptroller of the Currency Testifies on Regulatory Priorities\",\n",
+    "        \"publisher\": \"The Washington Post\",\n",
+    "    }\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "fe55edc1-43f3-4e13-a3f7-53f24593a0a5",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'headline': 'The Impact Of Evolving Cloud Regulations On Financial Services',\n",
+       " 'publisher': 'Forbes',\n",
+       " 'text': 'relevant'}"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "news_predictor_chain.run(\n",
+    "    {\n",
+    "        \"headline\": \"The Impact Of Evolving Cloud Regulations On Financial Services\",\n",
+    "        \"publisher\": \"Forbes\",\n",
+    "    }\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d0a45c24-dfff-4af3-897f-5182c21f1d06",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "3"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "sum([1, 2])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e4bc7325-6748-4837-b6b0-bff4467c077b",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python [conda env:news_ai]",
+   "language": "python",
+   "name": "conda-env-news_ai-py"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

requirements.txt

@@ -0,0 +1,3 @@
+langchain>=0.0.11
+pandas
+openai