pages/Workflow & Model Overview.py

import streamlit as st

# Custom CSS for better styling
st.markdown("""

    <style>

        .main-title {

            font-size: 36px;

            color: #4A90E2;

            font-weight: bold;

            text-align: center;

        }

        .sub-title {

            font-size: 24px;

            color: #4A90E2;

            margin-top: 20px;

        }

        .section {

            background-color: #f9f9f9;

            padding: 15px;

            border-radius: 10px;

            margin-top: 20px;

        }

        .section h2 {

            font-size: 22px;

            color: #4A90E2;

        }

        .section p, .section ul {

            color: #666666;

        }

        .link {

            color: #4A90E2;

            text-decoration: none;

        }

    </style>

""", unsafe_allow_html=True)

# Introduction
st.markdown('<div class="main-title">Financial Sentiment Analysis with Spark NLP</div>', unsafe_allow_html=True)

st.markdown("""

<div class="section">

    <p>Welcome to the Spark NLP Financial Sentiment Analysis Demo App! Financial sentiment analysis is the process of identifying and categorizing the emotional tone of financial news articles, reports, tweets, and other textual data related to finance. Using Spark NLP, this app demonstrates how to accurately analyze the sentiment of financial texts.</p>

    <p>This app leverages Spark NLP's advanced models to detect sentiments in financial texts, helping users gain insights into market sentiment and make informed decisions.</p>

</div>

""", unsafe_allow_html=True)

st.image('images/financial-sentiment.png', use_column_width='auto')

# About Financial Sentiment Analysis
st.markdown('<div class="sub-title">About Financial Sentiment Analysis</div>', unsafe_allow_html=True)
st.markdown("""

<div class="section">

    <p>Financial sentiment analysis involves analyzing texts to determine whether the expressed sentiment is positive, negative, or neutral with respect to financial markets. It is widely used by investors, traders, and analysts to gauge market sentiment and predict market movements.</p>

    <p>Applications of financial sentiment analysis include analyzing news articles, earnings reports, social media posts, and more to identify trends and make data-driven investment decisions.</p>

</div>

""", unsafe_allow_html=True)

# Using Sentiment Analysis in Spark NLP
st.markdown('<div class="sub-title">Using Sentiment Analysis in Spark NLP</div>', unsafe_allow_html=True)
st.markdown("""

<div class="section">

    <p>The following pipeline demonstrates how to use Spark NLP for financial sentiment analysis:</p>

</div>

""", unsafe_allow_html=True)

# Setup Instructions
st.markdown('<div class="sub-title">Setup</div>', unsafe_allow_html=True)
st.markdown('<p>To install Spark NLP in Python, use your favorite package manager (conda, pip, etc.). For example:</p>', unsafe_allow_html=True)
st.code("""

pip install spark-nlp

pip install pyspark

""", language="bash")

st.markdown("<p>Then, import Spark NLP and start a Spark session:</p>", unsafe_allow_html=True)
st.code("""

import sparknlp



# Start Spark Session

spark = sparknlp.start()

""", language='python')

# Example Pipeline for Financial Sentiment Analysis
st.markdown('<div class="sub-title">Example Usage: Financial Sentiment Analysis with Spark NLP</div>', unsafe_allow_html=True)
st.code('''

from sparknlp.base import DocumentAssembler

from sparknlp.annotator import BertSentenceEmbeddings, ClassifierDLModel

from pyspark.ml import Pipeline



# Step 1: Document Assembler

document = DocumentAssembler()\\

  .setInputCol("text")\\

  .setOutputCol("document")



# Step 2: Sentence Embeddings

embeddings = BertSentenceEmbeddings\\

    .pretrained('sent_bert_wiki_books_sst2', 'en') \\

    .setInputCols(["document"])\\

    .setOutputCol("sentence_embeddings")



# Step 3: Sentiment Classifier

sentimentClassifier = ClassifierDLModel.pretrained("classifierdl_bertwiki_finance_sentiment", "en") \\

  .setInputCols(["sentence_embeddings"]) \\

  .setOutputCol("class_")



# Define the pipeline

financial_sentiment_pipeline = Pipeline(

    stages=[document, 

            embeddings, 

            sentimentClassifier])



# Sample Data

data = spark.createDataFrame(

    [["The company reported a significant increase in revenue for the last quarter."]],

    ["text"]

)



# Fit-transform to get predictions

result = financial_sentiment_pipeline.fit(data).transform(data)

result.select("text","class_.result").show(truncate=False)

''', language='python')

st.text("""

+----------------------------------------------------------------------------+----------+

|text                                                                        |result    |

+----------------------------------------------------------------------------+----------+

|The company reported a significant increase in revenue for the last quarter.|[positive]|

+----------------------------------------------------------------------------+----------+

""")

st.markdown("""

<p>The code snippet demonstrates how to set up a pipeline in Spark NLP to perform financial sentiment analysis on text data using pre-trained models. The resulting DataFrame contains the sentiment predictions.</p>

""", unsafe_allow_html=True)

# Conclusion
st.markdown("""

<div class="section">

    <h2>Conclusion</h2>

    <p>In this app, we demonstrated how to use Spark NLP's pre-trained models to perform financial sentiment analysis on text data. By integrating these models into your NLP pipelines, you can gain valuable insights into market sentiment and make informed financial decisions.</p>

</div>

""", unsafe_allow_html=True)

# References and Additional Information
st.markdown('<div class="sub-title">For additional information, please check the following references.</div>', unsafe_allow_html=True)

st.markdown("""

<div class="section">

    <ul>

        <li>Documentation :&nbsp;<a class="link" href="https://nlp.johnsnowlabs.com/docs/en/transformers#sentiment" target="_blank" rel="noopener">Financial Sentiment Analysis</a></li>

        <li>Python Docs :&nbsp;<a class="link" href="https://nlp.johnsnowlabs.com/api/python/reference/autosummary/sparknlp/annotator/classifierdl/index.html#sparknlp.annotator.ClassifierDLModel" target="_blank" rel="noopener">ClassifierDLModel</a></li>

        <li>Scala Docs :&nbsp;<a class="link" href="https://nlp.johnsnowlabs.com/api/com/johnsnowlabs/nlp/annotators/classifierdl/index.html" target="_blank" rel="noopener">ClassifierDLModel</a></li>

        <li>Example Notebook :&nbsp;<a class="link" href="https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/streamlit_notebooks/SENTIMENT_EN_FINANCE.ipynb" target="_blank" rel="noopener">Financial Sentiment Analysis</a></li>

        <li>Reference Article :&nbsp;<a class="link" href="https://medium.com/spark-nlp/financial-sentiment-analysis-using-sparknlp-achieving-95-accuracy-e2df27744617" target="_blank" rel="noopener">Financial Sentiment Analysis Using SparkNLP Achieving 95% Accuracy</a></li>

    </ul>

</div>

""", unsafe_allow_html=True)

st.markdown('<div class="sub-title">Community & Support</div>', unsafe_allow_html=True)
st.markdown("""

<div class="section">

    <ul>

        <li><a class="link" href="https://sparknlp.org/" target="_blank">Official Website</a>: Documentation and examples</li>

        <li><a class="link" href="https://join.slack.com/t/spark-nlp/shared_invite/zt-198dipu77-L3UWNe_AJ8xqDk0ivmih5Q" target="_blank">Slack</a>: Live discussion with the community and team</li>

        <li><a class="link" href="https://github.com/JohnSnowLabs/spark-nlp" target="_blank">GitHub</a>: Bug reports, feature requests, and contributions</li>

        <li><a class="link" href="https://medium.com/spark-nlp" target="_blank">Medium</a>: Spark NLP articles</li>

        <li><a class="link" href="https://www.youtube.com/channel/UCmFOjlpYEhxf_wJUDuz6xxQ/videos" target="_blank">YouTube</a>: Video tutorials</li>

    </ul>

</div>

""", unsafe_allow_html=True)