import streamlit as st # Custom CSS for better styling st.markdown(""" """, unsafe_allow_html=True) # Introduction st.markdown('
Detecting Sarcasm with Spark NLP
', unsafe_allow_html=True) st.markdown("""

Welcome to the Spark NLP Sarcasm Detection Demo App! Detecting sarcasm in text is crucial for understanding sentiment and context. This app utilizes advanced natural language processing techniques to identify instances of sarcasm with high accuracy.

This demo showcases the use of Spark NLP's ClassifierDLModel pretrained on Universal Sentence Encoder embeddings to classify text as sarcastic or normal.

""", unsafe_allow_html=True) st.image('images/sarcasm.jpg', use_column_width='auto') # About Sarcasm Detection st.markdown('
About Sarcasm Detection
', unsafe_allow_html=True) st.markdown("""

Sarcasm detection involves identifying language that is contrary to the literal meaning, often used to convey humor or irony. It plays a crucial role in sentiment analysis and understanding textual context.

Effective sarcasm detection models improve the accuracy of sentiment analysis and help in better understanding user intent.

""", unsafe_allow_html=True) # Using ClassifierDLModel in Spark NLP st.markdown('
Using ClassifierDLModel in Spark NLP
', unsafe_allow_html=True) st.markdown("""

The ClassifierDLModel in Spark NLP utilizes deep learning techniques to classify text into predefined categories, in this case, detecting sarcasm or normal text. It is trained on Universal Sentence Encoder embeddings for robust performance.

For more details, refer to the ClassifierDLModel documentation on Spark NLP's official website.

""", unsafe_allow_html=True) st.markdown('

Example Usage in Python

', unsafe_allow_html=True) st.markdown('

Here’s how you can implement sarcasm detection using the ClassifierDLModel in Spark NLP:

', unsafe_allow_html=True) # Setup Instructions st.markdown('
Setup
', unsafe_allow_html=True) st.markdown('

To use Spark NLP for sarcasm detection, follow these setup instructions:

', unsafe_allow_html=True) st.code(""" pip install spark-nlp pip install pyspark """, language="bash") st.markdown("

Then, import Spark NLP and start a Spark session:

", unsafe_allow_html=True) st.code(""" import sparknlp # Start Spark Session spark = sparknlp.start() """, language='python') # Sentiment Analysis Example st.markdown('
Example Usage: Sarcasm Detection with ClassifierDLModel
', unsafe_allow_html=True) st.code(''' from sparknlp.base import DocumentAssembler from sparknlp.annotator import UniversalSentenceEncoder, ClassifierDLModel from pyspark.ml import Pipeline # Step 1: DocumentAssembler document_assembler = DocumentAssembler() \\ .setInputCol("text") \\ .setOutputCol("document") # Step 2: UniversalSentenceEncoder use = UniversalSentenceEncoder.pretrained() \\ .setInputCols(["document"]) \\ .setOutputCol("sentence_embeddings") # Step 3: ClassifierDLModel for Sarcasm Detection sentimentdl = ClassifierDLModel.pretrained('classifierdl_use_sarcasm') \\ .setInputCols(["sentence_embeddings"]) \\ .setOutputCol("sentiment") # Define the NLP Pipeline nlpPipeline = Pipeline(stages=[document_assembler, use, sentimentdl]) # Example Text text = "Oh, great! Another meeting scheduled for Friday afternoon. That's just what I needed." # Process the text through the pipeline result = nlpPipeline.fit(spark.createDataFrame([[text]]).toDF("text")).transform(spark.createDataFrame([[text]]).toDF("text")).select('text', 'sentiment.result').show(truncate=False) ''', language='python') st.text(""" +-------------------------------------------------------------------------------------+---------+ |text |result | +-------------------------------------------------------------------------------------+---------+ |Oh, great! Another meeting scheduled for Friday afternoon. That's just what I needed.|[sarcasm]| +-------------------------------------------------------------------------------------+---------+ """) st.markdown("""

The above example demonstrates how to use Spark NLP's ClassifierDLModel to detect sarcasm in text using Universal Sentence Encoder embeddings.

""", unsafe_allow_html=True) # Benchmarking st.markdown('
Benchmarking
', unsafe_allow_html=True) st.markdown("""

Performance metrics of the sarcasm detection model:

    precision    recall  f1-score   support

    normal       0.98      0.89      0.93       495
    sarcasm      0.60      0.91      0.73        93

    accuracy                           0.89       588
    macro avg      0.79      0.90      0.83       588
    weighted avg   0.92      0.89      0.90       588
""", unsafe_allow_html=True) # Conclusion st.markdown("""

Conclusion

In this app, we explored how Spark NLP's ClassifierDLModel can be used to detect sarcasm in text. This capability enhances sentiment analysis and contextual understanding in various applications, improving the accuracy of natural language processing tasks.

""", unsafe_allow_html=True) # References and Additional Information st.markdown('
For additional information, please check the following references.
', unsafe_allow_html=True) st.markdown("""
""", unsafe_allow_html=True) st.markdown('
Community & Support
', unsafe_allow_html=True) st.markdown("""
""", unsafe_allow_html=True)