Extractive document summarization (EDS) is usually seen as a sequence labeling task, which extracts sentences from a document one by one to form a summary. However, extracting sentences separately ignores the relationship between the sentences and documents. One solution is to use sentence position information to enhance sentence representation, but this will cause the sentence-leading bias problem, especially in news datasets. In this paper, we propose a novel sentence centrality for the EDS task to address these two problems. The sentence centrality is based on directed graphs, while reflecting the sentence-document relationship, it also reflects the sentence position information in the document. We implicitly strengthen the relevance of sentences and documents by using sentence centrality to enhance sentence representation. Notably, we replaced the sentence position information with sentence centrality to reduce sentence-leading bias without causing model performance degradation. Experiments on the CNN/Daily Mail dataset showed that EDS models with sentence centrality significantly improved compared with baseline models. Automatic document summarization aims to produce a concise summary of a document while preserving its crucial information. Existing summarization methods can be divided into two categories: abstractive and extractive methods. Abstractive methods generate a summary word by word from scratch, and can introduce new words that do not appear in the document . ExtractIn recent years, extractive document summarization (EDS) based on neural networks has achieved great success –6. HowevThere is much excellent work based on the first approach. For example, Zhang et al.  proposedThe sentence centrality is usually based on undirected graphs and is widely used in unsupervised extractive summarization tasks to identify salient sentences in a document , 14. In Previous work considered sentence centrality as a signal to measure the importance of sentences –15. DiffFollowing our intuition mentioned before, we can learn that the sentence centrality is no longer restricted in unsupervised extractive methods. We develop two methods to apply sentence centrality to enhance sentences representation: (1) embedding the sentence centrality directly into the sentence representation output by the encoder; (2) updating the sentence representation indirectly via Graph Attention Network (GAT) . We builThe contributions of our work are as follows:We propose a novel sentence centrality for EDS task and two approaches to use sentence centrality to enhance sentence representation. With the help of the sentence centrality, the relationship between sentences and documents is implicitly strengthened, thus improving the performance of the extractive summarization.We propose to replace sentence position information with sentence centrality, which can reduce the sentence-leading bias in the news dataset caused by position information.The remainder of this article is arranged as follows. We introduce some related topics on the EDS in the section Related Work. In the Method section, we define the EDS and then introduce our sentence centrality-enhanced extractive summarization models. We present the training details, parameter settings and experimental results in the Experiment section. In the Discussion section, we discuss why sentence centrality works. Finally, we conclude our paper in the Conclusion section.To make the paper self-contained, we will introduce some related topics on the EDS and the sentence centrality-based summarization methods.The EDS task aims to extract sentences from the original document to form a summary. The task first encodes the sentences with the help of an encoder to obtain a sentence vector. The sentence vector is then passed through a classification layer to determine whether it should be included in the summary. Nallapati et al ; Zhou etAlthough these methods are effective, they mostly rely on sentence position information to enhance sentence representation. We introduce sentence centrality information in the model and remove sentence position information, which improves model performance and does not cause sentence-leading bias.The sentence centrality is often used to measure the importance of a sentence in unsupervised EDS tasks. In the task, a document is represented as a graph, with nodes representing sentences and edges connecting sentences weighted according to their similarity. TextRank  calculatThere are three key differences in our sentence centrality compared to the previous methods. (1) We calculate the centrality of a sentence by counting only the similarity between that sentence and the content that follows it, not the similarity of all other content. (2) The centrality of a sentence is considered a unique property of the associated sentence and document rather than just as a measure of the importance of the sentence. Therefore, we use sentence centrality to strengthen the sentence representation. (3) We applied sentence centrality into the supervised EDS.An essential step in the extractive document summarization task is to obtain sentence embeddings. Traditional sentence embedding methods are based on weighting and averaging words vectors to construct sentences’ vectors. Kedzie et al.  averagedTraditional sentence embedding methods are simple and effective. However, extractive document summarization is a document-level task, and the relationship between sentences and documents needs to be considered when obtaining sentence embeddings. Most works , 20, 21 Different from their work, we use sentence center information to enhance sentence representations. Compared to using sentence position information, our methods are able to achieve performance improvements while reducing the sentence lead bias problem.d that contains n sentences, d = {s1, s2, …, sn}, where si = {wi1, wi2, …, wim} is the i-th sentence in the document and wij is the j-th word in the i-th sentence. EDS can be seen as a sequence labeling task 2] to enhHeterogeneous summarization graph (HSG)  is an exeij, which are mapped to the multi-dimensional embedding space. The weights of the edge eij are the sum of the sentence centrality and the TF-IDF value of the words, because the types of nodes connected by the edge are different. The modified GAT layer is designed as follows:hi is the hidden state of input node, αij refers to the weight of attention between hi and hj. The residual connection is used to avoid gradient vanishing after several iterations. The final sentence representation is:Our modified HSG model is presented in G with word features Xw and sentence node features Xs, the sentence nodes are updated with their neighbor word nodes via the above GAT and feed-forward (FFN) layer:Given a constructed graph We conduct our experiment on the CNN/Daily  Mail, XsCNN/Daily Mail is a well-known news dataset for single document extractive summarization, which is split into three parts by Hermann et al.  for traiXSum is a one-sentence summary dataset to answer the question “What is the article about?”. We conduct experiments on this dataset to study whether sentence centrality-enhanced EDS models are still effective when dealing with dataset with short summaries.We only use the XSum dataset for ablation experiments, as the extraction results on this dataset are few and are insufficient to support our model performance comparison.We limit the sentence length to 50 words calculating sentence centrality. Both models are trained on a single GPU (GeForce RTX 3080).https://github.com/huggingface/pytorch-pretrained-BERT. The model is trained for 40000 steps. The best result on the validation set occurs at step 37000. Adam algorithm is applied for optimizing the loss function. Learning rate schedule is following Vaswani et al. Reviewers' comments:Reviewer's Responses to Questions  Comments to the Author1. Is the manuscript technically sound, and do the data support the conclusions?  The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: YesReviewer #2: Partly**********  2. Has the statistical analysis been performed appropriately and rigorously?   Reviewer #1: N/AReviewer #2: Yes**********  3. Have the authors made all data underlying the findings in their manuscript fully available?PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.  The Reviewer #1: YesReviewer #2: Yes**********  4. Is the manuscript presented in an intelligible fashion and written in standard English?  PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.Reviewer #1: YesReviewer #2: Yes**********  5. Review Comments to the Author  Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. Reviewer #1: The manuscript “Improving Extractive Document Summarization with Sentence Centrality” highlights a way to enhance sentence representation for extractive document summarization which in turns boost performance of existing EDS techniques. The paper is well written and contributions are clearly laid out and explained, used methodology is sound. Still some issues are cracking through. The following is a list of items which should be addressed:1. In the introduction, the manuscript references findings of Zheng and Lapata that “similarity with the previous content will damage centrality”. In the method proposed here, forward edges are removed and only backward edges are considered, but this seems confusing. Why is this the right approach if this tends to damage centrality score?2. Acronym EDS in line 10 is not defined – please add full text of the meaning.3. The structure of the paper is missing at the end of Introduction.4. While being sufficient example of a graph, figure 1 does not convey idea of degree centrality in a meaningful way. Maybe node size or colour should be varied based on centrality score to reinforce the idea that some sentences are more important than the other.5. In the approach involving heterogeneous graph neural network, edge features were extended with sentence centrality. If sentence centrality is a node characteristic, why is it used to enhance edge features?6. In the equation 6, indices in LHS appear to be duplicated. Equation 7 contains similar problem. Although it may be obvious to knowledgeable reader, terms of the equation 11 should still be defined for rigour, and it should be done for all other equations as well. For example, in equation (1) hi and hi+ are not defined.7. It is not quite clear how is centrality embedding (EmbSCi) is obtained. Specifically, what are the terms on the RHS of the equation 9? If SCi is centrality of sentence i, what is the meaning of the exponents 1 to emb? Furthermore, equation 9 defines EmbSCi as a set of terms SCi, but this seems unusual w.r.t embeddings usually being vectors.8. Please define HSG in line 168.9. Please add short explanation of Trigram Blocking.10. How does table 3 relate to table 1 and table 2? Are these various configurations of input data to SCBERT and SCHSG models from previous tables? If it is indeed so, please make sure it is more clear from the text to prevent any misunderstandings.11. Please cite everything which is not original work presented in the paper, e.g. ROUGE, bert-base-uncased model, and some other instances.12. Please in the appendix define used ROUGE scores.13. “Analysis” section contains reference to “ORACLE summary”, please elaborate some more on what the ORACLE is, how does it work, and cite the relevant paper.14. Consider renaming Analysis Chapter into Discussion and expand it.15. Axes on figure 3 should be appropriately labelled to make the figure stands on it’s own.16. It would be useful to know how the proposed model performs with other similar node-local measures such as selectivity measure. It might be useful as a basis for future work.17. Introduction contains abbreviation “EDT” which is never elaborated or mentioned again. This seems like a very minor typographical error, and presumably was meant to say "EDS". In the same vein, “Ablation Study” section contains sentence “The results show that the experimental performance on ROUGE-L, ROUGE-2, ROUGE-L”. Is the first ROUGE-L in line 244 ROUGE-1?18. On several places there is a blank after comma missing.Reviewer #2: ### Overview and general recommendationThe paper addresses the problem of extractive document summarization. It uses sentence centrality information to enhance sentence representation. This information should reflect the sentence-document relationship and the sentence position information as well. The sentence representation is enhanced in two ways: one is embedded directly into the sentence representation output and the other updates the sentence representation indirectly via a graph attention network.Because of advances in abstractive summarization, the task of extractive document summarization is probably no longer very challenging or interesting for the research field. The paper provides a good method for comparing the impact of sentence position vs. sentence centrality. However, one of the conclusions is not well supported by the experimental results, but overall, this is a very well-written paper with sound methodology, ablations studies, and a well-formulated research question.### Major comments- According to Table 3, the results of different sentence information don't support the conclusion that sentence centrality is a better choice than sentence position. Given that ROUGE is a poor metric, the difference of approximately 0.1 is insignificant.- More ideas for future research could be added to the conclusion.- The related research section could be expanded as well, e.g., adding a section on sentence embeddings, since the paper works on that level of representation.- Some typos should be fixed, and some sentences could be rewritten to make them more clear; Take a look at minor remarks- There is no link to the code repository.### Minor comments- Typo in the caption of Fig. 1, “documentt”- Line 176: “using” → “use”- Table 3: “summarizers” → “summarizer”- Fig. 2: the first sentence should be corrected- Line 244: the first “ROUGE-L” should be “ROUGE-1”**********  6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.If you choose “no”, your identity will remain anonymous but your review may still be made public.Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.  Reviewer #1: NoReviewer #2: Nohttps://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool,  18 Apr 2022We greatly appreciate the reviewers taking the time to provide constructive comments and helpful suggestions. There is no doubt that the suggestions have significantly raised the quality if the manuscript and have enable us to improve the manuscript. Each suggested revision and comment brought forward by the reviewers was accurately incorporated and considered. We have carefully addressed all the reviewer's concerns. Please see our replies. Changes highlighted in red have been made accordingly in the revised manuscript.Comments to the Author________________________________________1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: PartlyResponse: Thanks for your review. We added experiments to verify the effectiveness of our method. We conducted experiments on XSum dataset. The results demonstrate the superiority of sentence centrality compared to positional information.________________________________________2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: N/A Reviewer #2: Yes Response: Thank you for agreeing that our analysis is appropriate and rigorous.________________________________________3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes Response: Thank you for your approval of our data.________________________________________4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: YesResponse: Thank you for your approval.________________________________________Review Comments to the Author________________________________________Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics.  Reviewer #1: The manuscript “Improving Extractive Document Summarization with Sentence Centrality” highlights a way to enhance sentence representation for extractive document summarization which in turns boost performance of existing EDS techniques. The paper is well written and contributions are clearly laid out and explained, used methodology is sound. Still some issues are cracking through. The following is a list of items which should be addressed: 1. In the introduction, the manuscript references findings of Zheng and Lapata that “similarity with the previous content will damage centrality”. In the method proposed here, forward edges are removed and only backward edges are considered, but this seems confusing. Why is this the right approach if this tends to damage centrality score?Response: We appreciate the reviewer for asking questions about the details of sentence centrality. According to the original paper of Zheng and Lapata, the centrality score of s_i based on the directed graph can be defined as follows:centrality(s_i )=λ_1 ∑_(ji)e_ij ,where the optimal λ_1 tends to be negative.In our paper, we do not calculate the similarity between the sentence and its previous content of the sentence, which means that we set the weights of forward-looking directed edges λ_1 are equal to 0. We think the descriptions of “forward-looking” and “forward” make our point unclear, so we revised this part of content to convey our idea clearly, in lines 43-45. The description of “Inspired by their work, we remove the forward edges of sentences on directed graphs and calculate the sentence centrality based only on the weights of backward edges” is modified by us to “Inspired by their work, we calculate the centrality score of a sentence based only on the similarity between the sentence and its following content”.________________________________________2. Acronym EDS in line 10 is not defined – please add full text of the meaning.Response: We appreciate the reviewer for his/her careful review. We have added the full text of the meaning of EDS in line 10.________________________________________3. The structure of the paper is missing at the end of Introduction.Response: We thank the reviewer for reminding us to describe the structure of our paper, and there is no doubt that this suggestion makes the paper more readable. We have added a description of the paper structure in lines 76-81.________________________________________4. While being sufficient example of a graph, figure 1 does not convey idea of degree centrality in a meaningful way. Maybe node size or colour should be varied based on centrality score to reinforce the idea that some sentences are more important than the other.Response: We thank the reviewer for the helpful suggestions on our figure 1. We now updated the figure 1. we used different colors to indicate different sentences, and used the size of the nodes to indicate the centrality scores. We also increased the number of sentence nodes in order to convey idea our ideas more clearly. ________________________________________5. In the approach involving heterogeneous graph neural network, edge features were extended with sentence centrality. If sentence centrality is a node characteristic, why is it used to enhance edge features?Response: We are grateful to the reviewer for questions about our method. We use sentence centrality to enhance edge features is to modify the graph attention (GAT) layer. In the heterogeneous graph neural network, the sentence representations are updated with their neighbor word nodes via a GAT layer and feed-forward (FFN) layer. The GAT layer is modified by infusing the scalar edge weights e_ij (described in equation 13), which are mapped to the multi-dimensional embedding space. The weights of the edge e_ij are the sum of the sentence centrality and the TF-IDF value of the words, because the types of nodes connected by the edge are different.We added equations and a textual explanation to describe how sentence representations are updated by GAT and FFN in equation 17, equation 18 and lines 228-236. We also do a textual explanation of why we use sentence centrality to enhance edge features in lines 216-221.________________________________________6. In the equation 6, indices in LHS appear to be duplicated. Equation 7 contains similar problem. Although it may be obvious to knowledgeable reader, terms of the equation 11 should still be defined for rigour, and it should be done for all other equations as well. For example, in equation (1) hi and hi+ are not defined.Response: We appreciate the reviewer for his/her careful review and we feel sorry for our lack of rigour. We have corrected equation 6 and equation 7. For terms in the equations that are not strictly defined, we have carefully checked.For equation 1, we defined h_i and h_i^+, and explained the meaning of sim in lines 150-152.For equation 6, we explained the meaning of the term u_ij in line 184.For equation 9 and equation 10, we modified these two equations to make the meaning of 〖EmbSC〗_i clearer. We defined each term in lines 187-192.For equation 11, we define each term of the equation and explain the meaning in lines198-200. ________________________________________7. It is not quite clear how is centrality embedding (EmbSCi) is obtained. Specifically, what are the terms on the RHS of the equation 9? If SCi is centrality of sentence i, what is the meaning of the exponents 1 to emb? Furthermore, equation 9 defines EmbSCi as a set of terms SCi, but this seems unusual w.r.t embeddings usually being vectors.Response: We appreciate the reviewer for his/her careful review. Our definition of equation 9 is not rigorous, leaving our point unclear. We feel sorry for this. We modified the equation 9. Now the equation 9 is:EmbSC_i=W_sc (SC_i ) ,where W_sc is a weight matrix with the weights set to 1. EmbSC_i is the centrality embedding of sentence s_i, which has the same dimension as the sentence embedding.EmbSC_i is obtained by mapping the normalized scalar sentence centrality to the multi-dimensional embedding space. The RHS of the new equation 9 W_sc (SC_i ) means that (SC_i ) is mapped to a higher dimensional space by W_sc. The RHS of our previous equation 9 was trying to convey the same meaning as the new equation 9, but we are sorry that we did not make it clearer. The exponents 1 to emb means that we map sentence centrality to the emb-dimensional space in the previous equation 9. ________________________________________8. Please define HSG in line 168.Response: We thank the reviewer for reminding us to define HSG. We add full text of the meaning HSG in lines 202-204, and give our sentence centrality-enhanced extractive document summarization model based on HSG in Fig.3.________________________________________9. Please add short explanation of Trigram Blocking.Response: We thank the reviewer for reminding us to add short explanation of Trigram Blocking. We added the short explanation of Trigram Blocking in lines 270-273. ________________________________________10. How does table 3 relate to table 1 and table 2? Are these various configurations of input data to SCBERT and SCHSG models from previous tables? If it is indeed so, please make sure it is clearer from the text to prevent any misunderstandings.Response: We thank the reviewer for his/her careful review. All the models in Table 3 are based on the pre-trained language model BERT, where the SCES model is exactly our SCBERT in Table 1. The various configurations of experimental parameters for the SCES model are the same as for the SCBERT model, except that the datasets used are different. We added the relevant description in lines 324-327.________________________________________11. Please cite everything which is not original work presented in the paper, e.g. ROUGE, bert-base-uncased model, and some other instances.Response: We thank the reviewer for his/her careful review. We checked our paper carefully and added citations to the work that needed to be cited.Line 17, we added a reference to transformer.Line 213, we added a reference to Convolutional Neural Network. Line 216, we added a reference to Bidirectional Long and Short-Term Memory.Line 239, we added references to CNN/Daily Mail and XSum datasets.Line 297, we added references to ROUGE.https://github.com/huggingface/pytorch-pretrained-BERT, we put the URL in lines 256-257.“bert-base-uncased model” is published in ________________________________________12. Please in the appendix define used ROUGE scores.Response: We thank the reviewer for reminding us to add the definition of ROUGE scores. We defined the used ROUGE scores in S1 Appendix.________________________________________13. “Analysis” section contains reference to “ORACLE summary”, please elaborate some more on what the ORACLE is, how does it work, and cite the relevant paper.Response: We thank the reviewer for his/her rigorous review. We elaborated the ORACLE summary in lines 353-359, including that what the ORACLE is, how does it work. We also cite the relevant paper.s________________________________________14. Consider renaming Analysis Chapter into Discussion and expand it.Response: We thank the reviewers for the suggestions on the structure of our article.We have renamed Analysis Chapter into Discussion. In this part, we added the content of ORACLE according to your comment 13. We discuss why sentence centrality is a better choice than sentence position information.________________________________________15. Axes on figure 3 should be appropriately labelled to make the figure stands on it’s own.Response: We thank the reviewer for his/her careful review. Since we added a heterogeneous graph model graph, the original figure 3 is now figure 4. The axes on figure 4 are now appropriately labeled.________________________________________16. It would be useful to know how the proposed model performs with other similar node-local measures such as selectivity measure. It might be useful as a basis for future work.Response: We thank the reviewer for his/her constructive suggestion. This suggestion makes us realize that our exploration of sentence centrality needs to go further. We have written this suggestion into future work. Thanks again for the constructive suggestion.________________________________________17. Introduction contains abbreviation “EDT” which is never elaborated or mentioned again. This seems like a very minor typographical error, and presumably was meant to say "EDS". In the same vein, “Ablation Study” section contains sentence “The results show that the experimental performance on ROUGE-L, ROUGE-2, ROUGE-L”. Is the first ROUGE-L in line 244 ROUGE-1?Response: We appreciate the reviewer for his/her careful review. “EDT” is a typographical error; we have corrected it in line 41. The first “ROUGE-L” is corrected to “ROUGE-1” in line 306.________________________________________18. On several places there is a blank after comma missing.Response: We appreciate the reviewer for his/her careful review. We checked our paper carefully and added a blank for commas.We again thank the reviewer for taking the time to review our article. From the review comments, we can feel the rigorous attitude of the reviewers to academics. There is no doubt that the comments of the reviewers make our manuscript more rigorous and clearer. ________________________________________Reviewer #2: ### Overview and general recommendation The paper addresses the problem of extractive document summarization. It uses sentence centrality information to enhance sentence representation. This information should reflect the sentence-document relationship and the sentence position information as well. The sentence representation is enhanced in two ways: one is embedded directly into the sentence representation output and the other updates the sentence representation indirectly via a graph attention network. Because of advances in abstractive summarization, the task of extractive document summarization is probably no longer very challenging or interesting for the research field. The paper provides a good method for comparing the impact of sentence position vs. sentence centrality. However, one of the conclusions is not well supported by the experimental results, but overall, this is a very well-written paper with sound methodology, ablations studies, and a well-formulated research question. ### Major comments - According to Table 3, the results of different sentence information don't support the conclusion that sentence centrality is a better choice than sentence position. Given that ROUGE is a poor metric, the difference of approximately 0.1 is insignificant.Response: We thank the reviewer for pointing out potential limitation in our study. We agree with the reviewer that the superiority of sentence centrality cannot be demonstrated only from the ROUGE scores.In the extractive summarization task, position information is usually used to enhance sentence representation. Although doing so will improve model extraction performance significantly, it will cause sentence-leading bias, especially in news datasets. We present this phenomenon in lines 23-30.we replaced the sentence position information with sentence centrality to reduce sentence-leading bias without causing model performance degradation, which can be seen in the figure 5 and table 3. We added experiments in the news dataset XSum. The experimental results show that the replacement of sentence position information by sentence centrality will not cause model performance degradation.Before reaching the conclusion "sentence centrality is a better choice than sentence position", we added a description of the advantages of sentence centrality in reducing sentence leading bias, discussed in the Discussion section.Combining the advantages of sentence centrality in reducing sentence leading bias and experimental results in table 3, we can conclude that sentence centrality information has certain advantages over sentence position information the extractive summarization task.We are grateful to the reviewer for his/her constructive comments, which made our logic more rigorous and greatly improved the quality of our paper. ________________________________________- More ideas for future research could be added to the conclusion.Response: We thank the reviewer for reminding us to expand our future work. We add more ideas in future research, including exploring whether sentence centrality is also effective in other tasks, etc., which are presented in lines 374-381. ________________________________________- The related research section could be expanded as well, e.g., adding a section on sentence embeddings, since the paper works on that level of representation. Response: We thank the reviewer for his/her constructive suggestion. Adding a section on sentence embeddings will make our article more rigorous. We have expanded our related work in lines 113-131.________________________________________- Some typos should be fixed, and some sentences could be rewritten to make them. Take a look at minor remarksResponse: We thank the reviewer for his/her careful review. We have carefully checked our article and corrected errors. We present the modification details in the ### Minor comments section.________________________________________- There is no link to the code repository.Response: We are pleased that the reviewer is interested in our work.https://github.com/GongShuai8210/SCES.The code is released at ________________________________________### Minor comments - Typo in the caption of Fig. 1, “documentt” - Line 176: “using” → “use” - Table 3: “summarizers” → “summarizer” - Fig. 2: the first sentence should be corrected - Line 244: the first “ROUGE-L” should be “ROUGE-1”Response: We thank the reviewer for taking the time to review our article carefully. We have corrected the typo now.-Typo in the caption of Fig. 1, “documentt” → “document”- Line 213: “using” has been corrected to “use” - Table 3: “summarizers” has been corrected to “summarizer” - Fig. 2: the first sentence has been corrected to “EmbSC_i is the centrality embedding of sentence s_i, which is directly embedded in the sentence representation generated by BERT”. - Line 306: the first “ROUGE-L” has been corrected to “ROUGE-1”.We again thank the reviewer for his/her careful review and constructive suggestions. There is no doubt that the reviewer's suggestions improve the quality of our article.AttachmentResponse to Reviewers.pdfSubmitted filename: Click here for additional data file. 27 Apr 2022Improving Extractive Document Summarization with Sentence CentralityPONE-D-22-05139R1Dear Dr. Gong,We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at onepress@plos.org.If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact Kind regards,Sanda Martinčić-Ipšić, PhDAcademic EditorPLOS ONEAdditional Editor Comments :I have reviewed the revised manuscript, responses to reviewer comments, and availability of data and SW. The authors have addressed all reviewer comments and improved the quality of the manuscript. I am pleased to report that the current manuscript revision adequately addresses all issues and meets the required PlosOnNE criteria.Reviewers' comments: 14 Jul 2022PONE-D-22-05139R1 Improving Extractive Document Summarization with Sentence Centrality Dear Dr. Zhu:I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. onepress@plos.org.If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact plosone@plos.org. If we can help with anything else, please email us at Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staffon behalf ofDr. Sanda Martinčić-Ipšić Academic EditorPLOS ONE


We report the coding-complete genome sequences of 25 severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) sublineage B.1.1.529 Omicron strains obtained from Bangladeshi individuals in samples collected between December 2021 and January 2022. Genomic data were generated by Nanopore sequencing using the amplicon sequencing approach developed by the ARTIC Network. Coronaviridae, genus Betacoronavirus) is a positive-sense single-stranded RNA virus (Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) (family NA virus . The mosNA virus .As part of the ongoing SARS-CoV-2 genomic surveillance (protocol IEDCR/IRB/2020/11) by the Institute of Epidemiology, Disease Control, and Research (IEDCR), Bangladesh, two specimens were obtained from individuals who had recently visited Africa and had reported coronavirus disease 2019 (COVID-19) symptoms on return to Bangladesh. These specimens were found to be positive for the SARS-CoV-2 nucleocapsid (N) gene but negative for the S gene by TaqPath COVID-19 Combo reverse transcription (RT)-PCR . Seventeen more SARS-CoV-2-positive specimens from the countrywide SARS-CoV-2 surveillance showed similar results. These specimens were further screened with the TaqMan SARS-CoV-2 mutation panel (Applied Biosystems), which indicated the presence of S:N501Y, one of the signature mutations of the SARS-CoV-2 Omicron variant. This S:N501Y mutation is not present in the S gene of the Delta variant, which was the predominant strain circulating in Bangladesh in the last few months of 2021. Overall, a total of 25 specimens were used as input for genome sequencing using the Oxford Nanopore Technologies sequencing platform. The detailed information for all 25 individuals is presented in https://artic.network/ncov-2019/ncov2019-bioinformatics-sop.html). In total, 4,324,431 reads were obtained . Compared to the reference Wuhan Hu-1 genome (GenBank accession number NC_045512.2), the signature amino acid alterations in the spike protein matching the genetic markers of sublineages B1.1.529.1 and B1.1.529.2 were identified. Among the 25 sequences, Pangolin (github.com/cov-lineages/pangolin) assigned 19 sequences to lineage B.1.1.529.1 (BA.1), and six strains were found to be lineage B.1.1.529.2 (BA.2) . Sequencing libraries were prepared using the multiplex PCR amplicon sequencing approach developed by the ARTIC Network , 5. Libr2 (BA.2) . These sThe data from this study can be found under GISAID accession numbers EPI_ISL_7404462, EPI_ISL_7404463, EPI_ISL_8146774, EPI_ISL_8414987, EPI_ISL_8146772, EPI_ISL_8146773, EPI_ISL_8414988, EPI_ISL_8096971, EPI_ISL_8414989, EPI_ISL_8414990, EPI_ISL_8215676, EPI_ISL_8215677, EPI_ISL_8415001, EPI_ISL_8215678, EPI_ISL_8414993, EPI_ISL_8415003, EPI_ISL_8415004, EPI_ISL_8414994, EPI_ISL_8414995, EPI_ISL_9456595, EPI_ISL_9456604, EPI_ISL_9456606, EPI_ISL_9456607, EPI_ISL_9456620, and EPI_ISL_9456621. The Sequence Read Archive (SRA) and GenBank accession numbers are listed in


Background: To elucidate the potential biological function of hsa_circ_0062270 in the malignant process of melanoma and its potential target.Methods: Quantitative real-time polymerase chain reaction (qRT-PCR) was conducted to examine relative level of hsa_circ_0062270 in melanoma tissues and normal skin tissues. The diagnostic and prognostic potentials of hsa_circ_0062270 in melanoma were evaluated. The regulatory effect of hsa_circ_0062270 on the expression of linear transcript Cell division cycle protein 45 (CDC45) was also examined.Results: Hsa_circ_0062270 was up-regulated in melanoma samples and cell lines, which displayed certain diagnostic and prognostic potentials in melanoma. Inhibition of hsa_circ_0062270 attenuated the proliferative, migratory and invasive functions. Hsa_circ_0062270 could stabilize the expression of linear transcript CDC45, and thus participated in the malignant process of melanoma.Conclusion: Hsa_circ_0062270 promotes proliferative, migratory and invasive functions of melanoma cells via stabilizing the linear transcript CDC45. Hsa_circ_0062270 can be used to diagnosis and treatment of melanoma. Melanoma is a highly malignant skin cancer derived from melanocytes. Melanoma accounts for more than 70% of skin cancer deaths . More seCircRNAs are novel noncoding RNAs to be widely analyzed. They are extensively involved in various fields of life sciences . Since cHsa_circ_0062270 is located on chromosome 22: 19496052-19502571, and its gene symbol is CDC45. Evidence has showed that hsa_circ_0062270 is obviously up-regulated in melanoma . A previThe normal skin tissues and melanoma tissues of 50 patients with melanoma in our hospital were selected. The ethics committee of The First People’s Hospital of Lianyungang approved our study. Signed written informed consents were obtained from all participants before the study.2, 37°C. Cell transfection was performed using Lipofectamine 3,000 as per the protocols. Cell proliferation was determined by EdU .Melanoma cells  and normal human epidermal melanocytes (NHEM) were provided by Cell Bank of Type Culture Collection . Cells were cultivated in DMEM containing 10% fetal bovine serum (FBS), 100 U/mL penicillin and 100 μg/ml streptomycin at 5% CO−ΔΔCt. Primers used were shown below: hsa_circ_0062270: Forward: 5′-AGGATGGCTCAGGGACAGAT-3′, reverse: 5′-AGGCCATGGTACAGCTTGTC-3′; CDC45: Forward: 5′-TTCGTGTCCGATTTCCGCAAA-3′, reverse: 5′-TGGAACCAGCGTATATTGCAC-3′; GAPDH: Forward: 5′-CGGAGTCAACGGATTTGGTCGTAT-3′, reverse: 5′-AGCCTTCTCCATGGTGGTGAAGAC-3′: U6: Forward: 5′-GCTGAGGTGACGGTCTCAAA-3′, reverse: 5′-GCCTCCCAGTTTCATGGACA-3′.RNAs isolation was done with TRIzol and were then reversely transcribed into cDNAs. U6 and GAPDH were used as the internal controls with the method of 2A375 cells were exposed to Actinomycin D (3 μg/ml). They were collected for isolating total RNAs. Expressions of hsa_circ_0062270 and CDC45 were detected by Quantitative real-time polymerase chain reaction. Cellular RNA (4 mg) was treated either with RNase R (10 U/μg) at 37°C for 30 min or not, followed by purification using RNeasy MinElute .Cells were seeded into the top chamber and bottom chamber. After 48-h incubation, cells in the bottom were fixed, dyed in crystal violet and captured. Migratory cells were counted in five randomly selected fields per sample. Invasion assay was conducted using transwell chamber precoated with 100 μL of Matrigel . In detail, Matrigel was diluted in serum-free medium at 1:3, which was coated on the top of a chamber.p < 0.05.Data were expressed as mean ± SD (standard deviation) and they were processed using Statistical Product and Service Solutions (SPSS) 20.0 . Prognostic value of hsa_circ_0062270 in melanoma were evaluated by Kaplan-Meier and receiver operating characteristic (ROC) method, respectively. The correlation between hsa_circ_0062270 and CDC45 levels was assessed through Pearson correlation test. A significant difference was set at p = 0.0234)  . Hsa_cir 0.0234) . We furt 0.0234) . RNase R 0.0234) . TherefoTo investigate the effects of hsa_circ_0062270 on melanoma cell proliferation, migration and invasion, cells were treated with hsa_circ_0062270 siRNA. Results indicated that transfection of hsa_circ_0062270 siRNA markedly down-regulated hsa_circ_0062270 level in A375 and A2058 cells . Knockdovia mediating expression levels of their linear transcripts. CDC45 was the linear transcript of hsa_circ_0062270 (Then we focused on the potential target of hsa_circ_0062270 in the regulation of phenotypes of melanoma. CircRNAs are involved in pathological process _0062270  and posi_0062270 . Identic_0062270 . Knockdo_0062270 . It is cTo further elucidate the effects of CDC45 on melanoma phenotypes, we established the overexpression models of CDC45. Transfection of overexpressed plasmid of CDC45 effectively up-regulated CDC45 in A375 and A2058 cells . In melaTo uncover the co-regulation of hsa_circ_0062270 and CDC45, cells were co-transfected using si-CDC45 and overexpressed plasmid of hsa_circ_0062270 . OverexpThe incidence of melanoma is not high, covering 4–5% of malignant tumors. Family history, multiple atypical moles and dysplastic moles are risk factors that trigger the carcinogenesis of melanoma . In addiCircRNAs, as a type of emerging noncoding RNAs, have been well concerned because of their unique structure and vital functions . They caIn vitro experiment results illustrated that knockdown of hsa_circ_0062270 remarkably suppressed proliferative, migratory and invasive functions of melanoma cells.We previously found that the expression of hsa_circ_0062270 in melanoma was up-regulated. The diagnostic and prognostic potentials of hsa_circ_0062270 in melanoma were verified through depicting ROC and Kaplan-Meier curves, respectively. Recent studies have demonstrated that circRNAs have an important role in disease progression by mediating expressions of their linear transcripts . Throughin vivo role of hsa_circ_0062270 in melanoma is not explored. Secondly, how hsa_circ_0062270 regulates CDC45 remains unclear. Thirdly, other cell phenotypes of melanoma, including apoptosis, epithelial-mesenchymal transition and cell cycle progression affected by hsa_circ_0062270 are not clear.Collectively, our present study was the first attempt to reveal that hsa_circ_0062270 was up-regulated in melanoma specimens and correlated to its prognosis. Hsa_circ_0062270 stimulated malignant process of melanoma by stabilizing its linear transcript CDC45. Our findings provide a new aspect for developing diagnostic and therapeutic strategies for melanoma. Several limitations of our study should be pointed out. First of all, via stabilizing the linear transcript CDC45. These findings provided strong evidence that hsa_circ_0062270 could be a novel promising therapeutic target used to diagnosis and treatment of melanoma.Hsa_circ_0062270 promotes proliferative, migrative and invasive functions in melanoma cells


Structural annotation of genetic variants in the context of intermolecular interactions and protein stability can shed light onto mechanisms of disease-related phenotypes. Three-dimensional structures of related proteins in complexes with other proteins, nucleic acids, or ligands enrich such functional interpretation, since intermolecular interactions are well conserved in evolution.We present d-StructMAn, a novel computational method that enables structural annotation of local genetic variants, such as single-nucleotide variants and in-frame indels, and implements it in a highly efficient and user-friendly tool provided as a Docker container. Using d-StructMAn, we annotated several very large sets of human genetic variants, including all variants from ClinVar and all amino acid positions in the human proteome. We were able to provide annotation for more than 46% of positions in the human proteome representing over 60% proteins.d-StructMAn is the first of its kind and a highly efficient tool for structural annotation of protein-coding genetic variation in the context of observed and potential intermolecular interactions. d-StructMAn is readily applicable to proteome-scale datasets and can be an instrumental building machine-learning tool for predicting genotype-to-phenotype relationships. Key PointsA novel bioinformatics tool for structural characterization of genetic variants is presented.Single-nucleotide variants and indels are described with respect to intermolecular interactions in homologous protein complexes.An efficient implementation using a Docker container allows for analysis of large whole proteome-scale datasets.In the age of next-generation sequencing, large-scale genetic diversity within populations became apparent. A single human individual of European ancestry carries around 3 million genetic variants , of whicAlthough most of sequence variants have no functional or pathogenic effect, in some cases, even a single mutation can be disease causing , 6. ExpeOf those tools that employ protein 3D structure to derive various predictive features, several put variants of interest in the context of protein–protein interactions  and inteAs far as structural annotation of larger variants is concerned, most methods are developed to annotate single amino acid replacements, and to the best of our knowledge, there is currently no structural annotation pipeline that is able to map indel-type genetic variants to protein 3D structures.In this study, we present d-StructMAn, a new improved implementation of our earlier tool StructMAn , shippedhttps://www.uniprot.org/uniprot/?query=human&fil=proteome%3AUP000005640+AND+organism%3A%22Homo+sapiens+%28Human%29+%5B9606%5D%22&sort=score# ). S.K. was supported by the IMPRS-CS graduate student fellowship and DFG project number 430158625. S.K.S. was partially supported by the UdS-HIPS-Tandem Interdisciplinary Graduate School for Drug Research. O.V.K. was supported by the Klaus Faber Foundation.A.G. devised the method and implemented the core functionality. S.K.S. and S.K. assisted with the method development and implemented the container. V.R. and O.V.K. conceived the project. All authors wrote the manuscript.We would like to thank Anne Tolkmitt, Nadezhda Azbukina, Amit Fenn and Olga Tsoy for testing d-StructMAn.giac086_GIGA-D-22-00032_Original_SubmissionClick here for additional data file.giac086_GIGA-D-22-00032_Revision_1Click here for additional data file.giac086_GIGA-D-22-00032_Revision_2Click here for additional data file.giac086_Response_to_Reviewer_Comments_Original_SubmissionClick here for additional data file.giac086_Response_to_Reviewer_Comments_Revision_1Click here for additional data file.giac086_Reviewer_1_Report_Original_SubmissionRoman Laskowski, PhD -- 4/4/2022 ReviewedClick here for additional data file.giac086_Reviewer_1_Report_Revision_1Roman Laskowski, PhD -- 7/14/2022 ReviewedClick here for additional data file.giac086_Reviewer_2_Report_Original_SubmissionEduard Porta Pardo -- 4/5/2022 ReviewedClick here for additional data file.giac086_Reviewer_2_Report_Revision_1Eduard Porta Pardo -- 8/1/2022 ReviewedClick here for additional data file.giac086_Supplemental_FilesClick here for additional data file.


The purpose of this research is to emphasize the importance of mental health and contribute to the overall well-being of humankind by detecting stress. Stress is a state of strain, whether it be mental or physical. It can result from anything that frustrates, incenses, or unnerves you in an event or thinking. Your body’s response to a demand or challenge is stress. Stress affects people on a daily basis. Stress can be regarded as a hidden pandemic. Long-term (chronic) stress results in ongoing activation of the stress response, which wears down the body over time. Symptoms manifest as behavioral, emotional, and physical effects. The most common method involves administering brief self-report questionnaires such as the Perceived Stress Scale. However, self-report questionnaires frequently lack item specificity and validity, and interview-based measures can be time- and money-consuming. In this research, a novel method used to detect human mental stress by processing audio-visual data is proposed. In this paper, the focus is on understanding the use of audio-visual stress identification. Using the cascaded RNN-LSTM strategy, we achieved 91% accuracy on the RAVDESS dataset, classifying eight emotions and eventually stressed and unstressed states. Today, 82 percent of Indians are stressed, as per the Cigna 360 Well-being study . Rising Stress is a condition of mental pressure for individuals facing problems relating to environmental and social well-being which leads to many diseases. It was discovered that academic exams, human relationships, interpersonal difficulties, life transitions, and career choices all contribute to stress. Such stress is commonly associated with psychological, physical, and behavioral issues .According to Lazarus and Folkman (1984), “stress is a mental or physical phenomenon formed through one’s cognitive appraisal of the stimulation and is a result of one’s interaction with the environment”. The existence of stress depends on the existence of the stressor. Feng (1992) and Volpe (2000) defined a stressor as “anything that challenges an individual’s adaptability or stimulates an individual’s body or mentality”. Stress can be caused by environmental factors, psychological factors, biological factors, and social factors, as shown in Human stress represents an imbalanced state  of an inEmotions are present in almost every decision and moment of our lives. Thus, recognizing emotions awakens interest, since knowing what others feel helps us to interact with them more effectively. Emotions are considered a psychological state . In the It must be considered that emotions are subjective to an individual, i.e., each subject may experience a different emotion in response to the same stimuli. Thus, emotions can be classified into two different models—the discrete model and the dimensional model. The discrete model includes basic emotions such as happiness, sadness, fear, disgust, anger, surprise, and mixed emotions such as motivation , self-awareness , etc. The dimensional model is expressed in terms of two emotions, valence  and arousal . The various emotions experienced by a human can be represented through the Plutchik wheel of emotion , as showSeveral researchers have analyzed human stress using basic emotions. It is possible to map emotions with the stress level. Stress can be detected based on emotions obtained from the audio-visual data. Human emotions are expressed in the voice as well as on the face. The emotional state is extracted from the audio-visual data first. Positive emotions such as happiness, joy, love, pride, and pleasure can have a positive effect, such as improving daily work performance, and negative emotions such as anger, terrible, sad and, disgust can have a negative impact on the health of a person. Positive and negative emotions are represented in The valence–arousal space, as illustrated in Our model reached an accuracy of 91% on the ‘The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS), outperforming some of the previous solutions evaluated in similar conditions. As far as we know, our study also represents the first attempt to combine speech and facial expressions to recognize the eight emotions in RAVDESS and finally conclude on a stressed or relaxed state.The rest of the paper is organized as follows: By outlining some of the difficulties that these systems encountered, we present earlier automatic stress detection techniques here. We describe the stress-inducing stimuli that were employed, how stress was measured, the signals that were gathered, and the machine learning techniques that were applied in these studies.Stress detection from speech signals has many applications. It is used in psychology to monitor the different stress levels of patients with different stress conditions and provide necessary treatments. The safety and security of a system can be established by monitoring the different stress levels of pilots, deep sea divers and military officials undertaking law enforcement. Stress detection is also useful in speaker identification, deception detection and identification of threatening calls in a few cases of crimes . In ordeKevin Tomba et al.  worked oSpeech and facial expression are two natural and effective ways of expressing emotions when human beings communicate with each other. During the last two decades, audio-visual emotion recognition integrating speech and facial expression has attracted extensive attention owing to its promising potential applications in human–computer interaction ,28. HoweG. Giannakakis et al.  recordedAudio-visual data were not considered for the stress detection, and only audio signals were used. Moreover, the accuracy of the results can be improved. Although there is much research discussing the recognition and analysis of the six basic emotions, i.e., anger, disgust, fear, happiness, sadness, and surprise, considerably less research has focused on stress and anxiety detection from audio visuals, as these states are considered as complex emotions that are linked to basic emotions . The results of emotion state recognition from audio-visual data can be improved using deep learning techniques, which can be further used to detect stress.Overall, this seems to be an interesting area of research, and the analysis of the existing work would help in carrying out future research. To sum up, despite the fact that other works in the literature also performed multimodal emotion recognition on RAVDESS, such as Wang et al. , who useIn the 1990s and 2000s, the face recognition community was dominated by holistic techniques. Faces are represented using holistic approaches utilising the complete facial region. Many of these approaches function by projecting facial photographs into a low-dimensional space that eliminates unimportant features and variances. PCA is one of the most prominent techniques in this field. Deep neural networks trained with extremely huge datasets have lately supplanted older approaches based on hand-crafted features and typical machine learning techniques. Deep face recognition algorithms, which employ hierarchical design to learn discriminative face representation, have significantly enhanced state-of-the-art performance and spawned a multitude of successful real-world applications. Deep learning employs many processing layers to discover data representations with numerous feature extraction levels ,35.Eckman and Friesen  created FACS divides the face into 46 action units (AUs), as shown in Almost any anatomically conceivable facial expression can be coded using FACS, which breaks it down into the specific action units (AUs) that give rise to the expression, as shown in OpenFace is a tool intended for computer vision and machine learning researchers, the affective computing community, and people interested in building interactive applications based on facial behavior analysis. OpenFace is the first toolkit capable of facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation with available source code for both running and training the models. Specifically, OpenFace can identify AUs 1, 2, 4, 5, 6, 7, 9, 10, 12, 14, 15, 17, 20, 23, 25, 26, 28 and 45.There are two ways to categorize AUs: intensity and presence. Presence  indicates whether an AU is visible on the face. On a scale of 1 to 5, intensity indicates the degree of AU intensity (min to max). Both of these scores are presented by OpenFace.These two scores are provided by OpenFace. The output file’s column AU01 c encodes 0 as not present and 1 as present for the presence of AU 1. The output file’s column AU01 r has continuous values in the range of 0 (not present), 1 (present at minimum intensity), and 5 (present at maximum intensity) for the intensity of AU 1.Our proposed stress detection framework includes two systems: a speech emotion recognizer and a face emotion recognizer. The outputs of these subsystems were integrated to identify the dominant emotion and eventually result in a stressed or unstressed state. In the current research, we made a point to highlight a novel method of implementing two different algorithms to function better than any single algorithm working individually. The proposed algorithm not only improves the overall accuracy in determining emotions but also is faster than each individual algorithm, as it uses the advantages of each algorithm and eliminates the disadvantages or time-consuming processes of each of them. Further, the work may seem complicated at the first glance; however, the accuracy improvement in the field of mental stress determination is what we are looking for, and our set objectives for the research work are met through the approach.The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) is licensed under CC BY-NA-SC 4.0. The paper by Livingstone SR and Russo FA (2018) described the construction and validation of the dataset.There are 7356 files in the RAVDESS. Each file was rated ten times for emotional validity, intensity, and authenticity. A group of 247 people who were typical untrained adult research participants from North America provided ratings. The second group of 72 people provided test–retest data. Emotional validity, interrater reliability, and test–retest intra-rater reliability were all reported to be high.The dataset included all 7356 RAVDESS files in their entirety . The three modality formats for each of the 24 actors were audio-only , audio-video , and video-only  (no sound). Please take note that Actor 18 did not have any song files.A total of 4948 samples were used for this task. Audio files were extracted from video-audio files using the “mp4 to wav” algorithm. The filenames for each of the 7356 RAVDESS files were distinctive. A seven-part numerical identifier comprised the filename . These codes specified the properties of the stimulus:The filename identifiers used are illustrated in Taking the example of the RAVDESS filename 02-01-06-01-02-01-12.mp4:Video-only (02)Speech (01)Fearful (06)Normal intensity (01)Statement “dogs” (02)1st Repetition (01)12th Actor (12)Female, as the actor ID number is even.ANN and/or CNN have been presented before in the literature, and an accuracy of around 80% has been reported for them. In our literature review, we did not find any individual algorithm which would improve the accuracy of prediction beyond 90%. So, we needed a different approach wherein we combined two relatively less processor-heavy algorithms to work on and improve the accuracy and simultaneously work at a faster rate. However, as rightly pointed out by the reviewer, in our continued plan for our research work, we will make a point to work on ANN- and CNN-based algorithms to either present a comparative analysis or to cascade them as per our intended method to verify their performance for the said cause. Recurrent neural networks (RNNs) have been successfully applied to sequence learning issues such as action identification, scene labeling, and language processing. An RNN has a recurrent connection, unlike feed-forward networks such as convolutional neural networks (CNNs), where the previous hidden state is an input to the subsequent state. An enhanced RNN, or sequential network, called a long short-term memory network, allows information to endure. It is capable of resolving the RNN’s vanishing gradient issue. Persistent memory is achieved via a recurrent neural network or RNN. Let us imagine that when reading a book or viewing a movie, you are aware of what happened in the preceding scene or chapter. RNNs function similarly; they retain the knowledge from the past and apply it to process the data at hand. Due to their inability to remember long-term dependencies, RNNs have this drawback. Long-term dependency issues are specifically avoided when designing LSTMs.In our case, RNN is used to classify data of facial landmark position with respect to time for visual data analysis and to classify the pitch of different frequencies of the audio signal with respect to time to determine the emotions.Speech and facial expressions are used to detect users’ emotional states. These modalities are combined by employing two independent models connected by a novel approach. By merging the information from aural and visual modalities, audio-visual emotion identification is vital for the human–machine interaction system. We propose a cascaded RNN-LSTM approach for audio-visual emotion recognition through correlation analysis. The emotions will finally be categorized as a stressed mental state or a relaxed mental state. We use the RAVDESS dataset for the verification of the proposed algorithm.The flowgraph for stress recognition using speech signals is shown in Our deep learning model contains two individual input streams, i.e., the audio network processing audio signals with the cascaded RNN-LSTM model, and the visual network processing visual data with the hybrid RNN-LSTM model. The flowchart for the algorithm is in In the proposed algorithm, audio files are extracted from the video files and processed separately. Librosa is used to process audio files while OpenFace is used to process video files. Overall, 66% of samples are used for training purposes, while the rest are used for testing the algorithm. In the algorithm, RNN and LSTM work parallelly to improve the speed of the feature extraction process. Audio signals need 20 neurons in the LSTM network while video signals need 40 neurons due to their signal processing requirements. MFCC is used as a filter for feature extraction. Dropout layers are used to prevent data from overfitting. Max pooling with convolution creates the final 8 required labels from the features. A dense sigmoid function is used for the final classification of the output with 10 neurons each. The separate outputs of both audio and video files are compared on a common platform to improve the accuracy by matching the missing labels. The following emotions are predicted in this model: “neutral”: “01”, “calm”: “02”, “happy”: “03”, “sad”: “04”, “angry”: “05”, “fearful”: “06”, “disgust”: “07”, “surprised”: “08”. Finally, 8 emotions are classified into 2 mental states—stressed and relaxed. First of all, we chose the method of comparing both audio and video files to avoid any misrepresentation of emotions due to the use of only one kind of file. In a scenario where the classification of both files is different, the average sum of scores of each signal will determine the probability of the inclination of the signals to a particular emotion. However, such a scenario has not yet occurred in our work, and hence the algorithm has not yet been validated.We used the Jupyter interface to run the program. LibROSA, a python package, was used for music and audio analysis, while the OpenFace package was used for facial motion tracking.We plotted the signal from a random file with audio and facial recognition separated as shown in Two facial recognition examples are illustrated in NumPy array was created for extracting Mel-frequency cepstral coefficients (MFCCs), while the classes for prediction were extracted from the name of the file.To apply the cascaded RNN-LSTM method effectively, we need to expand the dimensions of our array, adding a third one using the NumPy “expand_dims” feature.Layer (type) Output Shape Param #=================================================================conv1d_1 (Conv1D)  768_________________________________________________________________activation_1 (Activation)  0_________________________________________________________________dropout_1 (Dropout)  0.1_________________________________________________________________max_pooling1d_1  0_________________________________________________________________conv1d_2 (Conv1D)  82,048_________________________________________________________________activation_2 (Activation)  0_______________________________________________________________dropout_2 (Dropout)  0.5_________________________________________________________________flatten_1 (Flatten)  0_________________________________________________________________dense_1 (Dense)  6410_________________________________________________________________activation_3 (Activation)  0=================================================================Total params: 89,226Trainable params: 89,226Non-trainable params: 0The model loss of epochs based on training and test data is shown in the To understand the errors of the top solution, we extracted the confusion matrix of the SVM, LSTM, and RNN-LSTM approaches with an accuracy of 76%, 82%, and 91%, respectively. The confusion matrix displayed in the The proposed algorithm is compared with the conventional ones and the performance analysis is presented the Final Output:1633/1633 [==============================]—0s 125s/stepAccuracy: 91.00%The existing work was focused on either audio or facial images. In audio-visual data, the separate output of audio and video files was compared on a common platform to improve accuracy by matching the missing labels. In order to enhance the accuracy further, we increased the dimensions of the dataset, as LSTM works better with more data. The accuracy for prediction for the proposed algorithm for the RAVDESS dataset is 91%.Only image-based classification may give polarized results in cases where the image under processing lacks the overall gesture being conveyed. Moreover, using audio and visual signals will help to improve the emotion classification accuracy, which is needed to determine whether the algorithm further needs to be fully developed for the medical determination of mental stress. Although we used well-established packages for our work, we made several changes to the algorithm to make it work and provide novelty. The changes in the algorithm include cascading or the parallel operation of algorithms , the addition of dropout layers to adjust the blank values and to avoid overfitting of the data, and processing of both audio and video files to compare and improve classification accuracy. We would like to state that this method of implementing the algorithm has never been reported in the literature before.Detecting stress is essential before it turns chronic and leads to health issues. The current paper suggests that audio-visual data have the potential to detect stress. In our society, stress is becoming a major concern, and modern employment challenges such as heavy workloads and the need to adjust to ongoing change only make the situation worse. In addition to severe financial losses in businesses, people are experiencing health issues related to excessive amounts of stress. Therefore, it is crucial to regularly check your stress levels to detect stress in its preliminary stages and prevent harmful long-term consequences. The necessity for individuals to handle chronic stress gave rise to the concept of stress detection. The accuracy of the cascaded RNN-LSTM approach for the RAVDESS dataset is 91%. The obtained results are 15–20% better than those of other conventional algorithms. The proposed method is an excellent starting point to work towards mental health by detecting stress and improving one’s quality of life.The evaluation of the test results showed that the successful detection of stress is achieved, although further improvements and extensions can be made. The implementation of this system can be improved by using more efficient data structures and software to reduce delays and achieve real-time requirements.