ASLP-lab/OSUM-EChat · Apply for community grant: Academic project (gpu)

Dear Sir/Madam,
Our team has recently successfully developed and implemented an innovative conversational AI project — OSUM-EChat. As an end-to-end empathetic dialogue model extended from a speech understanding large model backbone, it breaks through the limitation of traditional dialogue systems that only focus on semantic comprehension. It is capable of deeply perceiving two core types of information in the input speech: on one hand, it accurately parses the textual semantics carried by the speech; on the other hand, it keenly captures paralinguistic signals such as intonation, speech rate, and tone (e.g., the lively intonation when one is happy, or the slow speech rate when one is down). By deeply integrating the emotional tendencies implied by paralinguistics with semantic content, the model ultimately generates empathetic responses that align with the user’s emotional state. For instance, when faced with a user’s inquiry delivered in an anxious tone, the model will not only respond to the question accurately but also adopt a gentle and comforting expression style, truly achieving a conversational effect of "understanding the content" and "perceiving emotions".
Currently, the technical details of this project have been compiled into a technical report and published on the arXiv platform. You can view the complete research results via the link: http://arxiv.org/abs/2508.09600. Meanwhile, the open-source code of the project has been synchronously deployed to the GitHub repository. For reference on implementation logic or secondary development, you may access the code repository at: https://github.com/ASLP-lab/OSUM.