add-audio-examples

README.md

@@ -3,12 +3,45 @@ license: apache-2.0
 pipeline_tag: audio-text-to-text
 ---
 
-# 1. Step-Audio-Chat
+## 1. Step-Audio-Chat
 
 This repository contains the Multimodal Large Language Model (LLM) component of Step-Audio. It is a 130 billion parameter multimodal LLM that is responsible for understanding and generating human speech. The model is specifically designed to seamlessly integrate functions such as speech recognition, semantic understanding, dialogue management, voice cloning, and speech generation.
 
-## 2. Evaluation
-### 2.1 LLM judge metrics(GPT-4o) on [**StepEval-Audio-360**](https://huggingface.co/datasets/stepfun-ai/StepEval-Audio-360)
+
+## 2. Examples
+### Clone audio
+| role   | prompt wav | clone wav |
+|:-------:|:-------:|:-------:|
+|于谦| <audio controls src="https://huggingface.co/stepfun-ai/Step-Audio-Examples/resolve/main/prompt_wav_yuqian.wav"></audio>|<audio controls src="https://huggingface.co/stepfun-ai/Step-Audio-Examples/resolve/main/clone_wav_yuqian.wav"></audio>|
+|李雪琴| <audio controls src="https://huggingface.co/stepfun-ai/Step-Audio-Examples/resolve/main/prompt_wav_lixueqin.wav"></audio>|<audio controls src="https://huggingface.co/stepfun-ai/Step-Audio-Examples/resolve/main/clone_wav_lixueqin.wav"></audio>|
+
+### Speed control
+| prompt | response |
+|:-------:|:-------:|
+|Human: 说一个绕口令<br>Assistant: 吃葡萄不吐葡萄皮，不吃葡萄倒吐葡萄皮<br>Human: 哎，你能把这个绕口令说的再快一点吗？|<audio controls src="https://huggingface.co/stepfun-ai/Step-Audio-Examples/resolve/main/speed_control1.wav"></audio>|
+|Human: 说一个绕口令<br>Assistant: 吃葡萄不吐葡萄皮，不吃葡萄倒吐葡萄皮<br>Human: 哎，你能把这个绕口令说的再快一点吗？<br>Assistant: 吃葡萄不吐葡萄皮，不吃葡萄倒吐葡萄皮<br>Human: 呃，你再用非常非常慢的速度说一遍的。|<audio controls src="https://huggingface.co/stepfun-ai/Step-Audio-Examples/resolve/main/speed_control2.wav"></audio>|
+
+### High EQ(emotional control & tone control)
+| prompt | response |
+|:-------:|:-------:|
+|Human: 你这语气又不撒娇又不卖萌的，要不你撒个娇卖个萌吧。|<audio controls src="https://huggingface.co/stepfun-ai/Step-Audio-Examples/resolve/main/tone_control.wav"></audio>|
+|Human: 怎么办？我感觉我的人生很失败。|<audio controls src="https://huggingface.co/stepfun-ai/Step-Audio-Examples/resolve/main/emotional_control1.wav"></audio>|
+|Human: 小跃。你真的是。特别厉害。|<audio controls src="https://huggingface.co/stepfun-ai/Step-Audio-Examples/resolve/main/emotional_control2.wav"></audio>|
+
+### Multilingual (e.g., Chinese, English, Japanese)
+| prompt | response |
+|:-------:|:-------:|
+|Human: What did the speaker mean when they said, it's raining cats and dogs?<br>Assistant: When they say "It's raining cats and dogs," it just means it's raining really hard. The speaker isn't literally saying cats and dogs are falling from the sky! It's just a fun way to describe heavy rain.|<audio controls src="https://huggingface.co/stepfun-ai/Step-Audio-Examples/resolve/main/multilingual2.wav"></audio>|
+|Human: こんにちは。（你好）<br>Assistant：こんにちは！何か手伝いましょうか？（您好！我可以帮你做点什么吗？）|<audio controls src="https://huggingface.co/stepfun-ai/Step-Audio-Examples/resolve/main/multilingual1.wav"></audio>|
+
+### Rap & Vocal
+| prompt | response |
+|:-------:|:-------:|
+|human：唱一段rap|<audio controls src="https://huggingface.co/stepfun-ai/Step-Audio-Examples/resolve/main/rap.wav"></audio>|
+
+
+## 3. Evaluation
+### 3.1 LLM judge metrics(GPT-4o) on [**StepEval-Audio-360**](https://huggingface.co/datasets/stepfun-ai/StepEval-Audio-360)
 <table>
     <caption>Comparison of fundamental capabilities of voice chat on the StepEval-Audio-360.</caption>
     <thead>
@@ -50,7 +83,7 @@ This repository contains the Multimodal Large Language Model (LLM) component of
 *Note: Moshi are marked with "\*" and should be considered for reference only.
 
 
-### 2.2 Public Test Set
+### 3.2 Public Test Set
 
 <table>
     <thead>
@@ -127,7 +160,7 @@ This repository contains the Multimodal Large Language Model (LLM) component of
 
 *TriviaQA dataset marked with "\*" indicates results are for reference only.*
 
-### 2.3 Audio instruction following
+### 3.3 Audio instruction following
 <table>
     <thead>
         <tr>
@@ -175,6 +208,6 @@ This repository contains the Multimodal Large Language Model (LLM) component of
 </table>
 
 
-## 3. More information
+## 4. More information
 
 For more information, please refer to our repository: [Step-Audio](https://github.com/stepfun-ai/Step-Audio).