BLSP-Emo: Towards Empathetic Large Language-Speech Models via Self-Alignment

Chen Wang, Minpeng Liao, Zhongqiang Huang,Junhong Wu, Chenqing Zong, Jiajun Zhang

Institute of Automation, Chinese Academy of Sciences

Alibaba Group

Introduction

• BLSP-Emo is designed to enable an end-to-end speech-language model to understand emotions in speech and generate empathetic responses, using only existing ASR and SER data.
• BLSP-Emo is built based on Whisper-large-v2 and Qwen-7B-Chat.

Example

More examples can be found in the project page. You can also try our model online at modelscope.

License

• The license of our project is [Apache License 2.0]()
• Our models are based on Qwen and Whisper. If you want to use our models, please do not violate the MIT License of whisper and the License of Qwen

Citation

If you find our project useful, hope you can star our repo and cite our paper as follows:

@misc{wang2024blspemo,
      title={BLSP-Emo: Towards Empathetic Large Speech-Language Models}, 
      author={Chen Wang and Minpeng Liao and Zhongqiang Huang and Junhong Wu and Chengqing Zong and Jiajun Zhang},
      year={2024},
      eprint={2406.03872},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}