Now, I’m a research scientist in ByteDance, working on large speech model and AI avatar. Our work are widely deployed in famous applications and services, such as Tiktok/抖音, Capcut/剪映, Volcano Engine(火山引擎), etc.

I graduated from the Department of Computer Science, Zhejiang University (浙江大学计算机科学与技术学院) with a bachelor’s degree in 2020. After that, in 2023, I graduated with a master’s degree in the Department of Computer Science, Zhejiang University, advised by Kejun Zhang (张克俊).

My research interest includes speech synthesis, music generation, avatar and translation. I have published more than 20 papers at the top international AI conferences such as NeurIPS, ICLR, ICML, ACL, AAAI, etc. I served as area chair for ACL and NAACL. Also, I served as reviewer for NeurIPS, ICLR, TMM, CVPR, etc.

I used to be a research intern at Tencent AI Lab and SEA AI Lab , collaborating with Shuicheng Yan (颜水成) and Yi Ren (任意). Before that, I was a research intern at ByteDance AI Lab , advised by Bilei Zhu (朱碧磊). Also, I had a one-year long internship at Microsoft Research Asia , Xu Tan (谭旭), Tao Qin (秦涛) and Tie-yan Liu (刘铁岩).

I’m one of the main contributors of a popular music open-source project: Muzic Github Stars.

🔥 News

  • 2024.10: One paper is accepted by TAFFC!
  • 2024.09: One paper is accepted by NeurIPS 2024!
  • 2024.07: One paper is accpeted by TASLP!
  • 2024.02: Our voice cloning is launched in Capcut at full stream!
  • 2024.01: Two papers are accepted by ICLR 2024!
  • 2023.06: One paper is accetped by ICML Workshop!
  • 2023.05: One paper is accepted by TMM!
  • 2023.05: One paper is accepted by INTERSPEECH 2023!
  • 2023.01: One paper is accepted by ICLR 2023!

📝 Publications

🎙 Speech Translation and Synthesis

🎼 Music Generation and Retrieval

🧑‍🎨 Multi-modal Learning

🎖 Honors and Awards

  • National Scholarship (Top 1%)
  • Zhijun He Scholarship (Top 1%)
  • Tianzhou Chen Scholarship (Top 1%)
  • Huawei Scholarship (Top 1%)
  • Outstanding Graduates of Zhejiang Province

📖 Educations

  • 2020.06 - 2023.06, Master, Zhejiang University, Hangzhou.
  • 2016.09 - 2020.06, Undergraduate, Zhejiang Univeristy, Hangzhou.

💬 Invited Talks

  • 2022.12, Music Generation with Domain Knowledge, Department of CS @ NUS.
  • 2021.08, Simulataneous Speech Translation Panel, IWSLT Workshop @ ACL 2021.
  • 2021.01, Speech Translation for Unwritten Languages, Live Share @ MSRA.

💻 Internships