📝 Publications
🎙 Speech Translation and Synthesis
TASLPRefXVC: Cross-Lingual Voice Conversion with Enhanced Reference LeveragingICLR 2024Mega-TTS 2: Zero-Shot Text-to-Speech with Arbitrary Length Speech Prompts. | ProjectICLR 2023Bag of Tricks for Unsupervised Text-to-Speech | ProjectICASSP 2021Denoising Text to Speech with Frame-Level Noise Modeling | ProjectAAAI 2021UWSpeech: Speech to Speech Translation for Unwritten Languages | ProjectINTERSPEECH 2023EE-TTS: Emphatic Expressive TTS with Linguistic Information | ProjectACL 2020SimulSpeech: End-to-End Simultaneous Speech to Text TranslationIJCAI 2020Task-Level Curriculum Learning for Non-Autoregressive Neural Machine Translation
🎼 Music Generation and Retrieval
TMMSDMuse: Stochastic Differential Music Editing and Generation via Hybrid Representation | ProjectTAFFCREMAST: Real-time Emotion-based Music Arrangement with Soft TransitionACM-MM 2022ReLyMe: Improving Lyric-to-Melody Generation by Incorporating Lyric-Melody Relationships | ProjectACM-MM 2022SongDriver: Real-time Music Accompaniment Generation without Logical Latency nor Exposure BiasISMIR 2022PDAugment: Data Augmentation by Pitch and Duration Adjustments for Automatic Lyrics TranscriptionACL 2022Automatic Song Translation for Tonal Languages | ProjectICASSP 2022S3T: Self-Supervised Pre-training with Swin Transformer for Music ClassificationEMNLP 2022TeleMelody: Lyric-to-Melody Generation with a Template-Based Two-Stage Method | Project
🧑🎨 Multi-modal Learning
NeurIPS 2024MimicTalk: Mimicking a personalized and expressive 3D talking face in few minutes | Code.
ICLR 2024Real3d-portrait: One-shot realistic 3d talking portrait synthesis | Project | Code.
ICML 2023 WorkshopAda-TTA: Towards Adaptive High-Quality Text-to-Talking Avatar Synthesis | VideoNeurIPS 2022Towards Effective Multi-Modal Interchanges in Zero-Resource Sounding Object LocalizationACM-MM 2020FastLR: Non-Autoregressive Lipreading Model with Integrate-and-FireIJCAI 2019Discriminative and Correlative Partial Multi-Label Learning