Zhikang Niu (牛志康)

Ph.D. Student. Cross Media (X-)Language Intelligence Lab
Department of Computer Science and Engineering, Shanghai Jiao Tong University.
Jointly trained with Shanghai Innovation Institute.
Shanghai, China
Research Interests Education Research Experience Publications Research Projects Honors and Awards Activities
if you have any questions, please feel free to contact me with zhikangniu@sjtu.edu.cn
[GitHub] [Google Scholar] [WeChat] [Resume]

Research Interest

 I work in the field of Audio Singal Processing, Speech Representation, and Multimodal Large Language Model supervised by Assoc. Prof. Xie Chen, I will try my best in the next five exciting years! 💪. I am also interested in building scalable systems. Currently, I focus on the following research topics:

Education

Research Experience

Publications

Speech Synthesis/Omni System
  • Zhikang Niu, Sanyuan Chen, Long Zhou, Ziyang Ma, Xie Chen, Shujie Liu.
    NDVQ: Robust Neural Audio Codec with Normal Distribution-Based Vector Quantization.
    SLT 2024, [Link] [PDF] [Code] [BibTeX]

  • Zhikang Niu, Shujie Hu, Jeongsoo Choi, Yushen Chen, Peining Chen, Pengcheng Zhu, Yunting Yang, et al.
    Semantic-VAE: Semantic-Alignment Latent Representation for Better Speech Synthesis.
    InterSpeech 2026, [Link] [PDF] [Code] [BibTeX]

  • Tianrui Wang, Ziyang Ma, Yizhou Peng, Haoyu Wang, Zhikang Niu, et al.
    Evaluating the Expressive Appropriateness of Speech in Rich Contexts.
    ACL 2026 Main, [Link] [PDF] [Code] [BibTeX]
  • Rixi Xu, Qingyu Liu, Haitao Li, Yushen Chen, Zhikang Niu, et al.
    X-Voice: Enabling Everyone to Speak 30 Languages via Zero-Shot Cross-Lingual Voice Cloning.
    arXiv 2026, [Link] [PDF] [Code] [Demo]
  • Wenxi Chen, Ruiqi Yan, Yushen Chen, Zhikang Niu, Ziyang Ma, Xiquan Li, et al.
    SAC: Neural Speech Codec with Semantic-Acoustic Dual-Stream Quantization.
    ACL 2026 Main, [Link] [PDF] [Code] [Demo] [BibTeX]
  • Jeongsoo Choi*, Zhikang Niu*, Ji-Hoon Kim, Chunhui Wang, Joon Son Chung, Xie Chen.
    Accelerating Diffusion-based Text-to-Speech Model Training with Dual Modality Alignment.
    InterSpeech 2025 Oral, [Link] [PDF] [Code] [BibTeX]

  • Yushen Chen, Zhikang Niu, Ziyang Ma, Keqi Deng, Chunhui Wang, Jian Zhao, Kai Yu, Xie Chen.
    F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching.
    ACL 2025 Main, [Link] [PDF] [Code] [BibTeX] [Talk]
    F5-TTS has collected 13,000+ stars on GitHub.
  • Wenhao Guan, Zhikang Niu, Ziyue Jiang, Kaidi Wang, Peijie Chen, Qingyang Hong, Lin Li, Xie Chen.
    UniVoice: Unifying Autoregressive ASR and Flow-Matching based TTS with Large Language Models.
    InterSpeech 2026, [Link] [PDF] [Code] [BibTeX]

  • Qixi zheng, Yushen Chen, Zhikang Niu, Ziyang Ma, Xiaofei Wang, Kai Yu, Xie Chen.
    Accelerating Flow-Matching-Based Text-to-Speech via Empirically Pruned Step Sampling.
    InterSpeech 2025, [Link] [PDF] [Code]

  • Wenxi Chen, Ziyang Ma, Ruiqi Yan, Yuzhe Liang, Xiquan Li, Ruiyang Xu, Zhikang Niu, et al.
    SLAM-Omni: Timbre-Controllable Voice Interaction System with Single-Stage Training.
    ACL 2025 Findings, [Link] [PDF] [Code] [BibTeX]

  • Chenpeng Du, Yiwei Guo, Hankun Wang, Yifan Yang, Zhikang Niu, Shuai Wang, Hui Zhang, Xie Chen.
    VALL-T: Decoder-Only Generative Transducer for Robust and Decoding-Controllable Text-to-Speech.
    ICASSP 2025, [Link] [PDF] [BibTeX]

  • Guanrou Yang, Ziyang Ma, Zhisheng Zheng, Yakun Song, Zhikang Niu, Xie Chen.
    Fast-HuBERT: An Efficient Training Framework for Self-Supervised Speech Representation Learning.
    ASRU 2023, [Link] [PDF] [Code] [BibTeX]

  • Yuzhe Liang, Wenzhe Liu, Chunyu Qiang, Zhikang Niu, Yushen Chen, Ziyang Ma, et al.
    Towards Flow-Matching-based TTS without Classifier-Free Guidance.
    [Link] [PDF]

  • Xiquan Li, Junxi Liu, Yuzhe Liang, Zhikang Niu, Wenxi Chen, Xie Chen.
    MeanAudio: Fast and Faithful Text-to-Audio Generation with Mean Flows. ACL 2026,
    [Link] [PDF] [Code] [BibTeX]

  • Yuxiang Zhao, Yunchong Xiao, Yushen Chen, Zhikang Niu, Shuai Wang, Kai Yu, Xie Chen.
    Traceable TTS: Toward Watermark-Free TTS with Strong Traceability.
    [Link] [PDF]

  • Qingyu Liu, Yushen Chen, Zhikang Niu, Chunhui Wang, Yunting Yang, Bowen Zhang, Jian Zhao, Pengcheng Zhu, Kai Yu, Xie Chen.
    Cross-Lingual F5-TTS: Towards Language-Agnostic Voice Cloning and Speech Synthesis.
    ICASSP 2026, [Link] [PDF]

  • Tianrui Wang, Meng Ge, Zhikang Niu, Cheng Gong, Chunyu Qiang, Haoyu Wang, Zhongqian Huang, Ziyang Ma, Xiaofei Wang, et al.
    A Progressive Generation Framework with Speech Pre-trained Model for Expressive Voice Conversion.
    ICME 2025.

  • Yakun Song, Xiaobin Zhuang, Jiawei Chen, Zhikang Niu, Guanrou Yang, Chenpeng Du, Dongya Jia, Zhuo Chen, Yuping Wang, Yuxuan Wang, Xie Chen.
    DiSTAR: Diffusion over a Scalable Token Autoregressive Representation for Speech Generation.
    [Link] [PDF]

  • Pengchao Feng, Yao Xiao, Ziyang Ma, Zhikang Niu, Shuai Fan, Yao Li, Sheng Wang, Xie Chen.
    Task Vector in TTS: Toward Emotionally Expressive Dialectal Speech Synthesis.
    ICASSP 2026, [Link] [PDF]

  • Yushen Chen, Junzhe Liu, Yujie Tu, Zhikang Niu, Yuzhe Liang, Chunyu Qiang, Chen Zhang, Kai Yu, Xie Chen.
    Habibi: Laying the Open-Source Foundation of Unified-Dialectal Arabic Speech Synthesis.
    [Link] [PDF]

  • Wenxi Chen, Dongya Jia, Yushen Chen, Zhikang Niu, Yuzhe Liang, Xiquan Li, Ruiqi Yan, Ziyang Ma, Guanrou Yang, et al.
    WavTTS: Towards High-Quality Zero-Shot TTS via Direct Raw Waveform Modeling.
    [Link] [PDF]

  • Guanrou Yang, Tian Tan, Qian Chen, Zhikang Niu, Yakun Song, Ziyang Ma, Yushen Chen, Zeyu Xie, Tianrui Wang, et al.
    WavCube: Unifying Speech Representation for Understanding and Generation via Semantic-Acoustic Joint Modeling.
    [Link] [PDF]

Benchmark
  • MMAE Team, core contributor.
    MMAE: A Massive Multitask Audio Editing Benchmark.
    [Link] [PDF]

  • MMAR Team.
    MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix.
    NeurIPS Dataset & Benchmark 2025, [Link] [PDF] [Code]

  • Ruiqi Yan, Xiquan Li, Wenxi Chen, Zhikang Niu, Chen Yang, Ziyang Ma, Kai Yu, Xie Chen.
    URO-Bench: A Comprehensive Benchmark for End-to-End Spoken Dialogue Models.
    EMNLP 2025 Findings, [Link] [PDF] [Code] [BibTeX]

  • Zihan Liu*, Zhikang Niu*, Qiuyang Xiao, Zhisheng Zheng, et al.
    STAR-Bench: Probing Deep Spatio-Temporal Reasoning as Audio 4D Intelligence.
    ICLR 2026, [Link] [PDF] [Code] [BibTeX]

Research Projects

 Open-Source Projects:
  • thorough-pytorch: A Chinese PyTorch tutorial and it has already collected 2,300 more stars and 333 forks on GitHub.
  • CSBasicKnowledge: This repo will record some knowledge about computer science, artificial intelligence and EE. It has already collected 560 more stars on GitHub.
  • More open-source contents can be found on my GitHub.
Research Projects

Honors and Awards
  • 2024, Third Prize, 21/1600, Wenxin Cup Entrepreneurship Competition, Baidu.
  • 2024, Stars of Tomorrow, Microsoft Research Asia.
  • 2022, National Scholarship, Ministry of Education in China.
  • 2021, Meritorious Winner, Interdisciplinary Contest In Modeling.
  • 2021, 2023, The First Prize Scholarship, Xidian University.

Activities
  • ICME Reviewer, 2025, 2026
  • 2023.09-2024.9, CS-BAOYAN owner (an open-source CS-BAOYAN organization).
  • 2021.11-Now, Datawhale member (an open-source AI organization), helped data science fans get involved in the AI community.
  • 2021.11-Now, Xmart forum maintainer (an open-source student forum from SJTU X-LANCE Lab), for helping students get involved in the speech AI community.



Last updated: