牛志康 (Zhikang Niu)

博士研究生，跨媒体语言智能实验室（X-LANCE Lab）
上海交通大学计算机科学与工程系
与上海创智学院联合培养。
中国上海
研究方向 教育经历 研究经历 论文发表 研究项目 荣誉与奖励 学术与社区活动
如有任何问题，欢迎通过邮件联系：zhikangniu@sjtu.edu.cn
[GitHub] [Google Scholar] [微信] [简历]

研究方向

 在陈谐副教授的指导下，主要从事音频信号处理、语音表征与多模态大语言模型研究。接下来充满挑战与机遇的五年里也会继续全力以赴！💪 同时对构建可扩展系统抱有浓厚兴趣。目前重点关注以下研究方向：

教育经历

研究经历

论文发表

语音合成与全模态系统
  • Zhikang Niu, Sanyuan Chen, Long Zhou, Ziyang Ma, Xie Chen, Shujie Liu.
    NDVQ: Robust Neural Audio Codec with Normal Distribution-Based Vector Quantization.
    SLT 2024, [Link] [PDF] [Code] [BibTeX]

  • Zhikang Niu, Shujie Hu, Jeongsoo Choi, Yushen Chen, Peining Chen, Pengcheng Zhu, Yunting Yang, et al.
    Semantic-VAE: Semantic-Alignment Latent Representation for Better Speech Synthesis.
    InterSpeech 2026, [Link] [PDF] [Code] [BibTeX]

  • Tianrui Wang, Ziyang Ma, Yizhou Peng, Haoyu Wang, Zhikang Niu, et al.
    Evaluating the Expressive Appropriateness of Speech in Rich Contexts.
    ACL 2026 Main, [Link] [PDF] [Code] [BibTeX]
  • Rixi Xu, Qingyu Liu, Haitao Li, Yushen Chen, Zhikang Niu, et al.
    X-Voice: Enabling Everyone to Speak 30 Languages via Zero-Shot Cross-Lingual Voice Cloning.
    arXiv 2026, [Link] [PDF] [Code] [Demo]
  • Wenxi Chen, Ruiqi Yan, Yushen Chen, Zhikang Niu, Ziyang Ma, Xiquan Li, et al.
    SAC: Neural Speech Codec with Semantic-Acoustic Dual-Stream Quantization.
    ACL 2026 Main, [Link] [PDF] [Code] [Demo] [BibTeX]
  • Jeongsoo Choi*, Zhikang Niu*, Ji-Hoon Kim, Chunhui Wang, Joon Son Chung, Xie Chen.
    Accelerating Diffusion-based Text-to-Speech Model Training with Dual Modality Alignment.
    InterSpeech 2025 Oral, [Link] [PDF] [Code] [BibTeX]

  • Yushen Chen, Zhikang Niu, Ziyang Ma, Keqi Deng, Chunhui Wang, Jian Zhao, Kai Yu, Xie Chen.
    F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching.
    ACL 2025 Main, [Link] [PDF] [Code] [BibTeX] [Talk]
    F5-TTS has collected 13,000+ stars on GitHub.
  • Wenhao Guan, Zhikang Niu, Ziyue Jiang, Kaidi Wang, Peijie Chen, Qingyang Hong, Lin Li, Xie Chen.
    UniVoice: Unifying Autoregressive ASR and Flow-Matching based TTS with Large Language Models.
    InterSpeech 2026, [Link] [PDF] [Code] [BibTeX]

  • Qixi zheng, Yushen Chen, Zhikang Niu, Ziyang Ma, Xiaofei Wang, Kai Yu, Xie Chen.
    Accelerating Flow-Matching-Based Text-to-Speech via Empirically Pruned Step Sampling.
    InterSpeech 2025, [Link] [PDF] [Code]

  • Wenxi Chen, Ziyang Ma, Ruiqi Yan, Yuzhe Liang, Xiquan Li, Ruiyang Xu, Zhikang Niu, et al.
    SLAM-Omni: Timbre-Controllable Voice Interaction System with Single-Stage Training.
    ACL 2025 Findings, [Link] [PDF] [Code] [BibTeX]

  • Chenpeng Du, Yiwei Guo, Hankun Wang, Yifan Yang, Zhikang Niu, Shuai Wang, Hui Zhang, Xie Chen.
    VALL-T: Decoder-Only Generative Transducer for Robust and Decoding-Controllable Text-to-Speech.
    ICASSP 2025, [Link] [PDF] [BibTeX]

  • Guanrou Yang, Ziyang Ma, Zhisheng Zheng, Yakun Song, Zhikang Niu, Xie Chen.
    Fast-HuBERT: An Efficient Training Framework for Self-Supervised Speech Representation Learning.
    ASRU 2023, [Link] [PDF] [Code] [BibTeX]

  • Yuzhe Liang, Wenzhe Liu, Chunyu Qiang, Zhikang Niu, Yushen Chen, Ziyang Ma, et al.
    Towards Flow-Matching-based TTS without Classifier-Free Guidance.
    [Link] [PDF]

  • Xiquan Li, Junxi Liu, Yuzhe Liang, Zhikang Niu, Wenxi Chen, Xie Chen.
    MeanAudio: Fast and Faithful Text-to-Audio Generation with Mean Flows. ACL 2026,
    [Link] [PDF] [Code] [BibTeX]

  • Yuxiang Zhao, Yunchong Xiao, Yushen Chen, Zhikang Niu, Shuai Wang, Kai Yu, Xie Chen.
    Traceable TTS: Toward Watermark-Free TTS with Strong Traceability.
    [Link] [PDF]

  • Qingyu Liu, Yushen Chen, Zhikang Niu, Chunhui Wang, Yunting Yang, Bowen Zhang, Jian Zhao, Pengcheng Zhu, Kai Yu, Xie Chen.
    Cross-Lingual F5-TTS: Towards Language-Agnostic Voice Cloning and Speech Synthesis.
    ICASSP 2026, [Link] [PDF]

  • Tianrui Wang, Meng Ge, Zhikang Niu, Cheng Gong, Chunyu Qiang, Haoyu Wang, Zhongqian Huang, Ziyang Ma, Xiaofei Wang, et al.
    A Progressive Generation Framework with Speech Pre-trained Model for Expressive Voice Conversion.
    ICME 2025.

  • Yakun Song, Xiaobin Zhuang, Jiawei Chen, Zhikang Niu, Guanrou Yang, Chenpeng Du, Dongya Jia, Zhuo Chen, Yuping Wang, Yuxuan Wang, Xie Chen.
    DiSTAR: Diffusion over a Scalable Token Autoregressive Representation for Speech Generation.
    [Link] [PDF]

  • Pengchao Feng, Yao Xiao, Ziyang Ma, Zhikang Niu, Shuai Fan, Yao Li, Sheng Wang, Xie Chen.
    Task Vector in TTS: Toward Emotionally Expressive Dialectal Speech Synthesis.
    ICASSP 2026, [Link] [PDF]

  • Yushen Chen, Junzhe Liu, Yujie Tu, Zhikang Niu, Yuzhe Liang, Chunyu Qiang, Chen Zhang, Kai Yu, Xie Chen.
    Habibi: Laying the Open-Source Foundation of Unified-Dialectal Arabic Speech Synthesis.
    [Link] [PDF]

  • Wenxi Chen, Dongya Jia, Yushen Chen, Zhikang Niu, Yuzhe Liang, Xiquan Li, Ruiqi Yan, Ziyang Ma, Guanrou Yang, et al.
    WavTTS: Towards High-Quality Zero-Shot TTS via Direct Raw Waveform Modeling.
    [Link] [PDF]

  • Guanrou Yang, Tian Tan, Qian Chen, Zhikang Niu, Yakun Song, Ziyang Ma, Yushen Chen, Zeyu Xie, Tianrui Wang, et al.
    WavCube: Unifying Speech Representation for Understanding and Generation via Semantic-Acoustic Joint Modeling.
    [Link] [PDF]

基准测试
  • MMAE Team, core contributor.
    MMAE: A Massive Multitask Audio Editing Benchmark.
    [Link] [PDF]

  • MMAR Team.
    MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix.
    NeurIPS Dataset & Benchmark 2025, [Link] [PDF] [Code]

  • Ruiqi Yan, Xiquan Li, Wenxi Chen, Zhikang Niu, Chen Yang, Ziyang Ma, Kai Yu, Xie Chen.
    URO-Bench: A Comprehensive Benchmark for End-to-End Spoken Dialogue Models.
    EMNLP 2025 Findings, [Link] [PDF] [Code] [BibTeX]

  • Zihan Liu*, Zhikang Niu*, Qiuyang Xiao, Zhisheng Zheng, et al.
    STAR-Bench: Probing Deep Spatio-Temporal Reasoning as Audio 4D Intelligence.
    ICLR 2026, [Link] [PDF] [Code] [BibTeX]

研究项目

 开源项目：
  • thorough-pytorch： 一份中文 PyTorch 教程，已在 GitHub 获得 2,300+ Stars 和 333 个 Forks。
  • CSBasicKnowledge： 记录计算机科学、人工智能和电子工程相关知识，已在 GitHub 获得 560+ Stars。
  • 更多开源内容可在个人 GitHub 主页中查看。
研究项目

荣誉与奖励
  • 2024，百度“文心杯”创业大赛三等奖（第 21 名，共 1,600 支队伍）。
  • 2024，微软亚洲研究院“明日之星”。
  • 2022，教育部国家奖学金。
  • 2021，美国大学生数学建模竞赛（MCM/ICM）Meritorious Winner。
  • 2021、2023，西安电子科技大学一等奖学金。

学术与社区活动
  • 2025、2026，ICME 审稿人。
  • 2023.09–2024.09，CS-BAOYAN 负责人；CS-BAOYAN 是一个计算机保研开源社区。
  • 2021.11–至今，Datawhale 成员；Datawhale 是一个人工智能开源组织，致力于帮助数据科学爱好者参与人工智能社区。
  • 2021.11–至今，Xmart 论坛维护者；Xmart 是上海交通大学 X-LANCE 实验室面向学生的开源论坛，致力于帮助学生参与语音人工智能社区。



更新时间：