 |
Zhikang Niu (牛志康) |
Research Interest
I work in the field of Audio Singal Processing, Audio Codec Model,Multimodal Large Language Model, Machine learning, and Deep learning supervised by Prof. Xie Chen, I will try my best in the next five exciting years! 💪. Currently, I focus on the following research topics:- Audio Codec Model (Audio Tokenizer)
- Speech Synthesis (Text to Speech)
- Multimodal Large Language Model
Education and Intern
- 2023.08-2024.09 Research Intern, Natural Language Computing Group (NLC), Microsoft Research Asia (MSRA). Supervised by Shujie Liu and Long Zhou. Focus on Audio Codec and Speech Synthesis.
- 2020.09-2024.06 Bachelor Degree of Engineering, School of Artificial Intelligence, Xidian University.
Publications
- Guanrou Yang, Ziyang Ma, Zhisheng Zheng, Yakun Song, Zhikang Niu, Xie Chen*.
Fast-HuBERT: An Efficient Training Framework for Self-Supervised Speech Representation Learning.
IEEE Automatic Speech Recognition and Understanding Workshop (ASRU Workshop), 2023. [Link] [PDF] [BibTeX]
- Chenpeng Du, Yiwei Guo, Hankun Wang, Yifan Yang, Zhikang Niu, Shuai Wang, Hui Zhang, Xie Chen.
VALL-T: Decoder-Only Generative Transducer for Robust and Decoding-Controllable Text-to-Speech.
[Link] [PDF] [BibTeX]
- Zhikang Niu, Sanyuan Chen, Long Zhou, Ziyang Ma, Xie Chen, Shujie Liu.
NDVQ: Robust Neural Audio Codec with Normal Distribution-Based Vector Quantization.
IEEE Spoken Language Technology Workshop (SLT 2024), [Link] [PDF] [BibTeX]
- Yushen Chen, Zhikang Niu, Ziyang Ma, Keqi Deng, Chunhui Wang, Jian Zhao, Kai Yu, Xie Chen.
F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching.
[Link] [PDF] [Code]
Projects
Open-Source Projects:- thorough-pytorch: A Chinese PyTorch tutorial and it has already collected 2,300 more stars and 333 forks on GitHub.
- CSBasicKnowledge: This repo will record some knowledge about computer science, artificial intelligence and EE. It has already collected 560 more stars on GitHub.
- More open-source contents can be found on my GitHub.
- encodec-pytorch: An unofficial PyTorch implementation of the High Fidelity Neural Audio Compression and it has already collected 129 stars on GitHub. [checkpoint]
Honors and Awards
- 2024, Stars of Tomorrow, Microsoft Research Asia.
- 2022, National Scholarship, Ministry of Education in China.
- 2021, Meritorious Winner, Interdisciplinary Contest In Modeling.
- 2021, 2023, The First Prize Scholarship, Xidian University.
Activities
- 2021.11-Now, Datawhale member (an open-source AI organization), helped data science fans get involved in the AI community.
Updating time: 2024.08.13