About me
I am a third-year Master’s student in the Wangxuan Institute of Computer Technology at Peking University. My advisor is Prof. Xiaojun Wan. Now I am also a visiting student at Yale NLP Lab, supervised by Arman Cohan. Previously, I obtained my Bachelor’s degree in the School of Electronics Engineering and Computer Science at Peking University.
My current research interest revolves around the evaluation of LLMs, text generation, and summarization. Specifically, I am interested in automatic evaluation, human evaluation, meta-evaluation, and reproducibility. I believe that evaluation is crucial in current research. Without a more reliable evaluation mechanism, it is difficult to accurately determine whether an innovation is a genuine advancement or merely an illusion, especially in the context of a large amount of incremental research.
I am looking for 25 Fall PhD opportunities! Please do not hesitate to contact me if you are interested in my studies.
Recent News
- 2024-05: One paper is accepted to ACL 2024.
- 2024-02: A survey paper on LLM-based NLG evaluation.
- 2023-05: Two papers are accepted to ACL 2023 (one at main conference, one at findings).
Preprints
Analyzing and Evaluating Correlation Measures in NLG Meta-Evaluation
Mingqi Gao, Xinyu Hu, Li Lin, Xiaojun Wan
arXiv:2410.16834 [pdf]LLM-based NLG Evaluation: Current Status and Challenges
Mingqi Gao*, Xinyu Hu*, Jie Ruan, Xiao Pu, Xiaojun Wan
arXiv:2402.01383 [pdf]
Selected Publications
( * indicates equal contribution)
Themis: A Reference-free NLG Evaluation Language Model with Flexibility and Interpretability
Xinyu Hu, Li Lin, Mingqi Gao, Xunjian Yin, Xiaojun Wan
EMNLP 2024 [pdf] [code]Are LLM-based Evaluators Confusing NLG Quality Criteria?
Xinyu Hu*, Mingqi Gao*, Sen Hu, Yang Zhang, Yicheng Chen, Teng Xu, Xiaojun Wan
ACL 2024 [pdf] [code]Is Summary Useful or Not? An Extrinsic Human Evaluation of Text Summaries on Downstream Tasks
Xiao Pu, Mingqi Gao, Xiaojun Wan
LREC-COLING 2024 [pdf]Better than Random: Reliable NLG Human Evaluation with Constrained Active Sampling
Jie Ruan, Xiao Pu, Mingqi Gao, Xiaojun Wan, Yuesheng Zhu
AAAI 2024 [pdf]Missing Information, Unresponsive Authors, Experimental Flaws: The Impossibility of Assessing the Reproducibility of Previous Human Evaluations in NLP
Anya Belz, Craig Thomson, Ehud Reiter, and 36 more authors
Fourth Workshop on Insights from Negative Results in NLP, 2023 [pdf]Reference Matters: Benchmarking Factual Error Correction for Dialogue Summarization with Fine-grained Evaluation Framework
Mingqi Gao, Xiaojun Wan, Jia Su, Zhefeng Wang, Baoxing Huai
ACL 2023 [pdf] [code]Evaluating Factuality in Cross-lingual Summarization
Mingqi Gao*, Wenqing Wang*, Xiaojun Wan, Yuemei Xu
Findings of ACL 2023 [pdf] [code]DialSummEval: Revisiting Summarization Evaluation for Dialogues
Mingqi Gao, Xiaojun Wan
NAACL 2022 [pdf] [code]
Academic Services
Served as a reviewer for:
- Conferences: AAAI 2023, EMNLP 2023, ACL Rolling Review 2023-2024, ICLR 2025.
- Workshops: HumEval @ RANLP 2023, LLMAgents @ ICLR 2024.