Yifei Wang
I am a researcher at OpenAI, working on self-improving LLMs.
My ever-lasting research interest is in self-supervised learning (SSL), i.e., how to learn from vast unlabeled data by creating supervision from the data itself. I developed some theories and methods of SSL for pretraining (contrastive, non-contrastive, reconstructive, and autoregressive), long-context learning (LongPPL), reasoning (self-correction, CoT scaling), and reinforcement learning (G1, SSL4RL, and IW-OPD). In addition, I enjoy thinking about puzzles in neural architectures, sparsity, and AI safety.
I was a postdoc at MIT CSAIL (2023-2025), advised by Stefanie Jegelka. I did my Ph.D in applied math at Peking University, advised by Yisen Wang, Zhouchen Lin, and Jiansheng Yang. I also did my B.S. in math and B.A. in philosophy at Peking University.
news
| December, 2025 | Our paper G1 has received the Best Paper Award at the NeurIPS 2025 NPGML Workshop. |
|---|---|
| August, 2025 | I gave an invited talk Two New Dimensions of Sparsity for Scaling LLMs at Google DeepMind on sparse long-context training (ICLR 2025) and sparse embedding (ICML 2025 Oral). |
| June, 2025 | Our ICML 2025 paper was featured in an MIT News article, Unpacking the bias of large language models, where we identified and theoretically proved the root causes of position bias in Transformers. |
| June, 2025 | I gave an invited talk at the ASAP Seminar on Your Next-Token Prediction and Transformers Are Biased for Long-Context Modeling—see the recording at YouTube. |
| April, 2025 | Our recent work When More is Less: Understanding Chain-of-Thought Length in LLMs received the Best Paper Runner-Up Award 🏆 at ICLR 2025 Workshop on Reasoning and Planning for LLMs. |
| April, 2025 | I will give a tutorial on the Principles of Self-supervised Learning in the Foundation Model Era at IJCAI 2025 (Aug 16 - Aug 22). |
selected papers (see full publication)
- ICLR
Best Paper Runner-up When More is Less: Understanding Chain-of-Thought Length in LLMsICLR, 2026🏆 Best Paper Runner-up Award at ICLR 2025 Workshop on Reasoning and Planning for LLMs - NeurIPS
Best Paper Award
at ICML-W’24A Theoretical Understanding of Self-Correction through In-context AlignmentIn NeurIPS, 2024🏆 Best Paper Award at ICML 2024 ICL Workshop
We proposed the first theoretical explanation of how LLM self-correction works (as in OpenAI o1) and showed its effectiveness against social bias and jailbreak attacks. - ICLR
JMLR Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation OverlapIn ICLR, 2022A new augmentation overlap theory for understanding the generalization of contrastive learning. Cited over 150 times. Extended version was accepted at JMLR.