Hello! I am a postdoctoral researcher at MIT CSAIL working with Stefanie Jegelka. I obtained my PhD in Applied Mathematics from Peking University (PKU) in 2023, advised by Yisen Wang, Jiansheng Yang, and Zhouchen Lin. Prior to that, I got my bachelor's degrees also from PKU math.

I am generally interested in modern machine learning and representation learning, with a focus on the following areas:

  • Self-supervised Learning (SSL). SSL is the driving engine of foundation models during the pretraining stage. I am mostly interested in theoretically understanding and principled design of SSL (e.g., contrastive / masked / autoregresive training), aimed at uncovering their inherent mechanisms and enhancing model interpretability.
  • Adversarial Learning. Powerful LLMs need to be aligned to human purposes with guardrails to avoid being abused. I explore when existing alignment measures will fail (e.g., by adversarial attacks or jailbreaks), and how to systematically develop robust foundation models against adversarial and real-world distribution shifts.
  • Neural architectures. I am interested in understanding the inherent mechanisms of backbone neural architectures, such as, Transformers and Graph Neural Networks.
Feel free to shoot me an email if you are interested in working with me.

Contact: yifei_w at mit.edu / Google Scholar / Github / X (Twitter)

News

  • 2024.04. Gave a talk on Non-negative Contrastive Learning at MIT LIDS Tea (slides).
  • 2024.04. Anthropic proposed many-shot jailbreaking, showing that extending our In-context Attack (ICA) from a few shots (5) to many shots (256) can jailbreak most prominent LLMs (GPT-3.5/4, Claude2, Llama2-70B).
  • 2024.02. I was honored to receive Wenjun Wu Outstanding Ph.D. Dissertation Runner-Up Award (top 14 nation-wide) from CAAI. Wenjun Wu invented Wu's method for automatic theorem proving and pioneered AI research in China. Thanks everyone!
  • 2023.12. I have joined MIT CSAIL as a postdoc.
  • 2023.09. I will be serving as an Area Chair for ICLR 2024.

Publications (* marks equal contribution)

  • Look Ahead or Look Around? A Theoretical Comparison Between Autoregressive and Masked Pretraining Qi Zhang, Tianqi Du, Haotian Huang, Yifei Wang, Yisen Wang ICML 2024 2024 PDF
  • OODRobustBench: a benchmark and large-scale analysis of adversarial robustness under distribution shift Lin Li, Yifei Wang, Chawin Sitawarin, Michael W. Spratling ICML 2024 2024 PDF
  • On the Duality Between Sharpness-Aware Minimization and Adversarial Training Yihao Zhang, Hangzhou He, Jingyu Zhu, Huanran Chen, Yifei Wang, Zeming Wei ICML 2024 2024 PDF
  • Non-negative Contrastive Learning Yifei Wang*, Qi Zhang*, Yaoyu Guo, Yisen Wang ICLR 2024 2024 PDF | Code | Slides
  • Do Generated Data Always Help Contrastive Learning? Yifei Wang*, Jizhe Zhang*, Yisen Wang ICLR 2024 2024 PDF | Code | Featured on Sync (CN)
  • On the Role of Discrete Tokenization in Visual Representation Learning Tianqi Du*, Yifei Wang*, Yisen Wang ICLR 2024 (Spotlight) 2024 PDF | Code
  • How to Craft Backdoors with Unlabeled Data Alone? Yifei Wang*, Wenhan Ma*, Stefanie Jegelka, Yisen Wang ICLR 2024 Workshop on Navigating and Addressing Data Problems for Foundation Models (DPFM) 2024 PDF
  • Virtual Classifier: A Reversed Approach for Robust Image Evaluation Jizhe Zhang*, Yifei Wang*, Yisen Wang ICLR 2024 Workshop on Navigating and Addressing Data Problems for Foundation Models (DPFM) 2024 PDF
  • Balance, Imbalance, and Rebalance: Understanding Robust Overfitting from a Minimax Game Perspective Yifei Wang*, Liangchen Li*, Jiansheng Yang, Zhouchen Lin, Yisen Wang NeurIPS 2023 2023 PDF | Code
  • Adversarial Examples Are Not Real Features Ang Li*, Yifei Wang*, Yiwen Guo, Yisen Wang NeurIPS 2023 2023 PDF | Code
  • Architecture Matters: Uncovering Implicit Mechanisms in Graph Contrastive Learning Xiaojun Guo*, Yifei Wang*, Zeming Wei, Yisen Wang NeurIPS 2023 2023 PDF | Code
  • Identifiable Contrastive Learning with Automatic Feature Importance Discovery Qi Zhang*, Yifei Wang*, Yisen Wang NeurIPS 2023 2023 PDF | Code
  • Laplacian Canonization: A Minimalist Approach to Sign and Basis Invariant Spectral Embedding George Ma*, Yifei Wang*, Yisen Wang NeurIPS 2023 2023 PDF | Code
  • On the Generalization of Multi-modal Contrastive Learning Qi Zhang*, Yifei Wang*, Yisen Wang ICML 2023 2023 PDF | Code
  • Rethinking Weak Supervision in Helping Contrastive Representation Learning Jingyi Cui*, Weiran Huang*, Yifei Wang*, Yisen Wang ICML 2023 2023 PDF
  • CFA: Class-wise Calibrated Fair Adversarial Training Zeming Wei, Yifei Wang, Yiwen Guo, Yisen Wang CVPR 2023 2023 PDF | Code
  • Equilibrium Image Denoising with Implicit Differentiation Qi Chen, Yifei Wang, Zhengyang Geng, Yisen Wang, Jiansheng Yang, and Zhouchen Lin IEEE Transactions on Image Processing (TIP) 2023 PDF
  • A Message Passing Perspective on Learning Dynamics of Contrastive Learning Yifei Wang*, Qi Zhang*, Tianqi Du, Jiansheng Yang, Zhouchen Lin, Yisen Wang ICLR 2023 2023 PDF | Code | Slides | Blog
  • Towards a Unified Theoretical Understanding of Non-contrastive Learning via Rank Differential Mechanism Zhijian Zhuo*, Yifei Wang*, Jinwen Ma, Yisen Wang ICLR 2023 2023 PDF | Code
  • Rethinking the Effect of Data Augmentation in Adversarial Contrastive Learning Rundong Luo*, Yifei Wang*, Yisen Wang ICLR 2023 2023 PDF | Code
  • ContraNorm: A Contrastive Learning Perspective on Oversmoothing and Beyond Xiaojun Guo*, Yifei Wang*, Tianqi Du*, Yisen Wang ICLR 2023 2023 PDF | Code
  • Unbiased Stochastic Proximal Solver for Graph Neural Networks with Equilibrium States Mingjie Li, Yifei Wang, Yisen Wang, Zhouchen Lin ICLR 2023 2023 PDF
  • What Contrastive Learning Learns Beyond Class-wise Features? Xingyuming Liu, Yifei Wang, Yisen Wang ICLR 2023 Workshop on Mathematical and Empirical Understanding of Foundation Models (ME-FoMo) 2023 PDF
  • Rethinking the Necessity of Labels in Backdoor Defense Zidi Xiong, Dongxian Wu, Yifei Wang, Yisen Wang ICLR 2023 Workshop on Backdoor Attacks and Defenses in Machine Learning (BANDS) 2023 PDF
  • On the Connection between Invariant Learning and Adversarial Training for Out-of-Distribution Generalization Shiji Xin, Yifei Wang, Jingtong Su, Yisen Wang AAAI 2023 (Oral) 2023 PDF
  • How Mask Matters: Towards Theoretical Understandings of Masked Autoencoders Qi Zhang*, Yifei Wang*, Yisen Wang NeurIPS 2022 (Spotlight, Top 5%) 2022 PDF | Code | Slides
  • Improving Out-of-distribution Robustness by Adversarial Training with Structured Priors Qixun Wang*, Yifei Wang*, Hong Zhu, Yisen Wang NeurIPS 2022 (Spotlight, Top 5%) 2022 PDF | Code | Slides
  • When Adversarial Training Meets Vision Transformers: Recipes from Training to Architecture Yichuan Mo, Dongxian Wu, Yifei Wang, Yiwen Guo, Yisen Wang NeurIPS 2022 (Spotlight, Top 5%) 2022 PDF | Code
  • Variational Energy-Based Models: A Probabilistic Framework for Contrastive Self-Supervised Learning Tianqi Du*, Yifei Wang*, Weiran Huang, Yisen Wang NeurIPS 2022 SSL Workshop 2022 PDF
  • AggNCE: Asymptotically Identifiable Contrastive Learning Jingyi Cui*, Weiran Huang*, Yifei Wang, Yisen Wang NeurIPS 2022 SSL Workshop (Oral) 2022 PDF
  • Efficient and Scalable Implicit Graph Neural Networks with Virtual Equilibrium Qi Chen, Yifei Wang, Yisen Wang, Jianlong Chang, Qi Tian, Jiansheng Yang, Zhouchen Lin IEEE BigData 2022 (Long Talk) 2022 PDF
  • Optimization-Induced Graph Implicit Nonlinear Diffusion Qi Chen, Yifei Wang, Yisen Wang, Jiansheng Yang, Zhouchen Lin ICML 2022 2022 PDF | Code
  • G2CN: Graph Gaussian Convolution Networks with Concentrated Graph Filters Mingjie Li, Xiaojun Guo, Yifei Wang, Yisen Wang, Zhouchen Lin ICML 2022 2022 PDF
  • Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap Yifei Wang*, Qi Zhang*, Yisen Wang, Jiansheng Yang, Zhouchen Lin ICLR 2022 2022 PDF | Code | Slides
  • A Unified Contrastive Energy-based Model for Understanding the Generative Ability of Adversarial Training Yifei Wang, Yisen Wang, Jiansheng Yang, Zhouchen Lin ICLR 2022 (πŸ† Silver Best Paper Award @ ICML 2021 AdvML workshop) 2022 PDF | Slides | Award
  • Residual Relaxation for Multi-view Representation Learning Yifei Wang, Zhengyang Geng, Feng Jiang, Chuming Li, Yisen Wang, Jiansheng Yang, Zhouchen Lin NeurIPS 2021 2021 PDF | Slides | Blog
  • Dissecting the Diffusion Process in Linear Graph Convolutional Networks Yifei Wang, Yisen Wang, Jiansheng Yang, Zhouchen Lin NeurIPS 2021 2021 PDF | Code | Slides | Blog
  • Reparameterized Sampling for Generative Adversarial Networks Yifei Wang, Yisen Wang, Jiansheng Yang, Zhouchen Lin ECML-PKDD 2021 2021 (πŸ† Best ML Paper Award (1/685) & Invited to Machine Learning Journal) PDF | Code | Slides | Media | Talk | Award
  • Train Once, and Decode as You Like Chao Tian, Yifei Wang, Hao Cheng, Yijiang Lian, Zhihua Zhang COLING 2020 2020 PDF
  • Look Ahead or Look Around? A Theoretical Comparison Between Autoregressive and Masked Pretraining Qi Zhang, Tianqi Du, Haotian Huang, Yifei Wang, Yisen Wang ICML 2024 2024 PDF
  • Non-negative Contrastive Learning Yifei Wang*, Qi Zhang*, Yaoyu Guo, Yisen Wang ICLR 2024 2024 PDF | Code | Slides
  • Do Generated Data Always Help Contrastive Learning? Yifei Wang*, Jizhe Zhang*, Yisen Wang ICLR 2024 2024 PDF | Code | Featured on Sync (CN)
  • On the Role of Discrete Tokenization in Visual Representation Learning Tianqi Du*, Yifei Wang*, Yisen Wang ICLR 2024 (Spotlight) 2024 PDF | Code
  • How to Craft Backdoors with Unlabeled Data Alone? Yifei Wang*, Wenhan Ma*, Stefanie Jegelka, Yisen Wang ICLR 2024 Workshop on Navigating and Addressing Data Problems for Foundation Models (DPFM) 2024 PDF
  • Virtual Classifier: A Reversed Approach for Robust Image Evaluation Jizhe Zhang*, Yifei Wang*, Yisen Wang ICLR 2024 Workshop on Navigating and Addressing Data Problems for Foundation Models (DPFM) 2024 PDF
  • Architecture Matters: Uncovering Implicit Mechanisms in Graph Contrastive Learning Xiaojun Guo*, Yifei Wang*, Zeming Wei, Yisen Wang NeurIPS 2023 2023 PDF | Code
  • Identifiable Contrastive Learning with Automatic Feature Importance Discovery Qi Zhang*, Yifei Wang*, Yisen Wang NeurIPS 2023 2023 PDF | Code
  • On the Generalization of Multi-modal Contrastive Learning Qi Zhang*, Yifei Wang*, Yisen Wang ICML 2023 2023 PDF | Code
  • Rethinking Weak Supervision in Helping Contrastive Representation Learning Jingyi Cui*, Weiran Huang*, Yifei Wang*, Yisen Wang ICML 2023 2023 PDF
  • A Message Passing Perspective on Learning Dynamics of Contrastive Learning Yifei Wang*, Qi Zhang*, Tianqi Du, Jiansheng Yang, Zhouchen Lin, Yisen Wang ICLR 2023 2023 PDF | Code | Slides | Blog
  • Towards a Unified Theoretical Understanding of Non-contrastive Learning via Rank Differential Mechanism Zhijian Zhuo*, Yifei Wang*, Jinwen Ma, Yisen Wang ICLR 2023 2023 PDF | Code
  • Rethinking the Effect of Data Augmentation in Adversarial Contrastive Learning Rundong Luo*, Yifei Wang*, Yisen Wang ICLR 2023 2023 PDF | Code
  • ContraNorm: A Contrastive Learning Perspective on Oversmoothing and Beyond Xiaojun Guo*, Yifei Wang*, Tianqi Du*, Yisen Wang ICLR 2023 2023 PDF | Code
  • What Contrastive Learning Learns Beyond Class-wise Features? Xingyuming Liu, Yifei Wang, Yisen Wang ICLR 2023 Workshop on Mathematical and Empirical Understanding of Foundation Models (ME-FoMo) 2023 PDF
  • Rethinking the Necessity of Labels in Backdoor Defense Zidi Xiong, Dongxian Wu, Yifei Wang, Yisen Wang ICLR 2023 Workshop on Backdoor Attacks and Defenses in Machine Learning (BANDS) 2023 PDF
  • How Mask Matters: Towards Theoretical Understandings of Masked Autoencoders Qi Zhang*, Yifei Wang*, Yisen Wang NeurIPS 2022 (Spotlight, Top 5%) 2022 PDF | Code | Slides
  • Variational Energy-Based Models: A Probabilistic Framework for Contrastive Self-Supervised Learning Tianqi Du*, Yifei Wang*, Weiran Huang, Yisen Wang NeurIPS 2022 SSL Workshop 2022 PDF
  • AggNCE: Asymptotically Identifiable Contrastive Learning Jingyi Cui*, Weiran Huang*, Yifei Wang, Yisen Wang NeurIPS 2022 SSL Workshop (Oral) 2022 PDF
  • Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap Yifei Wang*, Qi Zhang*, Yisen Wang, Jiansheng Yang, Zhouchen Lin ICLR 2022 2022 PDF | Code | Slides
  • A Unified Contrastive Energy-based Model for Understanding the Generative Ability of Adversarial Training Yifei Wang, Yisen Wang, Jiansheng Yang, Zhouchen Lin ICLR 2022 (πŸ† Silver Best Paper Award @ ICML 2021 AdvML workshop) 2022 PDF | Slides | Award
  • Residual Relaxation for Multi-view Representation Learning Yifei Wang, Zhengyang Geng, Feng Jiang, Chuming Li, Yisen Wang, Jiansheng Yang, Zhouchen Lin NeurIPS 2021 2021 PDF | Slides | Blog
  • Reparameterized Sampling for Generative Adversarial Networks Yifei Wang, Yisen Wang, Jiansheng Yang, Zhouchen Lin ECML-PKDD 2021 2021 (πŸ† Best ML Paper Award (1/685). Invited to Machine Learning Journal) PDF | Code | Slides | Media | Talk | Award
  • OODRobustBench: a benchmark and large-scale analysis of adversarial robustness under distribution shift Lin Li, Yifei Wang, Chawin Sitawarin, Michael W. Spratling ICML 2024 2024 PDF
  • On the Duality Between Sharpness-Aware Minimization and Adversarial Training Yihao Zhang, Hangzhou He, Jingyu Zhu, Huanran Chen, Yifei Wang, Zeming Wei ICML 2024 2024 PDF
  • How to Craft Backdoors with Unlabeled Data Alone? Yifei Wang*, Wenhan Ma*, Stefanie Jegelka, Yisen Wang ICLR 2024 Workshop on Navigating and Addressing Data Problems for Foundation Models (DPFM) 2024 PDF
  • Adversarial Examples Are Not Real Features Ang Li*, Yifei Wang*, Yiwen Guo, Yisen Wang NeurIPS 2023 2023 PDF | Code
  • Balance, Imbalance, and Rebalance: Understanding Robust Overfitting from a Minimax Game Perspective Yifei Wang*, Liangchen Li*, Jiansheng Yang, Zhouchen Lin, Yisen Wang NeurIPS 2023 2023 PDF | Code
  • CFA: Class-wise Calibrated Fair Adversarial Training Zeming Wei, Yifei Wang, Yiwen Guo, Yisen Wang CVPR 2023 2023 PDF | Code
  • Rethinking the Effect of Data Augmentation in Adversarial Contrastive Learning Rundong Luo*, Yifei Wang*, Yisen Wang ICLR 2023 2023 PDF
  • Rethinking the Necessity of Labels in Backdoor Defense Zidi Xiong, Dongxian Wu, Yifei Wang, Yisen Wang ICLR 2023 Workshop on Backdoor Attacks and Defenses in Machine Learning (BANDS) 2023 PDF
  • On the Connection between Invariant Learning and Adversarial Training for Out-of-Distribution Generalization Shiji Xin, Yifei Wang, Jingtong Su, Yisen Wang AAAI 2023 (Oral) 2023 PDF
  • Improving Out-of-distribution Robustness by Adversarial Training with Structured Priors Qixun Wang*, Yifei Wang*, Hong Zhu, Yisen Wang NeurIPS 2022 (Spotlight, Top 5%) 2022 PDF | Code | Slides
  • When Adversarial Training Meets Vision Transformers: Recipes from Training to Architecture Yichuan Mo, Dongxian Wu, Yifei Wang, Yiwen Guo, Yisen Wang NeurIPS 2022 (Spotlight, Top 5%) 2022 PDF | Code
  • A Unified Contrastive Energy-based Model for Understanding the Generative Ability of Adversarial Training Yifei Wang, Yisen Wang, Jiansheng Yang, Zhouchen Lin ICLR 2022 (πŸ† Silver Best Paper Award @ ICML 2021 AdvML workshop) 2022 PDF | Slides | Award
  • Laplacian Canonization: A Minimalist Approach to Sign and Basis Invariant Spectral Embedding George Ma*, Yifei Wang*, Yisen Wang NeurIPS 2023 2023 PDF | Code
  • Architecture Matters: Uncovering Implicit Mechanisms in Graph Contrastive Learning Xiaojun Guo*, Yifei Wang*, Zeming Wei, Yisen Wang NeurIPS 2023 2023 PDF | Code
  • ContraNorm: A Contrastive Learning Perspective on Oversmoothing and Beyond Xiaojun Guo*, Yifei Wang*, Tianqi Du*, Yisen Wang ICLR 2023 2023 PDF
  • Unbiased Stochastic Proximal Solver for Graph Neural Networks with Equilibrium States Mingjie Li, Yifei Wang, Yisen Wang, Zhouchen Lin ICLR 2023 2023 PDF
  • Efficient and Scalable Implicit Graph Neural Networks with Virtual Equilibrium Qi Chen, Yifei Wang, Yisen Wang, Jianlong Chang, Qi Tian, Jiansheng Yang, Zhouchen Lin IEEE BigData 2022 (Long Talk) 2022 PDF
  • Optimization-Induced Graph Implicit Nonlinear Diffusion Qi Chen, Yifei Wang, Yisen Wang, Jiansheng Yang, Zhouchen Lin ICML 2022 2022 PDF | Code
  • G2CN: Graph Gaussian Convolution Networks with Concentrated Graph Filters Mingjie Li, Xiaojun Guo, Yifei Wang, Yisen Wang, Zhouchen Lin ICML 2022 2022 PDF
  • Dissecting the Diffusion Process in Linear Graph Convolutional Networks Yifei Wang, Yisen Wang, Jiansheng Yang, Zhouchen Lin NeurIPS 2021 2021 PDF | Code | Slides | Blog

Selected Awards

Wenjun Wu Outstanding Ph.D. Dissertation Runner-Up Award (top 14 nation-wide), CAAI, 2024
Excellent Graduate (top 1 per department), Beijing, 2023
Excellent Graduate, Peking University, 2023
Baidu Scholarship Runner-Up (top 20 nation-wide), Baidu Inc, 2022
National Scholarship (top 0.1% nation-wide), China, 2021, 2022
Principal Scholarship (top 1% university-wide), Peking University, 2022
Best ML Paper Award (1/685), ECML-PKDD, 2021
Silver Best Paper Award, ICML AdvML workshop, 2021

Professional Services

Reviewer and/or program commitee member:
  • ML Conferences: NeurIPS (2022, 2023), ICML (2022), AISTATS (2024), LoG (2023), ECML-PKDD (2022)
  • Other conferences: CVPR (2023, 2024), ICCV (2023), ACL (2020, 2021)
  • Journal: IEEE TPAMI, TMLR
Senior roles:
  • Area Chair & Session Chair at ICLR 2024