avatar

Minhak Song

Undergraduate Student
KAIST
minhaksong (at) kaist.ac.kr


About Me

Hi! I am an undergraduate student majoring in Mathematical Sciences at KAIST, where I am fortunate to be advised by Chulhee Yun. I was also a visiting student at the University of Washington, working with Simon Shaolei Du and Sewoong Oh.

My research interests broadly span the foundations of deep learning, with the goal of bridging theory and practice. Recently, I have been focusing on understanding the optimization dynamics in deep learning, particularly in the pre-training and post-training of language models, and leveraging these insights to design principled and efficient optimization algorithms.

Update: Starting in Fall 2026, I will join Stanford University as a PhD student in Computer Science, supported by the Stanford School of Engineering Fellowship.

Research Interests

News

Publications

(* denotes equal contribution)

  1. Zeroth-Order Optimization at the Edge of Stability
    Minhak Song, Liang Zhang, Bingcong Li, Niao He, Michael Muehlebach, Sewoong Oh
    International Conference on Machine Learning (ICML) 2026
    ICLR 2026 Workshop on Scientific Methods for Understanding Deep Learning (Oral)
    arXiv
  2. Understanding the Performance Gap in Preference Learning: A Dichotomy of RLHF and DPO
    Ruizhe Shi*, Minhak Song*, Runlong Zhou, Zihan Zhang, Maryam Fazel, Simon S. Du
    International Conference on Machine Learning (ICML) 2026
    arXiv
  3. Implicit Bias of Per-Sample Adam on Separable Data: Departure from the Full-Batch Regime
    Beomhan Baek*, Minhak Song*, Chulhee Yun
    International Conference on Learning Representations (ICLR) 2026
    NeurIPS 2025 Workshop on Optimization for Machine Learning
    Paper arXiv
  4. Suspicious Alignment of SGD: A Fine-Grained Step Size Condition Analysis
    Shenyang Deng, Boyao Liao, Zhuoli Ouyang, Tianyu Pang, Minhak Song, Yaoqing Yang
    International Conference on Algorithmic Learning Theory (ALT) 2026 (Best Student Paper)
    Paper arXiv
  5. Through the River: Understanding the Benefit of Schedule-Free Methods for Language Model Training
    Minhak Song*, Beomhan Baek*, Kwangjun Ahn, Chulhee Yun
    Neural Information Processing Systems (NeurIPS) 2025
    ICML 2025 Workshop on High-dimensional Learning Dynamics
    Paper arXiv
  6. Understanding Sharpness Dynamics in NN Training with a Minimalist Example: The Effects of Dataset Difficulty, Depth, Stochasticity, and More
    Geonhui Yoo, Minhak Song, Chulhee Yun
    International Conference on Machine Learning (ICML) 2025
    Paper arXiv
  7. Does SGD really happen in tiny subspaces?
    Minhak Song, Kwangjun Ahn, Chulhee Yun
    International Conference on Learning Representations (ICLR) 2025
    ICML 2024 Workshop on High-dimensional Learning Dynamics
    Paper arXiv
  8. Linear attention is (maybe) all you need (to understand Transformer optimization)
    Kwangjun Ahn*, Xiang Cheng*, Minhak Song*, Chulhee Yun, Ali Jadbabaie, Suvrit Sra
    International Conference on Learning Representations (ICLR) 2024
    NeurIPS 2023 Workshop on Mathematics of Modern Machine Learning (Oral)
    Paper arXiv
  9. Trajectory Alignment: Understanding the Edge of Stability Phenomenon via Bifurcation Theory
    Minhak Song, Chulhee Yun
    Neural Information Processing Systems (NeurIPS) 2023
    Paper arXiv

Services

Conference/Workshop Reviewer