avatar

Minhak Song

Undergraduate Student
KAIST
minhaksong (at) kaist.ac.kr


About Me

Hi! My name is Minhak Song, and I am an undergraduate student at KAIST, majoring in Mathematical Sciences (minor in Industrial and Systems Engineering). My research interests center on the foundations of modern machine learning, spanning the theory of deep learning, language models, generative models, and interactive decision making, with the goal of bridging theory and practice.

I have recently been studying the training dynamics of optimization algorithms in deep learning, advised by Prof. Chulhee Yun at KAIST AI and closely collaborating with Kwangjun Ahn at Microsoft Research. I was also a visiting student researcher at the University of Washington, hosted by Prof. Simon Shaolei Du, where I worked on reinforcement learning from human feedback (RLHF) from an optimization perspective. Starting this summer, I am working with Prof. Sewoong Oh at the University of Washington on zeroth-order optimization.

I’m always happy to discuss research and potential collaborations. Feel free to reach out!

Research Interests

News

Publications

(* denotes equal contribution)

  1. Understanding the Performance Gap in Preference Learning: A Dichotomy of RLHF and DPO
    Ruizhe Shi*, Minhak Song*, Runlong Zhou, Zihan Zhang, Maryam Fazel, Simon S. Du
    Manuscript
    arXiv
  2. Implicit Bias of Per-Sample Adam on Separable Data: Departure from the Full-Batch Regime
    Beomhan Baek*, Minhak Song*, Chulhee Yun
    International Conference on Learning Representations (ICLR) 2026
    NeurIPS 2025 Workshop on Optimization for Machine Learning
    arXiv
  3. Suspicious Alignment of SGD: A Fine-Grained Step Size Condition Analysis
    Shenyang Deng, Boyao Liao, Zhuoli Ouyang, Tianyu Pang, Minhak Song, Yaoqing Yang
    International Conference on Algorithmic Learning Theory (ALT) 2026
    arXiv
  4. Through the River: Understanding the Benefit of Schedule-Free Methods for Language Model Training
    Minhak Song*, Beomhan Baek*, Kwangjun Ahn, Chulhee Yun
    Neural Information Processing Systems (NeurIPS) 2025
    ICML 2025 Workshop on High-dimensional Learning Dynamics
    Paper arXiv
  5. Understanding Sharpness Dynamics in NN Training with a Minimalist Example: The Effects of Dataset Difficulty, Depth, Stochasticity, and More
    Geonhui Yoo, Minhak Song, Chulhee Yun
    International Conference on Machine Learning (ICML) 2025
    Paper arXiv
  6. Does SGD really happen in tiny subspaces?
    Minhak Song, Kwangjun Ahn, Chulhee Yun
    International Conference on Learning Representations (ICLR) 2025
    ICML 2024 Workshop on High-dimensional Learning Dynamics
    Paper arXiv
  7. Linear attention is (maybe) all you need (to understand Transformer optimization)
    Kwangjun Ahn*, Xiang Cheng*, Minhak Song*, Chulhee Yun, Ali Jadbabaie, Suvrit Sra
    International Conference on Learning Representations (ICLR) 2024
    NeurIPS 2023 Workshop on Mathematics of Modern Machine Learning (Oral)
    Paper arXiv
  8. Trajectory Alignment: Understanding the Edge of Stability Phenomenon via Bifurcation Theory
    Minhak Song, Chulhee Yun
    Neural Information Processing Systems (NeurIPS) 2023
    Paper arXiv

Services

Conference/Workshop Reviewer