avatar

Minhak Song

Undergraduate Student
KAIST
minhaksong (at) kaist.ac.kr


About Me

Hi! My name is Minhak Song, and I am an undergraduate student at KAIST, majoring in Mathematical Sciences (minor in Industrial and Systems Engineering). My research interests center on the foundations of modern machine learning, spanning the theory of deep learning, language models, generative models, and interactive decision making, with the goal of bridging theory and practice.

I have recently been studying the training dynamics of optimization algorithms in deep learning, advised by Prof. Chulhee Yun at KAIST AI and closely collaborating with Kwangjun Ahn at Microsoft Research. I was also a visiting student researcher at the University of Washington, hosted by Prof. Simon Shaolei Du, where I worked on reinforcement learning from human feedback (RLHF) from an optimization perspective. Starting this summer, I am working with Prof. Sewoong Oh at the University of Washington on zeroth-order optimization.

I’m always happy to discuss research and potential collaborations. Feel free to reach out!

Research Interests

News

Publications

(* denotes equal contribution)

  1. Implicit Bias of Per-sample Adam on Separable Data: Departure from the Full-batch Regime
    Beomhan Baek*, Minhak Song*, Chulhee Yun
    Manuscript, 2025.
    NeurIPS 2025 Workshop on Optimization for Machine Learning.
    arXiv
  2. Understanding the Performance Gap in Preference Learning: A Dichotomy of RLHF and DPO
    Ruizhe Shi*, Minhak Song*, Runlong Zhou, Zihan Zhang, Maryam Fazel, Simon Du
    Manuscript, 2025.
    arXiv
  3. Through the River: Understanding the Benefit of Schedule-Free Methods for Language Model Training
    Minhak Song*, Beomhan Baek*, Kwangjun Ahn, Chulhee Yun
    Neural Information Processing Systems (NeurIPS), 2025.
    ICML 2025 Workshop on High-dimensional Learning Dynamics.
    Paper arXiv
  4. Understanding Sharpness Dynamics in NN Training with a Minimalist Example: The Effects of Dataset Difficulty, Depth, Stochasticity, and More
    Geonhui Yoo, Minhak Song, Chulhee Yun
    International Conference on Machine Learning (ICML), 2025.
    Paper arXiv
  5. Does SGD really happen in tiny subspaces?
    Minhak Song, Kwangjun Ahn, Chulhee Yun
    International Conference on Learning Representations (ICLR), 2025.
    ICML 2024 Workshop on High-dimensional Learning Dynamics.
    Paper arXiv
  6. Linear attention is (maybe) all you need (to understand Transformer optimization)
    Kwangjun Ahn*, Xiang Cheng*, Minhak Song*, Chulhee Yun, Ali Jadbabaie, Suvrit Sra
    International Conference on Learning Representations (ICLR), 2024.
    NeurIPS 2023 Workshop on Mathematics of Modern Machine Learning (Oral).
    Paper arXiv
  7. Trajectory Alignment: Understanding the Edge of Stability Phenomenon via Bifurcation Theory
    Minhak Song, Chulhee Yun
    Neural Information Processing Systems (NeurIPS), 2023.
    Paper arXiv

Services

Conference/Workshop Reviewer