avatar

Minhak Song

Undergraduate Student
KAIST
minhaksong (at) kaist.ac.kr


About Me

Hi! My name is Minhak Song, and I am an undergraduate student at KAIST, majoring in Mathematical Sciences (minor in Industrial and Systems Engineering). My research interests center on the theoretical foundations of modern machine learning, with the goal of bridging theory and practice.

I have recently been studying the training dynamics of optimization algorithms in deep learning, advised by Prof. Chulhee Yun. I was also a visiting student at the University of Washington, hosted by Prof. Simon Du, where I worked on reinforcement learning from human feedback (RLHF) from an optimization perspective. This summer, I am working with Prof. Sewoong Oh at UW on zeroth-order optimization.

I’m always open to conversations and potential collaborations. Feel free to reach out!

Research Interests

News

Publications

(* denotes equal contribution)

  1. Through the River: Understanding the Benefit of Schedule-Free Methods for Language Model Training
    Minhak Song*, Beomhan Baek*, Kwangjun Ahn, Chulhee Yun
    ICML 2025 Workshop on High-dimensional Learning Dynamics.
    arXiv
  2. Understanding the Performance Gap in Preference Learning: A Dichotomy of RLHF and DPO
    Ruizhe Shi*, Minhak Song*, Runlong Zhou, Zihan Zhang, Maryam Fazel, Simon Du
    Manuscript, 2025.
    arXiv
  3. Understanding Sharpness Dynamics in NN Training with a Minimalist Example: The Effects of Dataset Difficulty, Depth, Stochasticity, and More
    Geonhui Yoo, Minhak Song, Chulhee Yun
    International Conference on Machine Learning (ICML), 2025.
    Paper arXiv
  4. Does SGD really happen in tiny subspaces?
    Minhak Song, Kwangjun Ahn, Chulhee Yun
    International Conference on Learning Representations (ICLR), 2025.
    ICML 2024 Workshop on High-dimensional Learning Dynamics.
    Paper arXiv
  5. Linear attention is (maybe) all you need (to understand Transformer optimization)
    Kwangjun Ahn*, Xiang Cheng*, Minhak Song*, Chulhee Yun, Ali Jadbabaie, Suvrit Sra
    International Conference on Learning Representations (ICLR), 2024.
    NeurIPS 2023 Workshop on Mathematics of Modern Machine Learning (Oral).
    Paper arXiv
  6. Trajectory Alignment: Understanding the Edge of Stability Phenomenon via Bifurcation Theory
    Minhak Song, Chulhee Yun
    Neural Information Processing Systems (NeurIPS), 2023.
    Paper arXiv

Services

Conference/Workshop Reviewer