ICML 2024 Workshop HiLD Submissions

A Universal Class of Sharpness-Aware Minimization Algorithms
Behrooz Tahmasebi, Ashkan Soleymani, Dara Bahri, Stefanie Jegelka, Patrick Jaillet
- Published: 16 Jun 2024, Last Modified: 20 Jul 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone
Hidden Learning Dynamics of Capability before Behavior in Diffusion Models
Core Francisco Park, Maya Okawa, Andrew Lee, Ekdeep Singh Lubana, Hidenori Tanaka
- Published: 16 Jun 2024, Last Modified: 16 Jun 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone
Why Pruning and Conditional Computation Work: A High-Dimensional Perspective
Erdem Koyuncu
- Published: 16 Jun 2024, Last Modified: 21 Jul 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone
Neural network learns low-dimensional polynomials with SGD near the information-theoretic limit
Jason D. Lee, Kazusato Oko, Taiji Suzuki, Denny Wu
- Published: 16 Jun 2024, Last Modified: 08 Jul 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone
The Butterfly Effect: Tiny Perturbations Cause Neural Network Training to Diverge
Gül Sena Altıntaş, Devin Kwok, David Rolnick
- Published: 16 Jun 2024, Last Modified: 20 Jul 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone
Adam Exploits $\ell_\infty$-geometry of Loss Landscape via Coordinate-wise Adaptivity
Shuo Xie, Mohamad Amin Mohamadi, Zhiyuan Li
- Published: 16 Jun 2024, Last Modified: 16 Jun 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone
A Hessian-Aware Stochastic Differential Equation for Modelling SGD
Xiang Li, Zebang Shen, Liang Zhang, Niao He
- Published: 16 Jun 2024, Last Modified: 16 Jun 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone
InfoNCE: Identifying the Gap Between Theory and Practice
Evgenia Rusak, Patrik Reizinger, Attila Juhos, Oliver Bringmann, Roland S. Zimmermann, Wieland Brendel
- Published: 16 Jun 2024, Last Modified: 16 Jul 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone
Feature Learning Dynamics under Grokking in a Sparse Parity Task
Javier Sanguino Bautiste, Gregor Bachmann, Bobby He, Lorenzo Noci, Thomas Hofmann
- Published: 16 Jun 2024, Last Modified: 29 Jan 2025
- HiLD at ICML 2024 Poster
- Readers: Everyone
SGD vs GD: Rank Deficiency in Linear Networks
Aditya Varre, Margarita Sagitova, Nicolas Flammarion
- Published: 16 Jun 2024, Last Modified: 19 Jun 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone
Exploring the development of complexity over depth and time in deep neural networks
Hannah Pinson, Aurélien Boland, Vincent Ginis, Mykola Pechenizkiy
- Published: 16 Jun 2024, Last Modified: 20 Jul 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone
Neural Symmetry Detection for Learning Neural Network Constraints
Alex Gabel, Rick Quax, Stratis Gavves
- Published: 16 Jun 2024, Last Modified: 16 Jun 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone
Closed form of the Hessian spectrum for some Neural Networks
Sidak Pal Singh, Thomas Hofmann
- Published: 16 Jun 2024, Last Modified: 19 Jul 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone
Landscaping Linear Mode Connectivity
Sidak Pal Singh, Linara Adilova, Michael Kamp, Asja Fischer, Bernhard Schölkopf, Thomas Hofmann
- Published: 16 Jun 2024, Last Modified: 23 Jun 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone
Understanding Adversarially Robust Generalization via Weight-Curvature Index
Yuelin Xu, Xiao Zhang
- Published: 16 Jun 2024, Last Modified: 19 Jun 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone
Gradient Descent Robustly Learns the Intrinsic Dimension of Data in Training Convolutional Neural Networks
Chenyang Zhang, Gao Peifeng, Difan Zou, Yuan Cao
- Published: 16 Jun 2024, Last Modified: 19 Jul 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone
Analyzing & Eliminating Learning Rate Warmup in GPT Pre-Training
Atli Kosson, Bettina Messmer, Martin Jaggi
- Published: 16 Jun 2024, Last Modified: 19 Jul 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone
Random matrix theory analysis of neural network weight matrices
Matthias Thamm, Max Staats, Bernd Rosenow
- Published: 16 Jun 2024, Last Modified: 15 Jul 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone
Boundary between noise and information applied to filtering neural network weight matrices
Max Staats, Matthias Thamm, Bernd Rosenow
- Published: 16 Jun 2024, Last Modified: 02 Jul 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone
The Implicit Bias of Adam on Separable Data
Chenyang Zhang, Difan Zou, Yuan Cao
- Published: 16 Jun 2024, Last Modified: 19 Jul 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone
Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning
Daniel Kunin, Allan Raventos, Clémentine Carla Juliette Dominé, Feng Chen, David Klindt, Andrew M Saxe, Surya Ganguli
- Published: 16 Jun 2024, Last Modified: 11 Jul 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone
Fine-grained Analysis of In-context Linear Estimation
Yingcong Li, Ankit Singh Rawat, Samet Oymak
- Published: 16 Jun 2024, Last Modified: 17 Jul 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone
Gradient Dissent in Language Model Training and Saturation
Andrei Mircea, Ekaterina Lobacheva, Irina Rish
- Published: 16 Jun 2024, Last Modified: 19 Jul 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone
Rank Minimization, Alignment and Weight Decay in Neural Networks
David Yunis, Kumar Kshitij Patel, Samuel Wheeler, Pedro Henrique Pamplona Savarese, Gal Vardi, Karen Livescu, Michael Maire, Matthew Walter
- Published: 16 Jun 2024, Last Modified: 19 Jun 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone
How Truncating Weights Improves Reasoning in Language Models
Lei Chen, Joan Bruna, Alberto Bietti
- Published: 16 Jun 2024, Last Modified: 16 Jun 2024
- HiLD at ICML 2024 Poster
- Readers: Everyone