ICML 2026 Highlights Model Efficiency Breakthrough: Selective Activation Sparsity Achieves 3x Size Reduction
Research presented at ICML 2026 introduced selective activation sparsity, a training method enabling models to use only the most relevant parameters for specific tasks. Models trained with this technique matched the performance of models three times their size on reasoning benchmarks, with significant implications for training and inference costs on resource-constrained devices like phones and laptops.