Optimization & Theory

Stochastic Collapse: How Gradient Noise Attracts SGD Dynamics Towards Simpler Subnetworks NeurIPS 2023 Stochastic Gradient Descent-Induced Drift of Representation in a Two-Layer Neural Network ICML 2023 Straightening Out the Straight-Through Estimator: Overcoming Optimization Challenges in Vector Quantized Networks ICML 2023 StreamNet: Memory-Efficient Streaming Tiny Deep Learning Inference on the Microcontroller NeurIPS 2023 Strong and Efficient Baselines for Open Domain Conversational Question Answering EMNLP 2023 Structured Pruning for Efficient Generative Pre-trained Language Models ACL 2023 Sub-network Discovery and Soft-masking for Continual Learning of Mixed Tasks EMNLP 2023 SUBP: Soft Uniform Block Pruning for 1×N Sparse CNNs Multithreading Acceleration NeurIPS 2023 Subspace Chronicles: How Linguistic Information Emerges, Shifts and Interacts during Language Model Training EMNLP 2023 SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionality NeurIPS 2023 SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient ICML 2023 Symbolic Discovery of Optimization Algorithms NeurIPS 2023 Synthetic data for model selection ICML 2023 TACCL: Guiding Collective Algorithm Synthesis using Communication Sketches NSDI 2023 TaskMet: Task-driven Metric Learning for Model Learning NeurIPS 2023 TCRA-LLM: Token Compression Retrieval Augmented Large Language Model for Inference Cost Reduction EMNLP 2023 Teacher Intervention: Improving Convergence of Quantization Aware Training for Ultra-Low Precision Transformers EACL 2023 Temporal Dynamic Quantization for Diffusion Models NeurIPS 2023 Tensor Compressive Sensing Fused Low-Rankness and Local-Smoothness AAAI 2023 Testing the Channels of Convolutional Neural Networks AAAI 2023 TexQ: Zero-shot Network Quantization with Texture Feature Distribution Calibration NeurIPS 2023 The Analysis of Deep Neural Networks by Information Theory: From Explainability to Generalization AAAI 2023 The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networks NeurIPS 2023 The Cost of Compression: Investigating the Impact of Compression on Parametric Knowledge in Language Models EMNLP 2023 The Emergence of Essential Sparsity in Large Pre-trained Models: The Weights that Matter NeurIPS 2023