Optimization & Theory

Transformer Quality in Linear Time ICML 2022 Transformers from an Optimization Perspective NeurIPS 2022 Transformers meet Stochastic Block Models: Attention with Data-Adaptive Sparsity and Cost NeurIPS 2022 Transition to Linearity of General Neural Networks with Directed Acyclic Graph Architecture NeurIPS 2022 Transkimmer: Transformer Learns to Layer-wise Skim ACL 2022 TREC: Transient Redundancy Elimination-based Convolution NeurIPS 2022 Tricks for Training Sparse Translation Models NAACL 2022 TRIPS: Efficient Vision-and-Language Pre-training with Text-Relevant Image Patch Selection EMNLP 2022 Truncation Sampling as Language Model Desmoothing EMNLP 2022 Tutoring Helps Students Learn Better: Improving Knowledge Distillation for BERT with Tutor Network EMNLP 2022 TVConv: Efficient Translation Variant Convolution for Layout-Aware Visual Processing CVPR 2022 Two-layer neural network on infinite dimensional data: global optimization guarantee in the mean-field regime NeurIPS 2022 UDC: Unified DNAS for Compressible TinyML Models for Neural Processing Units NeurIPS 2022 Uncertainty Modeling in Generative Compressed Sensing ICML 2022 Understanding and Improving Knowledge Distillation for Quantization Aware Training of Large Transformer Encoders EMNLP 2022 Understanding and Increasing Efficiency of Frank-Wolfe Adversarial Training CVPR 2022 Understanding and Leveraging Overparameterization in Recursive Value Estimation ICLR 2022 Understanding Square Loss in Training Overparametrized Neural Network Classifiers NeurIPS 2022 Understanding Stochastic Optimization Behavior at the Layer Update Level (Student Abstract) AAAI 2022 Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction NeurIPS 2022 Understanding Uncertainty Maps in Vision With Statistical Testing CVPR 2022 Uniform Subdivision of Omnidirectional Camera Space for Efficient Spherical Stereo Matching CVPR 2022 Unifying and Boosting Gradient-Based Training-Free Neural Architecture Search NeurIPS 2022 Universality of Group Convolutional Neural Networks Based on Ridgelet Analysis on Groups NeurIPS 2022 Universality of Winning Tickets: A Renormalization Group Perspective ICML 2022