Neural Network Optimization

Combined Depth Space Based Architecture Search for Person Re-Identification CVPR 2021 Compatibility-Aware Heterogeneous Visual Search CVPR 2021 Cross-Iteration Batch Normalization CVPR 2021 Deep Spiking Neural Network with Neural Oscillation and Spike-Phase Information AAAI 2021 Delving into Variance Transmission and Normalization: Shift of Average Gradient Makes the Network Collapse AAAI 2021 Demystifying and Generalizing BinaryConnect NeurIPS 2021 Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers ICML 2021 Differentiable Spike: Rethinking Gradient-Descent for Training Spiking Neural Networks NeurIPS 2021 DNN-based Topology Optimisation: Spatial Invariance and Neural Tangent Kernel NeurIPS 2021 Does Preprocessing Help Training Over-parameterized Neural Networks? NeurIPS 2021 DPFPS: Dynamic and Progressive Filter Pruning for Compressing Convolutional Neural Networks from Scratch AAAI 2021 Effects of Parameter Norm Growth During Transformer Training: Inductive Bias from Gradient Descent EMNLP 2021 Elastic Consistency: A Practical Consistency Model for Distributed Stochastic Gradient Descent AAAI 2021 Enhancing Robustness of Neural Networks through Fourier Stabilization ICML 2021 Fast and Accurate Model Scaling CVPR 2021 Frivolous Units: Wider Networks Are Not Really That Wide AAAI 2021 Generalization Performance of Multi-pass Stochastic Gradient Descent with Convex Loss Functions JMLR 2021 GNNAdvisor: An Adaptive and Efficient Runtime System for GNN Acceleration on GPUs OSDI 2021 HardCoRe-NAS: Hard Constrained diffeRentiable Neural Architecture Search ICML 2021 Heavy Ball Neural Ordinary Differential Equations NeurIPS 2021 How Does Topology Influence Gradient Propagation and Model Performance of Deep Networks With DenseNet-Type Skip Connections? CVPR 2021 HR-NAS: Searching Efficient High-Resolution Neural Architectures With Lightweight Transformers CVPR 2021 Improving Gradient Flow with Unrolled Highway Expectation Maximization AAAI 2021 Is Each Layer Non-trivial in CNN? (Student Abstract) AAAI 2021 Label Noise SGD Provably Prefers Flat Global Minimizers NeurIPS 2021