conftrace_

relu activation

93 papers

Explore in graph

Also known as

RELU

Co-occurring keywords

neural network (6616) gradient descent (1144) learning theory (519) relu network (143) neural network optimization (1293) deep neural network (1803) rectified linear unit (24) neural network training (309) single neuron (11) generalization bound (652)

Papers

ReLUPruner: Rethinking ReLU Importance with Taylor Expansion for Efficient Private Inference AAAI 2026
Posterior and Variational Inference for Deep Neural Networks with Heavy-Tailed Weights JMLR 2025
How does PDE order affect the convergence of PINNs? NIPS 2024
A faster and simpler algorithm for learning shallow networks COLT 2024
Topological obstruction to the training of shallow ReLU neural networks NIPS 2024
Hamiltonian Monte Carlo on ReLU Neural Networks is Inefficient NIPS 2024
Improving Lipschitz-Constrained Neural Networks by Learning Activation Functions JMLR 2024
The Inhibitor: ReLU and Addition-Based Attention for Efficient Transformers (Student Abstract) AAAI 2024
On the Effect of Initialization: The Scaling Path of 2-Layer Neural Networks JMLR 2024
Over-Parameterization Exponentially Slows Down Gradient Descent for Learning a Single Neuron COLT 2023
Mnemonist: Locating Model Parameters that Memorize Training Examples UAI 2023
Benign Overfitting in Two-layer ReLU Convolutional Neural Networks ICML 2023
Training Neural Networks is NP-Hard in Fixed Dimension NIPS 2023
Exact Verification of ReLU Neural Control Barrier Functions NIPS 2023
Benign Overfitting in Deep Neural Networks under Lazy Training ICML 2023
Norm-based Generalization Bounds for Sparse Neural Networks NIPS 2023
Kernel Logistic Regression Approximation of an Understandable ReLU Neural Network ICML 2023
Robustly Learning a Single Neuron via Sharpness ICML 2023
Does a sparse ReLU network training problem always admit an optimum ? NIPS 2023
Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spaces JMLR 2023
Linearity Grafting: Relaxed Neuron Pruning Helps Certifiable Robustness ICML 2022
Improved Bounds on Neural Complexity for Representing Piecewise Linear Functions NIPS 2022
Local SGD Optimizes Overparameterized Neural Networks in Polynomial Time AISTATS 2022
Training Two-Layer ReLU Networks with Gradient Descent is Inconsistent JMLR 2022
SQ Lower Bounds for Learning Single Neurons with Massart Noise NIPS 2022