rectified linear unit

How Does the ReLU Activation Affect the Implicit Bias of Gradient Descent on High-dimensional Neural Network Regression? COLT 2026 The Median is Easier than it Looks: Approximation with a Constant-Depth, Linear-Width ReLU Network COLT 2026 Optimizing Neural Network Effectiveness via Non-Monotonicity Refinement WACV 2025 Random ReLU Neural Networks as Non-Gaussian Processes JMLR 2025 Agnostic Learning of General ReLU Activation Using Gradient Descent ICLR 2023 How do Minimum-Norm Shallow Denoisers Look in Function Space? NeurIPS 2023 Learning to Linearize Deep Neural Networks for Secure and Efficient Private Inference ICLR 2023 Improved Bounds on Neural Complexity for Representing Piecewise Linear Functions NeurIPS 2022 The Effects of Mild Over-parameterization on the Optimization Landscape of Shallow ReLU Neural Networks COLT 2021 When does gradient descent with logistic loss interpolate using deep networks with smoothed ReLU activations? COLT 2021 Adaptive Convolutional ReLUs AAAI 2020 Analytic Marching: An Analytic Meshing Solution from Deep Implicit Surface Networks ICML 2020 Channel Equilibrium Networks for Learning Deep Representation ICML 2020 Generating Accurate Pseudo-Labels in Semi-Supervised Learning and Avoiding Overconfident Predictions via Hermite Polynomial Activations CVPR 2020 Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networks ICML 2020 Generalized Tensor Models for Recurrent Neural Networks ICLR 2019 Initialization of ReLUs for Dynamical Isometry NeurIPS 2019 Jumpout : Improved Dropout for Deep Neural Networks with ReLUs ICML 2019 Monotone Learning with Rectified Wire Networks JMLR 2019 Time/Accuracy Tradeoffs for Learning a ReLU with respect to Gaussian Marginals NeurIPS 2019 Understanding Deep Neural Networks with Rectified Linear Units ICLR 2018 When is a Convolutional Filter Easy to Learn? ICLR 2018 Learning ReLUs via Gradient Descent NeurIPS 2017 Net-Trim: Convex Pruning of Deep Neural Networks with Performance Guarantee NeurIPS 2017 Nonlinear Hybrid Planning with Deep Net Learned Transition Models and Mixed-Integer Linear Programming IJCAI 2017