conftrace_

relu activation

93 papers

Explore in graph

Also known as

RELU

Co-occurring keywords

neural network (6616) gradient descent (1144) learning theory (519) relu network (143) neural network optimization (1293) deep neural network (1803) rectified linear unit (24) neural network training (309) single neuron (11) generalization bound (652)

Papers

Theoretical analysis of deep neural networks for temporally dependent observations NIPS 2022
Local Identifiability of Deep ReLU Neural Networks: the Theory NIPS 2022
Neural Network Architecture Beyond Width and Depth NIPS 2022
Improved Bounds on Neural Complexity for Representing Piecewise Linear Functions NIPS 2022
A proof of convergence for the gradient descent optimization method with random initializations in the training of neural networks with ReLU activation for piecewise linear target functions JMLR 2022
SQ Lower Bounds for Learning Single Neurons with Massart Noise NIPS 2022
Local SGD Optimizes Overparameterized Neural Networks in Polynomial Time AISTATS 2022
Zhestyatsky at SemEval-2021 Task 2: ReLU over Cosine Similarity for BERT Fine-tuning IJCNLP 2021
On Learnability via Gradient Method for Two-Layer ReLU Neural Networks in Teacher-Student Setting ICML 2021
Robust Learning for Data Poisoning Attacks ICML 2021
A single gradient step finds adversarial examples on random two-layers neural networks NIPS 2021
A Convergence Analysis of Gradient Descent on Graph Neural Networks NIPS 2021
Zhestyatsky at SemEval-2021 Task 2: ReLU over Cosine Similarity for BERT Fine-tuning SEMEVAL 2021
Sparsifying Networks via Subdifferential Inclusion ICML 2021
DeepReDuce: ReLU Reduction for Fast Private Inference ICML 2021
On the Generalization Power of Overfitted Two-Layer Neural Tangent Kernel Models ICML 2021
Zhestyatsky at SemEval-2021 Task 2: ReLU over Cosine Similarity for BERT Fine-tuning ACL 2021
Towards Lower Bounds on the Depth of ReLU Neural Networks NIPS 2021
Efficient Algorithms for Learning Depth-2 Neural Networks with General ReLU Activations NIPS 2021
Learning a Single Neuron with Bias Using Gradient Descent NIPS 2021
Numerical influence of ReLU’(0) on backpropagation NIPS 2021
Efficiently Learning One Hidden Layer ReLU Networks From Queries NIPS 2021
When does gradient descent with logistic loss interpolate using deep networks with smoothed ReLU activations? COLT 2021
Near-Optimal SQ Lower Bounds for Agnostically Learning Halfspaces and ReLUs under Gaussian Marginals NIPS 2020
Statistical-Query Lower Bounds via Functional Gradients NIPS 2020