conftrace_

linear transformer

13 papers

Explore in graph

Also known as

LT

Co-occurring keywords

language modeling (660) in-context learning (1405) attention mechanism (4001) linear attention (52) language model (4599) recurrent neural network (1790) gradient descent initialization (1) sparse contextual bigram (1) kernel function (40) preconditioned gradient descent (6)

Papers

Learning Advanced Self-Attention for Linear Transformers in the Singular Value Domain IJCAI 2025
Learning and Transferring Sparse Contextual Bigrams with Linear Transformers NIPS 2024
Linear Transformers are Versatile In-Context Learners NIPS 2024
Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers AISTATS 2024
Proxyformer: Nyström-Based Linear Transformer with Trainable Proxy Tokens AAAI 2024
Linear Transformers with Learnable Kernel Functions are Better In-Context Models ACL 2024
Parallelizing Linear Transformers with the Delta Rule over Sequence Length NIPS 2024
In-Context Learning of a Linear Transformer Block: Benefits of the MLP Component and One-Step GD Initialization NIPS 2024
Practical Computational Power of Linear Transformers and Their Recurrent and Self-Referential Extensions EMNLP 2023
Head-Free Lightweight Semantic Segmentation with Linear Transformer AAAI 2023
The Devil in Linear Transformer EMNLP 2022
Going Beyond Linear Transformers with Recurrent Fast Weight Programmers NIPS 2021
Towards Incremental Transformers: An Empirical Analysis of Transformer Models for Incremental NLU EMNLP 2021