Skip to main content
← Back to Bibliography

Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth

Authors: Yihe Dong, Jean-Baptiste Cordonnier, Andreas Loukas

Year: 2021

Venue: ICML

Type: article

URL: https://arxiv.org/abs/2103.03404

arXiv: 2103.03404

Cite as: [@dong2021attention]

Raw Files

No raw files yet. Run node scripts/fetch-bibliography-raw.mjs --only dong2021attention to populate, or drop files into raw/bibliography/dong2021attention/.

BibTeX

@inproceedings{dong2021attention,
  title = {Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth},
  author = {Yihe Dong and Jean-Baptiste Cordonnier and Andreas Loukas},
  year = {2021},
  booktitle = {ICML},
  url = {https://arxiv.org/abs/2103.03404}
}

Notes

No notes yet. Create notes/dong2021attention.md to add notes.