Skip to main content
← Back to Bibliography

Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

Authors: Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, Bryan Catanzaro

Year: 2020

Venue: arXiv

Type: article

URL: https://arxiv.org/abs/1909.08053

arXiv: 1909.08053

Cite as: [@shoeybi2020megatronlm]

Raw Files

No raw files yet. Run node scripts/fetch-bibliography-raw.mjs --only shoeybi2020megatronlm to populate, or drop files into raw/bibliography/shoeybi2020megatronlm/.

BibTeX

@article{shoeybi2020megatronlm,
  title = {Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism},
  author = {Mohammad Shoeybi and Mostofa Patwary and Raul Puri and Patrick LeGresley and Jared Casper and Bryan Catanzaro},
  year = {2020},
  journal = {arXiv},
  url = {https://arxiv.org/abs/1909.08053}
}

Notes

No notes yet. Create notes/shoeybi2020megatronlm.md to add notes.