Stochastic parrot

From HandWiki
Short description: Term used in machine learning

In machine learning, a stochastic parrot is a term highlighting the opinion that large language models, even though they are good at generating convincing language, do not actually understand the meaning of the language being processed.[1][2] The term was coined by Emily M. Bender[2][3] in the 2021 artificial intelligence research paper "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜" by Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell.[4]

Definition and implications

A stochastic parrot, according to Bender, is an entity "for haphazardly stitching together sequences of linguistic forms … according to probabilistic information about how they combine, but without any reference to meaning."[3] (A stochastic process is one whose outcome is random.)

More formally, the term refers to "large language models that are impressive in their ability to generate realistic-sounding language but ultimately do not truly understand the meaning of the language they are processing."[2]

According to Lindholm, et. al., the analogy highlights two vital limitations:[1]

  1. The predictions made by a learning machine are essentially repeating back the contents of the data, with some added noise (or stochasticity) caused by the limitations of the model.
  2. The machine learning algorithm does not understand the problem it has learnt. It can't know when it is repeating something incorrect, out of context, or socially inappropriate.

They go on to note that because of these limitations, a learning machine might produce results which are "dangerously wrong".[1]

Origin

The term was first used in the paper "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜" by Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell (using the pseudonym "Shmargaret Shmitchell").[4] The paper covered the risks of very large language models, regarding their environmental and financial costs, inscrutability leading to unknown dangerous biases, the inability of the models to understand the concepts underlying what they learn, and the potential for using them to deceive people.[5] The paper and subsequent events resulted in Gebru and Mitchell losing their jobs at Google, and a subsequent protest by Google employees.[6][7]

Subsequent usage

In July 2021, the Alan Turing Institute hosted a keynote and panel discussion on the paper.[8] (As of May 2023), the paper has been cited in 1,529 publications.[9] The term has been used in publications in the fields of law,[10] grammar,[11] narrative,[12] and humanities.[13] The authors continue to maintain their concerns about the dangers of chatbots based on large language models, such as GPT-4.[14]

See also

References

  1. ↑ 1.0 1.1 1.2 Lindholm et al. 2022, pp. 322–3.
  2. ↑ 2.0 2.1 2.2 Uddin, Muhammad Saad (April 20, 2023). "Stochastic Parrots: A Novel Look at Large Language Models and Their Limitations" (in en-US). https://towardsai.net/p/machine-learning/stochastic-parrots-a-novel-look-at-large-language-models-and-their-limitations. 
  3. ↑ 3.0 3.1 Weil, Elizabeth (March 1, 2023). "You Are Not a Parrot". New York. https://nymag.com/intelligencer/article/ai-artificial-intelligence-chatbots-emily-m-bender.html. Retrieved 2023-05-12. 
  4. ↑ 4.0 4.1 Bender, Emily M.; Gebru, Timnit; McMillan-Major, Angelina; Shmitchell, Shmargaret (2021-03-01). "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜". Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. FAccT '21. New York, NY, USA: Association for Computing Machinery. pp. 610–623. doi:10.1145/3442188.3445922. ISBN 978-1-4503-8309-7. 
  5. ↑ Haoarchive, Karen (4 December 2020). "We read the paper that forced Timnit Gebru out of Google. Here's what it says." (in en). https://www.technologyreview.com/2020/12/04/1013294/google-ai-ethics-research-paper-forced-out-timnit-gebru/. 
  6. ↑ Lyons, Kim (5 December 2020). "Timnit Gebru's actual paper may explain why Google ejected her". https://www.theverge.com/2020/12/5/22155985/paper-timnit-gebru-fired-google-large-language-models-search-ai. 
  7. ↑ Taylor, Paul (2021-02-12). "Stochastic Parrots" (in en). https://www.lrb.co.uk/blog/2021/february/stochastic-parrots. 
  8. ↑ Weller (2021).
  9. ↑ "Bender: On the Dangers of Stochastic Parrots". https://scholar.google.com/scholar?cluster=415069420329958137. 
  10. ↑ Arnaudo, Luca (April 20, 2023). Artificial Intelligence, Capabilities, Liabilities: Interactions in the Shadows of Regulation, Antitrust – And Family Law. doi:10.2139/ssrn.4424363. 
  11. ↑ Bleackley, Pete; BLOOM (2023). "In the Cage with the Stochastic Parrot". Speculative Grammarian CXCII (3). https://specgram.com/CXCII.3/07.bloom.cage.html. Retrieved 2023-05-13. 
  12. ↑ Gáti, Daniella (2023). "Theorizing Mathematical Narrative through Machine Learning.". Journal of Narrative Theory (Project MUSE) 53 (1): 139–165. doi:10.1353/jnt.2023.0003. 
  13. ↑ Rees, Tobias (2022). "Non-Human Words: On GPT-3 as a Philosophical Laboratory". Daedalus 151 (2): 168–82. doi:10.1162/daed_a_01908. 
  14. ↑ Goldman, Sharon (March 20, 2023). "With GPT-4, dangers of 'Stochastic Parrots' remain, say researchers. No wonder OpenAI CEO is a 'bit scared'" (in en-US). https://venturebeat.com/ai/with-gpt-4-dangers-of-stochastic-parrots-remain-say-researchers-no-wonder-openai-ceo-is-a-bit-scared-the-ai-beat/. 

Works cited

  • Lindholm, A.; Wahlström, N.; Lindsten, F. ; Schön, T. B. (2022). Machine Learning: A First Course for Engineers and Scientists. Cambridge University Press. ISBN 978-1108843607. 
  • Weller, Adrian (July 13, 2021). On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 (video). Alan Turing Institute. Keynote by Emily Bender. The presentation was followed by a panel discussion.

Further reading

External links



Retrieved from "https://handwiki.org/wiki/index.php?title=Stochastic_parrot&oldid=3433158"