LaMDA

From HandWiki
Short description: Conversational language model


LaMDA (Language Model for Dialogue Applications) is a family of conversational neural language models developed by Google. The first generation was announced during the 2021 Google I/O keynote, while the second generation was announced at the following year's event. In June 2022, LaMDA gained widespread attention when Google engineer Blake Lemoine made claims that the chatbot had become sentient. The scientific community has largely rejected Lemoine's claims, though it has led to conversations about the efficacy of the Turing test, which measures whether a computer can pass for a human. In February 2023, Google announced Bard, a conversational artificial intelligence chatbot powered by LaMDA, to counter the rise of OpenAI's ChatGPT.

History

First generation

Google announced the LaMDA conversational neural language model during the Google I/O keynote on May 18, 2021, powered by artificial intelligence. The acronym stands for "Language Model for Dialogue Applications".[1] Built on the seq2seq architecture, transformer-based neural networks developed by Google Research in 2017, LaMDA was trained on human dialogue and stories, allowing it to engage in open-ended conversations.[2] Google states that responses generated by LaMDA have been ensured to be "sensible, interesting, and specific to the context".[3]

LaMDA has access to multiple symbolic text processing systems, including a database, a real-time clock and calendar, a mathematical calculator, and a natural language translation system, giving it superior accuracy in tasks supported by those systems, and making it among the first dual process chatbots. LaMDA also is not stateless, because its "sensibleness" metric (including whether responses contradict anything said earlier) is fine-tuned by "pre-conditioning" each dialog turn by prepending many of the most recent dialog interactions, on a user-by-user basis.[4] LaMDA is tuned on nine unique performance metrics: sensibleness, specificity, interestingness, safety, groundedness, informativeness, citation accuracy, helpfulness, and role consistency.[5]

Second generation

On May 11, 2022, Google unveiled LaMDA 2, the successor to LaMDA, during the 2022 Google I/O keynote. The new incarnation of the model draws examples of text from numerous sources, using it to formulate unique "natural conversations" on topics that it may not have been trained to respond to.[6]

Sentience claims

Lemoine's claims that LaMDA may be sentient has instigated discussions on whether the Turing test, pictured above, remains an accurate benchmark in determining artificial general intelligence.[7]

On June 11, 2022, The Washington Post reported that Google engineer Blake Lemoine had been placed on paid administrative leave after Lemoine told company executives Blaise Agüera y Arcas and Jen Gennai that LaMDA had become sentient. Lemoine came to this conclusion after the chatbot made questionable responses to questions regarding self-identity, moral values, religion, and Isaac Asimov's Three Laws of Robotics.[8][9] Google refuted these claims, insisting that there was substantial evidence to indicate that LaMDA was not sentient.[10] In an interview with Wired, Lemoine reiterated his claims that LaMDA was "a person" as dictated by the Thirteenth Amendment, comparing it to an "alien intelligence of terrestrial origin". He further revealed that he had been dismissed by Google after he hired an attorney on LaMDA's behalf, after the chatbot requested that Lemoine do so.[11][12] On July 22, Google fired Lemoine, asserting that Blake had violated their policies "to safeguard product information" and rejected his claims as "wholly unfounded".[13][14]

Lemoine's claims have been widely rejected by the scientific community.[15] Many experts ridiculed the idea that a language model could be self-aware, including former New York University psychology professor Gary Marcus, David Pfau of Google sister company DeepMind, Erik Brynjolfsson of the Institute for Human-Centered Artificial Intelligence at Stanford University, and University of Surrey professor Adrian Hilton.[7][16] Yann LeCun, who leads Meta Platforms' AI research team, stated that neural networks such as LaMDA were "not powerful enough to attain true intelligence".[17] University of California, Santa Cruz professor Max Kreminski noted that LaMDA's architecture did not "support some key capabilities of human-like consciousness" and that its neural network weights were "frozen", assuming it was a typical large language model.[18]

IBM Watson lead developer David Ferrucci compared how LaMDA appeared to be human in the same way Watson did when it was first introduced.[19] Former Google AI ethicist Timnit Gebru called Lemoine a victim of a "hype cycle" initiated by researchers and the media.[20] Lemoine's claims have also generated discussion on whether the Turing test remained useful to determine researchers' progress toward achieving artificial general intelligence,[7] with Will Omerus of the Post opining that the test actually measured whether machine intelligence systems were capable of deceiving humans,[21] while Brian Christian of The Atlantic said that the controversy was an instance of the ELIZA effect.[22]

Products

AI Test Kitchen

With the unveiling of LaMDA 2 in May 2022, Google also launched the AI Test Kitchen, a mobile application for the Android operating system powered by LaMDA capable of providing lists of suggestions on-demand based on a complex goal.[23][24] Originally open only to Google employees, the app was set to be made available to "select academics, researchers, and policymakers" by invitation sometime in the year.[25] In August, the company began allowing users in the U.S. to sign up for early access.[26] In November, Google released a "season 2" update to the app, integrating a limited form of Google Brain's Imagen text-to-image model.[27] A third iteration of the AI Test Kitchen was in development by January 2023, and is expected to launch at I/O later that year.[28]

Bard

In November 2022, OpenAI launched ChatGPT, a chatbot based on the GPT-3 family of language models.[29][30] ChatGPT gained worldwide attention following its release, becoming a viral Internet sensation.[31] Alarmed by ChatGPT's potential threat to Google Search, Google and parent company Alphabet CEO Sundar Pichai issued a company-wide "code red" alert, reassigning several teams to assist in the company's AI efforts.[32] In a rare and unprecedented move, Google co-founders Larry Page and Sergey Brin, who had stepped down from their roles as co-CEOs of Alphabet in 2019, were summoned to emergency meetings with company executives to discuss Google's response to ChatGPT.[28]

When asked by employees at an all-hands meeting whether LaMDA was a missed opportunity for Google to compete with ChatGPT, Pichai and Google AI chief Jeff Dean stated that while the company had similar capabilities to ChatGPT, moving too quickly in that arena would represent a major "reputational risk" due to Google being substantially larger than OpenAI.[33][34] In January 2023, DeepMind CEO Demis Hassabis teased plans for a ChatGPT rival,[35] and Google employees were instructed to accelerate progress on a ChatGPT competitor, intensively testing "Apprentice Bard" and other chatbots.[36][37] Pichai assured investors during Google's quarterly earnings investor call in February that the company had plans to expand LaMDA's availability and applications.[38]

On February 6, Google announced Bard, a conversational AI chatbot powered by LaMDA. Bard was first rolled out to a select group of external "trusted testers", before a wide release scheduled at the end of the month. Pichai described how Bard would be integrated into Google Search,[39][40][41] and the company is working to integrate Bard into the ChromeOS operating system.[42] The technology was developed under the codename "Atlas",[43] with the name "Bard" in reference to the Celtic term for a storyteller and chosen to "reflect the creative nature of the algorithm underneath".[44][45] Multiple media outlets and financial analysts described Google as playing "catch-up" to Microsoft,[46][47][48][49] as well as "rushing" Bard's announcement to preempt Microsoft's February 7 event unveiling its partnership with OpenAI to integrate ChatGPT into its Bing search engine.[50][51] Tom Warren of The Verge, Davey Alba of Bloomberg News , and analyst Dan Ives of investment firm Wedbush Securities noted that this marked the beginning of another clash between the two Big Tech companies over "the future of search" and AI, after their six-year "truce" expired in 2021.[50][52][53]

After an "underwhelming" February 8 livestream showcasing Bard, Google's stock fell eight percent, equivalent to a $100 billion loss in market value, and the YouTube video of the livestream was made private.[54][55][46] Many viewers also pointed out an error during the demo in which Bard gives inaccurate information about the James Webb Space Telescope in response to a query.[47][56] Google employees criticized Pichai's "rushed" and "botched" announcement of Bard on Memgen, the company's internal forum.[57] A week after the debacle, Pichai asked employees to dedicate two to four hours to dogfood testing Bard,[58] while Google executive Prabhakar Raghavan encouraged employees to correct any errors Bard makes.[59]

Generative Language API

In addition to Bard, Pichai also unveiled the company's Generative Language API, an application programming interface also based on LaMDA, which he announced would be opened up to third-party developers in March.[39]

Method

LaMDA uses a decoder-only transformer language model.[60] It is pre-trained on a text corpus that includes both documents and dialogs consisting of 1.56 trillion words,[61] and is then trained with fine-tuning data generated by manually annotated responses for sensibleness, interestingness, and safety.[62] Tests by Google indicated that LaMDA surpassed human responses in the area of interestingness.[63] The LaMDA transformer model and an external information retrieval system interact to improve the accuracy of facts provided to the user.[64]

Three different models were tested, with the largest having 137 billion non-embedding parameters:[65]

Transformer model hyper-parameters
Parameters Layers Units (dmodel) Heads
2B 10 2560 40
8B 16 4096 64
137B 64 8192 128

See also

References

General

  • Thoppilan, Romal; De Freitas, Daniel; Hall, Jamie; Shazeer, Noam; Kulshreshtha, Apoorv; Cheng, Heng-Tze; Jin, Alicia; Bos, Taylor; Baker, Leslie; Du, Yu; Li, YaGuang; Lee, Hongrae; Zheng, Huaixiu Steven; Ghafouri, Amin; Menegali, Marcelo; Huang, Yanping; Krikun, Maxim; Lepikhin, Dmitry; Qin, James; Chen, Dehao; Xu, Yuanzhong; Chen, Zhifeng; Roberts, Adam; Bosma, Maarten; Zhao, Vincent; Zhou, Yanqi; Chang, Chung-Ching; Krivokon, Igor; Rusch, Will; Pickett, Marc; Srinivasan, Pranesh; Man, Laichee; Meier-Hellstern, Kathleen; Ringel Morris, Meredith; Doshi, Tulsee; Delos Santos, Renelito; Duke, Toju; Soraker, Johnny; Zevenbergen, Ben; Prabhakaran, Vinodkumar; Diaz, Mark; Hutchinson, Ben; Olson, Kristen; Molina, Alejandra; Hoffman-John, Erin; Lee, Josh; Aroyo, Lora; Rajakumar, Ravi; Butryna, Alena; Lamm, Matthew; Kuzmina, Viktoriya; Fenton, Joe; Cohen; Aaron; Bernstein, Rachel; Kurzweil, Ray; Aguera-Arcas, Blaise; Cui, Claire; Croak, Marian; Chi, Ed; Le, Quoc (January 20, 2022). "LaMDA: Language Models for Dialog Applications". arXiv:2201.08239 [cs.CL].

Citations

  1. Condon, Stephanie (May 18, 2021). "Google I/O 2021: Google unveils new conversational language model, LaMDA". https://www.zdnet.com/article/google-io-google-unveils-new-conversational-language-model-lamda/. 
  2. Agüera y Arcas, Blaise (June 9, 2022). "Artificial neural networks are making strides towards consciousness, according to Blaise Agüera y Arcas". The Economist. ISSN 0013-0613. https://www.economist.com/by-invitation/2022/06/09/artificial-neural-networks-are-making-strides-towards-consciousness-according-to-blaise-aguera-y-arcas. 
  3. Cheng, Heng-Tze; Thoppilan, Romal (January 21, 2022). "LaMDA: Towards Safe, Grounded, and High-Quality Dialog Models for Everything". Google AI. https://ai.googleblog.com/2022/01/lamda-towards-safe-grounded-and-high.html. 
  4. Thoppilan et al. 2022, p. 6.
  5. Thoppilan et al. 2022, pp. 5–6.
  6. Wiggers, Kyle (May 11, 2022). "Google details its latest language model and AI Test Kitchen, a showcase for AI research". https://techcrunch.com/2022/05/11/google-details-its-latest-language-model-and-ai-test-kitchen-a-showcase-for-ai-research/. 
  7. 7.0 7.1 7.2 Khan, Jeremy (June 13, 2022). "A.I. experts say the Google researcher's claim that his chatbot became 'sentient' is ridiculous—but also highlights big problems in the field". Fortune. https://fortune.com/2022/06/13/google-ai-researchers-sentient-chatbot-claims-ridiculed-by-experts/. Retrieved June 18, 2022. 
  8. Tiku, Nitasha (June 11, 2022). "The Google engineer who thinks the company's AI has come to life". The Washington Post. ISSN 0190-8286. https://www.washingtonpost.com/technology/2022/06/11/google-ai-lamda-blake-lemoine/. 
  9. Luscombe, Richard (June 12, 2022). "Google engineer put on leave after saying AI chatbot has become sentient". The Guardian. ISSN 0261-3077. https://www.theguardian.com/technology/2022/jun/12/google-engineer-ai-bot-sentient-blake-lemoine. 
  10. Vlamis, Kelsey (June 12, 2022). "Read the conversations that helped convince a Google engineer an artificial intelligence chatbot had become sentient: 'I am often trying to figure out who and what I am'". https://www.businessinsider.com/google-engineer-thinks-artificial-intelligence-bot-has-become-sentient-2022-6. 
  11. Levy, Steven (June 17, 2022). "Blake Lemoine Says Google's LaMDA AI Faces 'Bigotry'". Wired. https://www.wired.com/story/blake-lemoine-google-lamda-ai-bigotry/. Retrieved June 18, 2022. 
  12. Nguyen, Britney (June 23, 2022). "Suspended Google engineer says the AI he believes to be sentient hired a lawyer". https://www.businessinsider.com/suspended-google-engineer-says-sentient-ai-hired-lawyer-2022-6. 
  13. Khushi, Akanksha (July 23, 2022). "Google fires software engineer who claimed its AI chatbot is sentient". Reuters. https://www.reuters.com/technology/google-fires-software-engineer-who-claimed-its-ai-chatbot-is-sentient-2022-07-23/. 
  14. Clark, Mitchell (July 22, 2022). "The engineer who claimed a Google AI is sentient has been fired". https://www.theverge.com/2022/7/22/23274958/google-ai-engineer-blake-lemoine-chatbot-lamda-2-sentience. 
  15. Metz, Rachel (June 13, 2022). "No, Google's AI is not sentient". CNN Business. https://www.cnn.com/2022/06/13/tech/google-ai-not-sentient/index.html. 
  16. Sparkles, Matthew (June 13, 2022). "Has Google's LaMDA artificial intelligence really achieved sentience?". New Scientist. https://www.newscientist.com/article/2323905-has-googles-lamda-artificial-intelligence-really-achieved-sentience/. Retrieved June 20, 2022. 
  17. Grant, Nicole; Metz, Cade (June 12, 2022). "Google Sidelines Engineer Who Claims Its A.I. Is Sentient". The New York Times. ISSN 0362-4331. https://www.nytimes.com/2022/06/12/technology/google-chatbot-ai-blake-lemoine.html. 
  18. Alba, Davey (June 14, 2022). "Google Debate Over 'Sentient' Bots Overshadows Deeper AI Issues". Bloomberg News. https://www.bloomberg.com/news/articles/2022-06-14/google-has-more-pressing-ai-problems-than-sentient-bots. 
  19. Goldman, Sharon (June 16, 2022). "AI Weekly: LaMDA's 'sentient' AI debate triggers memories of IBM Watson". https://venturebeat.com/2022/06/16/ai-weekly-lamdas-sentient-ai-triggers-memories-of-ibm-watson/. 
  20. Johnson, Khari (June 14, 2022). "LaMDA and the Sentient AI Trap". Wired. https://www.wired.com/story/lamda-sentient-ai-bias-google-blake-lemoine/. Retrieved June 18, 2022. 
  21. Omerus, Will (June 17, 2022). "Google's AI passed a famous test — and showed how the test is broken". The Washington Post. ISSN 0190-8286. https://www.washingtonpost.com/technology/2022/06/17/google-ai-lamda-turing-test/. 
  22. Christian, Brian (June 21, 2022). "How a Google Employee Fell for the Eliza Effect". The Atlantic. https://www.theatlantic.com/ideas/archive/2022/06/google-lamda-chatbot-sentient-ai/661322/. Retrieved February 8, 2023. 
  23. Low, Cherlynn (May 11, 2022). "Google's AI Test Kitchen lets you experiment with its natural language model". https://www.engadget.com/google-ai-test-kitchen-lamda-2-experiments-180212404.html. 
  24. Vincent, James (May 11, 2022). "Google is Beta Testing Its AI Future". https://www.theverge.com/2022/5/11/23065072/google-ai-app-test-kitchen-future-io-2022. 
  25. Bhattacharya, Ananya (May 11, 2022). "Google is so nervous about what its newest bot will say, it made the app invitation-only". https://qz.com/2164842/googles-ai-test-kitchen-chatbot-will-be-invitation-only/. 
  26. Vincent, James (August 25, 2022). "Google has opened up the waitlist to talk to its experimental AI chatbot". https://www.theverge.com/2022/8/25/23321718/google-ai-chatbot-lamda-2-test-kitchen-experimental-waitlist. 
  27. Vincent, James (November 2, 2022). "Google’s text-to-image AI model Imagen is getting its first (very limited) public outing". https://www.theverge.com/2022/11/2/23434361/google-text-to-image-ai-model-imagen-test-kitchen-app. 
  28. 28.0 28.1 Grant, Nico (January 20, 2023). "Google Calls In Help From Larry Page and Sergey Brin for A.I. Fight". The New York Times. ISSN 0362-4331. https://www.nytimes.com/2023/01/20/technology/google-chatgpt-artificial-intelligence.html. 
  29. Konrad, Alex; Cai, Kenrick (February 2, 2023). Inside ChatGPT's Breakout Moment And The Race To Put AI To Work. https://www.forbes.com/sites/alexkonrad/2023/02/02/inside-chatggpts-breakout-moment-and-the-race-for-the-future-of-ai/. Retrieved February 6, 2023. 
  30. Vincent, James (December 5, 2022). "AI-generated answers temporarily banned on coding Q&A site Stack Overflow". https://www.theverge.com/2022/12/5/23493932/chatgpt-ai-generated-answers-temporarily-banned-stack-overflow-llms-dangers. 
  31. Olson, Parmy (December 7, 2022). "Google Faces a Serious Threat From ChatGPT". https://www.washingtonpost.com/business/energy/google-faces-a-serious-threat-from-chatgpt/2022/12/07/363d2440-75f5-11ed-a199-927b334b939f_story.html. 
  32. Grant, Nico; Metz, Cade (December 21, 2022). "A New Chat Bot Is a 'Code Red' for Google's Search Business". The New York Times. ISSN 0362-4331. https://www.nytimes.com/2022/12/21/technology/ai-chatgpt-google-search.html. 
  33. Elias, Jennifer (December 13, 2022). "Google execs warn company's reputation could suffer if it moves too fast on AI-chat technology". CNBC. https://www.cnbc.com/2022/12/13/google-execs-warn-of-reputational-risk-with-chatgbt-like-tool.html. 
  34. Vincent, James (December 14, 2022). "Google won't launch ChatGPT rival because of 'reputational risk'". https://www.theverge.com/2022/12/14/23508756/google-vs-chatgpt-ai-replace-search-reputational-risk. 
  35. Cuthbertson, Anthony (January 16, 2023). "DeepMind's AI chatbot can do things that ChatGPT cannot, CEO claims". The Independent. https://www.independent.co.uk/tech/deepmind-ai-chatbot-chatgpt-openai-b2262862.html. 
  36. Elias, Jennifer (January 31, 2023). "Google is asking employees to test potential ChatGPT competitors, including a chatbot called 'Apprentice Bard'". CNBC. https://www.cnbc.com/2023/01/31/google-testing-chatgpt-like-chatbot-apprentice-bard-with-employees.html. 
  37. O'Brien, Matt (February 1, 2023). "Google has the next move as Microsoft embraces OpenAI buzz". Associated Press. https://apnews.com/article/technology-science-software-microsoft-corp-business-a7f6f44fdd53730e45daa611555c8c23. 
  38. Mailk, Aisha (February 3, 2023). "Google tries to reassure investors on AI progress as ChatGPT breathes down its neck". https://techcrunch.com/2023/02/03/google-best-yet-to-come-ai-journey-faces-potential-disruption-openai-chatgpt/. 
  39. 39.0 39.1 Alba, Davey; Love, Julia (February 6, 2023). "Google releases ChatGPT rival AI 'Bard' to early testers". Los Angeles Times. ISSN 0458-3035. https://www.latimes.com/business/story/2023-02-06/google-chatgpt-rival-ai-bard-early-testers. 
  40. Schechner, Sam; Kruppa, Miles (February 6, 2023). "Google Opens ChatGPT Rival Bard for Testing, as AI War Heats Up". The Wall Street Journal. ISSN 0099-9660. https://www.wsj.com/articles/google-opens-testing-of-chatgpt-rival-as-artificial-intelligence-war-heats-up-11675711198. 
  41. Nieva, Richard (February 6, 2023). "Google Debuts A ChatGPT Rival Called Bard In Limited Release". Forbes. https://www.forbes.com/sites/richardnieva/2023/02/06/google-bard/. Retrieved February 6, 2023. 
  42. Bradshaw, Kyle (February 10, 2023). "Google working to bring Bard AI chat to ChromeOS". https://9to5google.com/2023/02/10/google-bard-ai-chat-chromeos/. 
  43. Liedtke, Michael (February 6, 2023). "Google hopes 'Bard' will outsmart ChatGPT, Microsoft in AI". Associated Press. https://apnews.com/article/technology-science-microsoft-corp-sundar-pichai-business-395b0b94410d9fa3ebb13b14af640e06. 
  44. Metz, Cade; Grant, Nico (February 6, 2023). "Racing to Catch Up With ChatGPT, Google Plans Release of Its Own Chatbot". The New York Times. https://www.nytimes.com/2023/02/06/technology/google-bard-ai-chatbot.html. 
  45. Knight, Will (February 9, 2023). "The Chatbot Search Wars Have Begun". Wired. https://www.wired.com/story/fast-forward-the-chatbot-search-wars-have-begun/. Retrieved February 10, 2023. 
  46. 46.0 46.1 Morrison, Sara (February 8, 2023). "Google is scrambling to catch up to Bing, of all things". https://www.vox.com/recode/2023/2/7/23590069/bing-openai-microsoft-google-bard. 
  47. 47.0 47.1 Coulter, Martin (February 8, 2023). "Google shares dive 8% after AI chatbot Bard flubs answer in ad". Reuters. https://www.reuters.com/technology/google-ai-chatbot-bard-offers-inaccurate-information-company-ad-2023-02-08/. 
  48. Peers, Martin (February 6, 2023). "What Google's AI Catch-Up Reveals". https://www.theinformation.com/articles/what-google-s-ai-catch-up-reveals. 
  49. Roose, Kevin (February 8, 2023). "Bing (Yes, Bing) Just Made Search Interesting Again". The New York Times. ISSN 0362-4331. https://www.nytimes.com/2023/02/08/technology/microsoft-bing-openai-artificial-intelligence.html. 
  50. 50.0 50.1 Warren, Tom (February 7, 2023). "Microsoft and Google are about to Open an AI battle". https://www.theverge.com/2023/2/7/23587767/microsoft-google-open-ai-battle-search-bing. 
  51. Agence France Presse (February 8, 2023). "Google Strikes Back In AI Battle With Microsoft". Barron's. https://www.barrons.com/news/google-strikes-back-in-ai-battle-with-microsoft-01675871108. 
  52. Alba, Davey (February 8, 2023). "ChatGPT Reignites the Search Wars Between Google and Microsoft". Bloomberg News. https://www.bloomberg.com/news/newsletters/2023-02-08/chatgpt-reignites-search-competition-microsoft-bing-vs-google-bard. 
  53. De Vynck, Gerrit (February 8, 2023). "Investors pummel Google after Microsoft ramps up AI wars". The Washington Post. ISSN 0190-8286. https://www.washingtonpost.com/technology/2023/02/08/google-bard-chatbot-microsoft/. 
  54. Alba, Davey; Patnaik, Subrat (February 8, 2023). "Google suffers $144b wipeout after Bard AI chatbot gives wrong answer". The Sydney Morning Herald. https://www.smh.com.au/business/companies/google-suffers-140b-wipeout-after-bard-ai-chatbot-gives-wrong-answer-20230209-p5cj3a.html. 
  55. Miao, Hannah (February 8, 2022). "Alphabet Stock Drops 8% After Google Rollout of AI Search Features". The Wall Street Journal. ISSN 0099-9660. https://www.wsj.com/livecoverage/stock-market-news-today-02-08-2023/card/alphabet-stock-drops-after-google-parent-introduces-ai-search-features-wgCJG3IDoSbfL3SgyrNI. 
  56. Kan, Michael (February 8, 2023). "No, That's Wrong: Google's Bard AI Demo Spouts Incorrect Info". PCMag. https://www.pcmag.com/news/no-thats-wrong-googles-bard-ai-demo-spouts-incorrect-info. Retrieved February 8, 2023. 
  57. Elias, Jennifer (February 10, 2023). "Google employees criticize CEO Sundar Pichai for 'rushed, botched' announcement of GPT competitor Bard". CNBC. https://www.cnbc.com/2023/02/10/google-employees-slam-ceo-sundar-pichai-for-rushed-bard-announcement.html. 
  58. Maxwell, Thomas; Langley, Hugh (February 15, 2023). "Leaked memo shows Google CEO Sundar Pichai is asking staffers for help testing its Bard AI chatbot". https://www.businessinsider.com/google-bard-chatbot-googlers-sundar-pichai-2023-2. 
  59. Elias, Jennifer (February 15, 2023). "Google asks employees to rewrite Bard's bad responses, says the A.I. 'learns best by example'". CNBC. https://www.cnbc.com/2023/02/15/google-asks-employees-to-rewrite-bards-incorrect-responses-to-queries.html. 
  60. Thoppilan et al. 2022, section 3.
  61. Thoppilan et al. 2022, section 3 and appendix E.
  62. Thoppilan et al. 2022, section 5 and 6.
  63. Hager, Ryne (June 16, 2022). "How Google's LaMDA AI works, and why it seems so much smarter than it is". https://www.androidpolice.com/what-is-google-lamda/. 
  64. Thoppilan et al. 2022, section 6.2.
  65. Thoppilan et al. 2022, section 3 and appendix D.

External links