Generative artificial intelligence
Generative artificial intelligence or generative AI is a type of artificial intelligence (AI) system capable of generating text, images, or other media in response to prompts.[1][2] Generative AI models learn the patterns and structure of their input training data, and then generate new data that has similar characteristics.[3][4]
Notable generative AI systems include ChatGPT (and its variant Bing Chat), a chatbot built by OpenAI using their GPT-3 and GPT-4 foundational large language models,[5] and Bard, a chatbot built by Google using their LaMDA foundation model.[6] Other generative AI models include artificial intelligence art systems such as Stable Diffusion, Midjourney, and DALL-E.[7]
Generative AI has potential applications across a wide range of industries, including art, writing, software development, product design, healthcare, finance, gaming, marketing, and fashion.[8][9][10] Investment in generative AI surged during the early 2020s, with large companies such as Microsoft, Google, and Baidu as well as numerous smaller firms developing generative AI models.[1][11][12] However, there are also concerns about the potential misuse of generative AI, such as in creating fake news or deepfakes, which can be used to deceive or manipulate people.[13]
History
Since its founding, the field of machine learning has used statistical models, including generative models, to model and predict data. Beginning in the late 2000s, the emergence of deep learning drove progress and research in image and video processing, text analysis, speech recognition, and other tasks. However, most deep neural networks were trained as discriminative models performing classification tasks such as convolutional neural network-based image classification.
In 2014, advancements such as the variational autoencoder and generative adversarial network produced the first practical deep neural networks capable of learning generative, rather than discriminative, models of complex data such as images. These deep generative models were the first able to output not only class labels for images, but to output entire images.
In 2017, the Transformer network enabled advancements in generative models, leading to the first Generative pre-trained transformer in 2018.[15] This was followed in 2019 by GPT-2 which demonstrated the ability to generalize unsupervised to many different tasks as a Foundation model.[16]
In 2021, the release of DALL-E, a transformer-based pixel generative model, followed by Midjourney and Stable Diffusion marked the emergence of practical high-quality artificial intelligence art from natural language prompts.
In January 2023, Futurism.com broke the story that CNET had been using an undisclosed internal AI tool to write at least 77 of its stories; after the news broke, CNET posted corrections to 41 of the stores.[17]
In March 2023, GPT-4 was released. A team from Microsoft Research argued that "it could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system".[18]
In April 2023, German tabloid Die Aktuelle published a fake AI-generated interview with reclusive former racing driver Michael Schumacher. The story included two possible disclosures: the cover included the line "deceptively real", and inside the magazine acknowledged at the end of the interview that the interview was AI-generated. The editor-in-chief was fired shortly thereafter amid the controversy.[19]
Modalities
A generative AI system is constructed by applying unsupervised or self-supervised machine learning to a data set. The capabilities of a generative AI system depend on the modality or type of the data set used.
Generative AI can be either unimodal or multimodal; unimodal systems take only one type of input, whereas multimodal systems can take more than one type of input.[20] For example, one version of OpenAI's GPT-4 accepts both text and image inputs.[21]
- Text: Generative AI systems trained on words or word tokens include GPT-3, LaMDA, LLaMA, BLOOM, GPT-4, and others (see List of large language models). They are capable of natural language processing, machine translation, and natural language generation and can be used as foundation models for other tasks.[22] Data sets include BookCorpus, Wikipedia, and others (see List of text corpora).
- Code: In addition to natural language text, large language models can be trained on programming language text, allowing them to generate source code for new computer programs.[23] Examples include OpenAI Codex.
- Molecules: Generative AI systems can be trained on sequences of amino acids or molecular representations such as SMILES representing DNA or proteins. These systems, such as AlphaFold, are used for protein structure prediction and drug discovery.[24] Datasets include various biological datasets.
- Music: Generative AI systems such as MusicLM can be trained on the audio waveforms of recorded music along with text annotations, in order to generate new musical samples based on text descriptions such as a calming violin melody backed by a distorted guitar riff.[25]
- Video: Generative AI trained on annotated video can generate temporally-coherent video clips. Examples include Gen1 and Gen2 by RunwayML[26] and Make-A-Video by Meta Platforms.[27]
- Robot actions: Generative AI trained on the motions of a robotic system can generate new trajectories for motion planning. For example, UniPi from Google Research uses prompts like "pick up blue bowl" or "wipe plate with yellow sponge" to control movements of a robot arm.[28]
See also
- Computational creativity – Multidisciplinary endeavour
- Artificial general intelligence – Hypothetical human-level or stronger AI
- Artificial imagination
- Artificial intelligence art – Machine application of knowledge of human aesthetic expressions
- Music and artificial intelligence
- Generative adversarial network – Deep learning method
- Generative pre-trained transformer – Type of large language model
- Large language model – Type of artificial neural network
References
- ↑ 1.0 1.1 Griffith, Erin; Metz, Cade (2023-01-27). "Anthropic Said to Be Closing In on $300 Million in New A.I. Funding". The New York Times. https://www.nytimes.com/2023/01/27/technology/anthropic-ai-funding.html. Retrieved 2023-03-14.
- ↑ Lanxon, Nate; Bass, Dina; Davalos, Jackie (March 10, 2023). "A Cheat Sheet to AI Buzzwords and Their Meanings". Bloomberg News. https://news.bloomberglaw.com/tech-and-telecom-law/a-cheat-sheet-to-ai-buzzwords-and-their-meanings-quicktake.
- ↑ Pasick, Adam (2023-03-27). "Artificial Intelligence Glossary: Neural Networks and Other Terms Explained" (in en-US). The New York Times. ISSN 0362-4331. https://www.nytimes.com/article/ai-artificial-intelligence-glossary.html.
- ↑ "Generative models". 2016-06-16. https://openai.com/research/generative-models.
- ↑ Metz, Cade (2023-03-14). "OpenAI Plans to Up the Ante in Tech's A.I. Race" (in en-US). The New York Times. ISSN 0362-4331. https://www.nytimes.com/2023/03/14/technology/openai-gpt4-chatgpt.html.
- ↑ Thoppilan, Romal; De Freitas, Daniel; Hall, Jamie; Shazeer, Noam; Kulshreshtha, Apoorv; Cheng, Heng-Tze; Jin, Alicia; Bos, Taylor; Baker, Leslie; Du, Yu; Li, YaGuang; Lee, Hongrae; Zheng, Huaixiu Steven; Ghafouri, Amin; Menegali, Marcelo; Huang, Yanping; Krikun, Maxim; Lepikhin, Dmitry; Qin, James; Chen, Dehao; Xu, Yuanzhong; Chen, Zhifeng; Roberts, Adam; Bosma, Maarten; Zhao, Vincent; Zhou, Yanqi; Chang, Chung-Ching; Krivokon, Igor; Rusch, Will; Pickett, Marc; Srinivasan, Pranesh; Man, Laichee; Meier-Hellstern, Kathleen; Ringel Morris, Meredith; Doshi, Tulsee; Delos Santos, Renelito; Duke, Toju; Soraker, Johnny; Zevenbergen, Ben; Prabhakaran, Vinodkumar; Diaz, Mark; Hutchinson, Ben; Olson, Kristen; Molina, Alejandra; Hoffman-John, Erin; Lee, Josh; Aroyo, Lora; Rajakumar, Ravi; Butryna, Alena; Lamm, Matthew; Kuzmina, Viktoriya; Fenton, Joe; Cohen; Aaron; Bernstein, Rachel; Kurzweil, Ray; Aguera-Arcas, Blaise; Cui, Claire; Croak, Marian; Chi, Ed; Le, Quoc (January 20, 2022). "LaMDA: Language Models for Dialog Applications". arXiv:2201.08239 [cs.CL].
- ↑ Roose, Kevin (2022-10-21). "A Coming-Out Party for Generative A.I., Silicon Valley's New Craze". https://www.nytimes.com/2022/10/21/technology/generative-ai.html.
- ↑ "Don't fear an AI-induced jobs apocalypse just yet". The Economist. 2023-03-06. https://www.economist.com/business/2023/03/06/dont-fear-an-ai-induced-jobs-apocalypse-just-yet. Retrieved 2023-03-14.
- ↑ Harreis, H.; Koullias, T.; Roberts, Roger. "Generative AI: Unlocking the future of fashion". https://www.mckinsey.com/industries/retail/our-insights/generative-ai-unlocking-the-future-of-fashion.
- ↑ "How Generative AI Can Augment Human Creativity". Harvard Business Review. 2023-06-16. ISSN 0017-8012. https://hbr.org/2023/07/how-generative-ai-can-augment-human-creativity.
- ↑ "The race of the AI labs heats up". The Economist. 2023-01-30. https://www.economist.com/business/2023/01/30/the-race-of-the-ai-labs-heats-up. Retrieved 2023-03-14.
- ↑ Yang, June; Gokturk, Burak (2023-03-14). "Google Cloud brings generative AI to developers, businesses, and governments". https://cloud.google.com/blog/products/ai-machine-learning/generative-ai-for-businesses-and-governments.
- ↑ Justin Hendrix (May 16, 2023). "Transcript: Senate Judiciary Subcommittee Hearing on Oversight of AI". https://techpolicy.press/transcript-senate-judiciary-subcommittee-hearing-on-oversight-of-ai/.
- ↑ "The Writers Strike Is Taking a Stand on AI" (in en). Time. 4 May 2023. https://time.com/6277158/writers-strike-ai-wga-screenwriting/.
- ↑ "finetune-transformer-lm". https://github.com/openai/finetune-transformer-lm.
- ↑ Radford, Alec; Wu, Jeffrey; Child, Rewon; Luan, David; Amodei, Dario; Sutskever, Ilya; others (2019). "Language models are unsupervised multitask learners". OpenAI blog 1 (8): 9.
- ↑ Roth, Emma (25 January 2023). "CNET found errors in more than half of its AI-written stories". The Verge. https://www.theverge.com/2023/1/25/23571082/cnet-ai-written-stories-errors-corrections-red-ventures.
- ↑ Bubeck, Sébastien; Chandrasekaran, Varun; Eldan, Ronen; Gehrke, Johannes; Horvitz, Eric; Kamar, Ece; Lee, Peter; Lee, Yin Tat; Li, Yuanzhi; Lundberg, Scott; Nori, Harsha; Palangi, Hamid; Ribeiro, Marco Tulio; Zhang, Yi (March 22, 2023). "Sparks of Artificial General Intelligence: Early experiments with GPT-4". arXiv:2303.12712 [cs.CL].
- ↑ "A magazine touted Michael Schumacher's first interview in years. It was actually AI". NPR. 28 April 2023. https://www.npr.org/2023/04/28/1172473999/michael-schumacher-ai-interview-german-magazine.
- ↑ https://www.marktechpost.com/2023/03/21/a-history-of-generative-ai-from-gan-to-gpt-4/
- ↑ "Explainer: What is Generative AI, the technology behind OpenAI's ChatGPT?". Reuters. March 17, 2023. https://www.reuters.com/technology/what-is-generative-ai-technology-behind-openais-chatgpt-2023-03-17/.
- ↑ Lua error in Module:Citation/CS1/Date_validation at line 764: attempt to index local 'date_string' (a nil value).
- ↑ Chen, Ming; Tworek, Jakub; Jun, Hongyu; Yuan, Qinyuan; Pinto, Hanyu Philippe De Oliveira; Kaplan, Jerry; Edwards, Haley; Burda, Yannick; Joseph, Nicholas; Brockman, Greg; Ray, Alvin (2021-07-06). "Evaluating Large Language Models Trained on Code". arXiv:2107.03374 [cs.LG].
- ↑ Heaven, Will Douglas (2023-02-15). "AI is dreaming up drugs that no one has ever seen. Now we've got to see if they work". Massachusetts Institute of Technology. https://www.technologyreview.com/2023/02/15/1067904/ai-automation-drug-development/.
- ↑ Agostinelli, Andrea; Denk, Timo I.; Borsos, Zalán; Engel, Jesse; Verzetti, Mauro; Caillon, Antoine; Huang, Qingqing; Jansen, Aren; Roberts, Adam; Tagliasacchi, Marco; Sharifi, Matt; Zeghidour, Neil; Frank, Christian (26 January 2023). "MusicLM: Generating Music From Text". arXiv:2301.11325 [cs.SD].
- ↑ Metz, Cade (April 4, 2023). "Instant Videos Could Represent the Next Leap in A.I. Technology" (in en). https://www.nytimes.com/2023/04/04/technology/runway-ai-videos.html.
- ↑ Queenie Wong (Sep 29, 2022). "Facebook Parent Meta's AI Tool Can Create Artsy Videos From Text". cnet.com. https://www.cnet.com/news/social-media/facebook-parent-metas-ai-tool-can-create-artsy-videos-from-text/. Retrieved Apr 4, 2023.
- ↑ Sherry Yang, Yilun Du (2023-04-12). "UniPi: Learning universal policies via text-guided video generation". Google Research, Brain Team. Google AI Blog. https://ai.googleblog.com/2023/04/unipi-learning-universal-policies-via.html.
 |  |
