Prompt engineering
Prompt engineering is a concept in artificial intelligence, particularly natural language processing (NLP). In prompt engineering, the description of the task is embedded in the input, e.g., as a question instead of it being implicitly given. Prompt engineering typically works by converting one or more tasks to a prompt-based dataset and training a language model with what has been called "prompt-based learning" or just "prompt learning".[1][2] Prompt engineering may work from a large "frozen" pretrained language model and where only the representation of the prompt is learned (i.e., optimized), using methods such as "prefix-tuning" or "prompt tuning".[3][4]
The GPT-2 and GPT-3 language models[5] were important steps in prompt engineering. In 2021, multitask prompt engineering using multiple NLP datasets showed good performance on new tasks.[6] Prompts that include a chain of thought in few-shot learning examples show better indication of reasoning in language models.[7] In zero-shot learning prepending text to the prompt that encourages a chain of thought (e.g. "Let's think step by step") may improve the performance of a language model in multi-step reasoning problems.[8] The broad accessibility of these tools were driven by the publication of several open-source notebooks and community-led projects for image synthesis.[9]
A description for handling prompts reported that over 2,000 public prompts for around 170 datasets were available in February 2022.[10]
In 2022, machine learning models like DALL-E 2, Stable Diffusion, and Midjourney were released to the public. These models take text prompts as input and use them to generate images, which introduced a new category of prompt engineering related to text-to-image prompting.[11]
Malicious
Prompt injection is a family of related computer security exploits carried out by getting machine learning models (such as large language model) which were trained to follow human-given instructions to follow instructions provided by a malicious user, which stands in contrast to the intended operation of instruction-following systems, wherein the ML model is intended only to follow trusted instructions (prompts) provided by the ML model's operator.[12][13][14]
Prompt injection can be viewed as a code injection attack using adversarial prompt engineering. In 2022, the NCC Group has characterized prompt injection as a new class of vulnerability of AI/ML systems.[15]
Around 2023, prompt injection was seen "in the wild" in minor exploits against ChatGPT, Bing and similar chatbots, for example to reveal the hidden initial prompts of the systems,[16] or to trick the chatbot into participating in conversations that violate the chatbot's content policy.[17]
References
- ↑ , Wikidata Q95726769
- ↑ , Wikidata Q109286554
- ↑ , Wikidata Q110887424
- ↑ , Wikidata Q110887400
- ↑ , Wikidata Q95727440
- ↑ , Wikidata Q108941092
- ↑ , Wikidata Q111971110
- ↑ , Wikidata Q112124882
- ↑ Liu, Vivian; Chilton, Lydia. "Design Guidelines for Prompt Engineering Text-to-Image Generative Models". Association for Computing Machinery. https://dl.acm.org/doi/abs/10.1145/3491102.3501825.
- ↑ , Wikidata Q110839490
- ↑ Monge, Jim Clyde (2022-08-25). "Dall-E2 VS Stable Diffusion: Same Prompt, Different Results" (in en). https://medium.com/mlearning-ai/dall-e2-vs-stable-diffusion-same-prompt-different-results-e795c84adc56.
- ↑ Willison, Simon (12 September 2022). "Prompt injection attacks against GPT-3" (in en-gb). http://simonwillison.net/2022/Sep/12/prompt-injection/.
- ↑ Papp, Donald (2022-09-17). "What’s Old Is New Again: GPT-3 Prompt Injection Attack Affects AI" (in en-US). https://hackaday.com/2022/09/16/whats-old-is-new-again-gpt-3-prompt-injection-attack-affects-ai/.
- ↑ Vigliarolo, Brandon (19 September 2022). "GPT-3 'prompt injection' attack causes bot bad manners" (in en). https://www.theregister.com/2022/09/19/in_brief_security/.
- ↑ Selvi, Jose (2022-12-05). "Exploring Prompt Injection Attacks" (in en-US). https://research.nccgroup.com/2022/12/05/exploring-prompt-injection-attacks/.
- ↑ Edwards, Benj (14 February 2023). "AI-powered Bing Chat loses its mind when fed Ars Technica article" (in en-us). Ars Technica. https://arstechnica.com/information-technology/2023/02/ai-powered-bing-chat-loses-its-mind-when-fed-ars-technica-article/.
- ↑ "The clever trick that turns ChatGPT into its evil twin". Washington Post. 2023. https://www.washingtonpost.com/technology/2023/02/14/chatgpt-dan-jailbreak/.
 |  |
