Exploration-exploitation dilemma
The exploration-exploitation dilemma, also known as the explore-exploit tradeoff, is a fundamental concept in decision-making that arises in many domains.[1][2] It is depicted as the balancing act between two opposing strategies. Exploitation involves choosing the best option based on current knowledge of the system (which may be incomplete or misleading), while exploration involves trying out new options that may lead to better outcomes in the future at the expense of an exploitation opportunity. Finding the optimal balance between these two strategies is a crucial challenge in many decision-making problems whose goal is to maximize long-term benefits.[3]
Application in machine learning
In the context of machine learning, the exploration-exploitation tradeoff is fundamental in reinforcement learning, a type of machine learning that involves training agents to make decisions based on feedback from the environment. Crucially, this feedback may be incomplete or delayed.[4] The agent must decide whether to exploit the current best-known policy or explore new policies to improve its performance. Various algorithms have been developed to address this challenge, such as epsilon-greedy, Thompson sampling, and the upper confidence bound.
References
- ↑ Berger-Tal, Oded; Nathan, Jonathan; Meron, Ehud; Saltz, David (22 April 2014). "The Exploration-Exploitation Dilemma: A Multidisciplinary Framework". PLOS ONE 9 (4): e95693. doi:10.1371/journal.pone.0095693. PMID 24756026.
- ↑ Rhee, Mooweon; Kim, Tohyun (2018). "Exploration and Exploitation" (in en). The Palgrave Encyclopedia of Strategic Management. London: Palgrave Macmillan UK. pp. 543–546. doi:10.1057/978-1-137-00772-8_388. ISBN 978-0-230-53721-7. http://link.springer.com/10.1057/978-1-137-00772-8_388.
- ↑ Fruit, R. (2019). Exploration-exploitation dilemma in Reinforcement Learning under various form of prior knowledge (Doctoral dissertation, Université de Lille 1, Sciences et Technologies; CRIStAL UMR 9189).
- ↑ Richard S. Sutton; Andrew G. Barto (2020). Reinforcement Learning: An Introduction (2nd edition). http://incompleteideas.net/book/the-book-2nd.html
 |  |
