View source for Action selection

'''Action selection''' is a way of characterizing the most basic problem of intelligent systems: what to do next. In [[Artificial intelligence|artificial intelligence]] and computational [[Cognitive science|cognitive science]], "the action selection problem" is typically associated with intelligent agents and [[Animat|animat]]s—artificial systems that exhibit complex behaviour in an agent environment. The term is also sometimes used in [[Biology:Ethology|ethology]] or animal behavior.
One problem for understanding action selection is determining the level of abstraction used for specifying an "act". At the most basic level of abstraction, an atomic act could be anything from ''contracting a muscle cell'' to ''provoking a war''. Typically for any one action-selection mechanism, the set of possible actions is predefined and fixed.

Most researchers working in this field place high demands on their agents:
* The acting [[Intelligent agent|agent]] typically must select its action in dynamic and unpredictable environments.
* The agents typically act in [[Real-time computing|real time]]; therefore they must make decisions in a timely fashion.
* The agents are normally created to perform several different tasks. These tasks may conflict for resource allocation (e.g. can the agent put out a fire and deliver a cup of coffee at the same time?)
* The environment the agents operate in may include [[Humans|humans]], who may make things more difficult for the agent (either intentionally or by attempting to assist.)
* The agents themselves are often intended to model animals or humans, and animal/human behaviour is quite complicated.

For these reasons action selection is not trivial and attracts a good deal of research.

== Characteristics of the action selection problem ==
The main problem for action selection is [[Complexity|complexity]]. Since all [[Computation|computation]] takes both time and space (in memory), agents cannot possibly consider every option available to them at every instant in time. Consequently, they must be biased, and constrain their search in some way. For AI, the question of action selection is ''what is the best way to constrain this search''? For biology and ethology, the question is ''how do various types of animals constrain their search? Do all animals use the same approaches? Why do they use the ones they do?''

One fundamental question about action selection is whether it is really a problem at all for an agent, or whether it is just a description of an [[Philosophy:Emergence|emergent]] property of an intelligent agent's behavior. However, if we consider how we are going to build an intelligent agent, then it becomes apparent there must be ''some'' mechanism for action selection. This mechanism may be highly distributed (as in the case of distributed organisms such as social insect colonies or [[Biology:Slime mold|slime mold]]) or it may be a special-purpose module.

The action selection mechanism (ASM) determines not only the agent's actions in terms of impact on the world, but also directs its perceptual [[Philosophy:Attention|attention]], and updates its [[Memory|memory]]. These egocentric sorts of actions may in turn result in modifying the agent's basic behavioural capacities, particularly in that updating memory implies some form of [[Machine learning|machine learning]] is possible. Ideally, action selection itself should also be able to learn and adapt, but there are many problems of [[Combinatorics|combinatorial complexity]] and computational [[Tractable problem|tractability]] that may require restricting the search space for learning.

In AI, an ASM is also sometimes either referred to as an [[Agent architecture|agent architecture]] or thought of as a substantial part of one.

== AI mechanisms ==
Generally, artificial action selection mechanisms can be divided into several categories: [[Automated planning and scheduling|symbol-based systems]] sometimes known as classical planning, distributed solutions, and reactive or [[Reactive planning|dynamic planning]]. Some approaches do not fall neatly into any one of these categories. Others are really more about providing scientific models than practical AI control; these last are described further in the next section.

=== Symbolic approaches ===
{{main|Automated planning and scheduling}}
Early in the [[History of artificial intelligence|history of artificial intelligence]], it was assumed that the best way for an agent to choose what to do next would be to compute a probably optimal plan, and then execute that plan. This led to the [[Physical symbol system|physical symbol system]] hypothesis, that a physical agent that can manipulate symbols is necessary and sufficient for intelligence. Many software agents still use this approach for action selection. It normally requires describing all sensor readings, the world, all of ones actions and all of one's goals in some form of predicate logic. Critics of this approach complain that it is too slow for real-time planning and that, despite the proofs, it is still unlikely to produce optimal plans because reducing descriptions of reality to logic is a process prone to errors.

[[Philosophy:Satisficing|Satisficing]] is a decision-making strategy that attempts to meet criteria for adequacy, rather than identify an optimal solution. A satisficing strategy may often, in fact, be (near) optimal if the costs of the decision-making process itself, such as the cost of obtaining complete information, are considered in the outcome calculus.

'''Goal driven architectures''' – In these [[Symbol|symbol]]ic architectures, the agent's behaviour is typically described by a set of goals. Each goal can be achieved by a process or an activity, which is described by a prescripted plan. The agent must just decide which process to carry on to accomplish a given goal. The plan can expand to subgoals, which makes the process slightly recursive. Technically, more or less, the plans exploits condition-rules. These architectures are [[Reactive planning|reactive]] or hybrid. Classical examples of goal driven architectures are implementable refinements of belief-desire-intention architecture like [http://www.marcush.net/IRS/irs_downloads.html JAM] or [https://web.archive.org/web/20060522224109/http://urtax.ms.mff.cuni.cz/ive/public/about.php IVE].

=== Distributed approaches ===
In contrast to the symbolic approach, distributed systems of action selection actually have no one "box" in the agent which decides the next action. At least in their idealized form, distributed systems have many modules running in parallel and determining the best action based on local expertise. In these idealized systems, overall coherence is expected to emerge somehow, possibly through careful design of the interacting components. This approach is often inspired by artificial neural networks research. In practice, there is almost always ''some'' centralised system determining which module is "the most active" or has the most salience. There is evidence real biological brains also have such executive decision systems which evaluate which of the competing systems deserves the most [[Philosophy:Attention|attention]], or more properly, has its desired actions disinhibited.
* {{visible anchor|ASMO}} is an attention-based architecture developed by Mary-Anne Williams, Benjamin Johnston and their PhD student Rony Novianto.<ref>Samsonovich, A. V. "[https://www.researchgate.net/publication/221313246_Attention_in_the_ASMO_Cognitive_Architecture Attention in the ASMO cognitive architecture]." Biologically Inspired Cognitive Architectures (2010): 98. {{Webarchive|url=https://web.archive.org/web/20221106215714/https://www.researchgate.net/publication/221313246_Attention_in_the_ASMO_Cognitive_Architecture |date=2022-11-06 }}</ref> It orchestrates a diversity of modular distributed processes that can use their own representations and techniques to perceive the environment, process information, plan actions and propose actions to perform.
* Various types of [[Winner-take-all in action selection|winner-take-all]] architectures, in which the single selected action takes full control of the motor system
* '''Spreading activation''' including Maes Nets (ANA)
* '''Extended Rosenblatt & Payton''' is a spreading activation architecture developed by Toby Tyrrell in 1993. The agent's behaviour is stored in the form of a hierarchical connectionism network, which Tyrrell named free-flow hierarchy. Recently exploited for example by [http://vrlab.epfl.ch/Publications/pdf/Sevin_Thalmann_CGI_05.pdf de Sevin & Thalmann] (2005) or [https://web.archive.org/web/20110717210629/http://cyber.felk.cvut.cz/gerstner/eth/download/dpdk2.pdf Kadleček] (2001).
* '''Behavior based AI''', was a response to the slow speed of robots using symbolic action selection techniques. In this form, separate modules respond to different stimuli and generate their own responses. In the original form, the [[Subsumption architecture|subsumption architecture]], these consisted of different layers which could monitor and suppress each other's inputs and outputs.
* '''Creatures''' are virtual pets from a computer game driven by three-layered [[Artificial neural network|neural network]], which is adaptive. Their mechanism is reactive since the network at every time step determines the task that has to be performed by the pet. The network is described well in the paper of [http://www.cp.eng.chula.ac.th/~vishnu/gameResearch/AI/creatures.pdf Grand et al.] (1997) and in [http://www.double.co.nz/creatures/ The Creatures Developer Resources]. See also the [http://creatureswiki.net/ Creatures Wiki].

=== Dynamic planning approaches ===
Because purely distributed systems are difficult to construct, many researchers have turned to using explicit hard-coded plans to determine the priorities of their system.

Dynamic or reactive planning methods compute just one next action in every instant based on the current context and pre-scripted plans. In contrast to classical planning methods, reactive or dynamic approaches do not suffer [[Combinatorial explosion|combinatorial explosion]]. On the other hand, they are sometimes seen as too rigid to be considered [[Artificial general intelligence|strong AI]], since the plans are coded in advance. At the same time, natural intelligence can be rigid in some contexts although it is fluid and able to adapt in others.

Example dynamic planning mechanisms include:
* '''Finite-state machines''' These are reactive architectures used mostly for computer game agents, in particular for first-person shooters bots, or for virtual movie actors. Typically, the state-machines are hierarchical. For concrete game examples, see [https://www.gamedeveloper.com/programming/gdc-2005-proceeding-handling-complexity-in-the-i-halo-2-i-ai Halo 2 bots paper] by Damian Isla (2005) or [http://www.kbs.twi.tudelft.nl/Publications/MSc/2001-VanWaveren-MSc.html the Master's Thesis about Quake III bots] by Jan Paul van Waveren (2001). For a movie example, see Softimage.
* Other '''structured reactive plans''' tend to look a little more like conventional plans, often with ways to represent hierarchical and sequential structure. Some, such as PRS's 'acts', have support for partial plans.<ref>{{cite web|url=http://www.ai.sri.com/~prs/|title=PRS-CL: A Procedural Reasoning System|author=Karen L. Myers|work=Artificial Intelligence Center|publisher=[[Organization:SRI International|SRI International]]|access-date=2013-06-13}}</ref> Many agent architectures from the mid-1990s included such plans as a "middle layer" that provided organization for low-level behavior modules while being directed by a higher level real-time planner. Despite this supposed [[Engineering:Interoperability|interoperability]] with automated planners, most structured reactive plans are hand coded (Bryson 2001, ch. 3). Examples of structured reactive plans include James Firby's [http://people.cs.uchicago.edu/~firby/raps/ RAP] System and the Nils Nilsson's [https://ai.stanford.edu/users/nilsson/trweb/tr.html Teleo-reactive plans]. PRS, RAPs & TRP are no longer developed or supported. One still-active (as of 2006) descendant of this approach is the Parallel-rooted Ordered Slip-stack Hierarchical (or [http://www.cs.bath.ac.uk/~jjb/web/posh.html POSH]) action selection system, which is a part of Joanna Bryson's Behaviour Oriented Design.

Sometimes to attempt to address the perceived inflexibility of dynamic planning, hybrid techniques are used. In these, a more conventional AI planning system searches for new plans when the agent has spare time, and updates the dynamic plan library when it finds good solutions. The important aspect of any such system is that when the agent needs to select an action, some solution exists that can be used immediately (see further [[Anytime algorithm|anytime algorithm]]).

=== Others ===
* [http://CogniTeam.com CogniTAO] is a decision making engine it based on [[Belief–desire–intention software model|BDI]] (belief-desire-intention), it includes built in teamwork capabilities.
* [[Soar (cognitive architecture)|Soar]] is a [[Symbol|symbol]]ic [[Cognitive architecture|cognitive architecture]]. It is based on condition-action rules known as [[Production system (computer science)|productions]]. Programmers can use the Soar development toolkit for building both reactive and planning agents, or any compromise between these two extremes.
* '''[http://www.ai-center.com/projects/excalibur/index.html Excalibur]''' was a research project led by Alexander Nareyek featuring any-time planning agents for computer games. The architecture is based on structural [[Constraint satisfaction|constraint satisfaction]], which is an advanced [[Artificial intelligence|artificial intelligence]] technique.
* [[Software:ACT-R|ACT-R]] is similar to Soar. It includes a [[Bayesian inference|Bayesian]] learning system to help prioritize the productions.
* ABL/Hap
* '''[[Fuzzy control system|Fuzzy architectures]]''' The [[Fuzzy logic|fuzzy approach]] in action selection produces more smooth behaviour than can be produced by architectures exploiting boolean condition-action rules (like Soar or POSH). These architectures are mostly [[Reactive planning|reactive]] and symbolic.

== Theories of action selection in nature ==
Many dynamic models of artificial action selection were originally inspired by research in [[Biology:Ethology|ethology]]. In particular, [[Biography:Konrad Lorenz|Konrad Lorenz]] and [[Biography:Nikolaas Tinbergen|Nikolaas Tinbergen]] provided the idea of an innate releasing mechanism to explain instinctive behaviors ([[Biology:Fixed action pattern|fixed action pattern]]s). Influenced by the ideas of [[Biography:William McDougall (psychologist)|William McDougall]], Lorenz developed this into a "psychohydraulic" model of the [[Philosophy:Motivation|motivation]] of behavior. In ethology, these ideas were influential in the 1960s, but they are now regarded as outdated because of their use of an [[Physics:Energy flow (ecology)|energy flow]] metaphor; the [[Biology:Nervous system|nervous system]] and the control of behavior are now normally treated as involving information transmission rather than energy flow. Dynamic plans and neural networks are more similar to information transmission, while spreading activation is more similar to the diffuse control of emotional / hormonal systems.

[[Biography:Stan Franklin|Stan Franklin]] has proposed that '''action selection''' is the right perspective to take in understanding the role and evolution of [[Philosophy:Mind|mind]]. See his page on [http://www.msci.memphis.edu/~franklin/paradigm.html the action selection paradigm]. {{Webarchive|url=https://web.archive.org/web/20061009021508/http://www.msci.memphis.edu/~franklin/paradigm.html |date=2006-10-09 }}

=== AI models of neural action selection ===
Some researchers create elaborate models of neural action selection. See for example:
* The [https://web.archive.org/web/20070628062431/http://ccnlab.colorado.edu/mambo/ Computational Cognitive Neuroscience Lab] (CU Boulder).
* The [http://www.abrg.group.shef.ac.uk/ Adaptive Behaviour Research Group] (Sheffield).

=== Catecholaminergic Neuron Electron Transport (CNET) ===
The locus coeruleus (LC) is one of the primary sources of [[Biology:Norepinephrine|noradrenaline]] in the brain, and has been associated with selection of [[Cognition|cognitive processing]], such as attention and behavioral tasks.<ref>{{Cite journal |last=Sara |first=Susan J |date=December 2015 |title=Locus Coeruleus in time with the making of memories |url=http://dx.doi.org/10.1016/j.conb.2015.07.004 |journal=Current Opinion in Neurobiology |volume=35 |pages=87–94 |doi=10.1016/j.conb.2015.07.004 |pmid=26241632 |s2cid=206952441 |issn=0959-4388}}</ref><ref>{{Cite journal |last1=Poe |first1=Gina R. |last2=Foote |first2=Stephen |last3=Eschenko |first3=Oxana |last4=Johansen |first4=Joshua P. |last5=Bouret |first5=Sebastien |last6=Aston-Jones |first6=Gary |last7=Harley |first7=Carolyn W. |last8=Manahan-Vaughan |first8=Denise |last9=Weinshenker |first9=David |last10=Valentino |first10=Rita |last11=Berridge |first11=Craig |last12=Chandler |first12=Daniel J. |last13=Waterhouse |first13=Barry |last14=Sara |first14=Susan J. |date=2020-09-17 |title=Locus coeruleus: a new look at the blue spot |url=http://dx.doi.org/10.1038/s41583-020-0360-9 |journal=Nature Reviews Neuroscience |volume=21 |issue=11 |pages=644–659 |doi=10.1038/s41583-020-0360-9 |pmid=32943779 |pmc=8991985 |issn=1471-003X}}</ref><ref>{{Cite journal |last1=McBurney-Lin |first1=Jim |last2=Yang |first2=Hongdian |date=2022-09-04 |title=The locus coeruleus mediates behavioral flexibility |url=http://dx.doi.org/10.1101/2022.09.01.506286 |access-date=2022-11-13 |journal=Cell Reports|volume=41 |issue=4 |page=111534 |doi=10.1016/j.celrep.2022.111534|biorxiv=10.1101/2022.09.01.506286 |pmid=36288712 |pmc=9662304 |s2cid=252187005 }}</ref><ref name=":0">{{Cite journal |last1=Feng |first1=Jiesi |last2=Zhang |first2=Changmei |last3=Lischinsky |first3=Julieta |last4=Jing |first4=Miao |last5=Zhou |first5=Jingheng |last6=Wang |first6=Huan |last7=Zhang |first7=Yajun |last8=Dong |first8=Ao |last9=Wu |first9=Zhaofa |date=2018-10-23 |title=A genetically encoded fluorescent sensor for rapid and specific''in vivo''detection of norepinephrine |doi=10.1101/449546 |doi-access=free }}</ref> The [[Biology:Pars compacta|substantia nigra pars compacta]] (SNc) is one of the primary sources of [[Biology:Dopamine|dopamine]] in the brain, and has been associated with action selection, primarily as part of the [[Biology:Basal ganglia|basal ganglia]].<ref>{{Cite journal |last1=Varazzani |first1=C. |last2=San-Galli |first2=A. |last3=Gilardeau |first3=S. |last4=Bouret |first4=S. |date=2015-05-20 |title=Noradrenaline and Dopamine Neurons in the Reward/Effort Trade-Off: A Direct Electrophysiological Comparison in Behaving Monkeys |journal=Journal of Neuroscience |volume=35 |issue=20 |pages=7866–7877 |doi=10.1523/jneurosci.0454-15.2015 |pmid=25995472 |pmc=6795183 |s2cid=6531661 |issn=0270-6474|doi-access=free }}</ref><ref>{{Cite journal |last1=Fan |first1=D. |last2=Rossi |first2=M. A. |last3=Yin |first3=H. H. |date=2012-04-18 |title=Mechanisms of Action Selection and Timing in Substantia Nigra Neurons |url=http://dx.doi.org/10.1523/jneurosci.5924-11.2012 |journal=Journal of Neuroscience |volume=32 |issue=16 |pages=5534–5548 |doi=10.1523/jneurosci.5924-11.2012 |pmid=22514315 |pmc=6703499 |issn=0270-6474}}</ref><ref>{{Cite journal |last1=Partanen |first1=Juha |last2=Achim |first2=Kaia |date=2022-09-06 |title=Neurons gating behavior—developmental, molecular and functional features of neurons in the Substantia Nigra pars reticulata |journal=Frontiers in Neuroscience |volume=16 |page=976209 |doi=10.3389/fnins.2022.976209 |pmid=36148148 |pmc=9485944 |issn=1662-453X|doi-access=free }}</ref><ref>{{Cite journal |last1=Stephenson-Jones |first1=Marcus |last2=Samuelsson |first2=Ebba |last3=Ericsson |first3=Jesper |last4=Robertson |first4=Brita |last5=Grillner |first5=Sten |date=July 2011 |title=Evolutionary Conservation of the Basal Ganglia as a Common Vertebrate Mechanism for Action Selection |journal=Current Biology |volume=21 |issue=13 |pages=1081–1091 |doi=10.1016/j.cub.2011.05.001 |pmid=21700460 |s2cid=9327412 |issn=0960-9822|doi-access=free }}</ref><ref>{{Citation |last1=Guatteo |first1=Ezia |title=Substantia Nigra Control of Basal Ganglia Nuclei |date=2009 |url=http://dx.doi.org/10.1007/978-3-211-92660-4_7 |work=Birth, Life and Death of Dopaminergic Neurons in the Substantia Nigra |pages=91–101 |place=Vienna |publisher=Springer Vienna |isbn=978-3-211-92659-8 |access-date=2022-11-13 |last2=Cucchiaroni |first2=Maria Letizia |last3=Mercuri |first3=Nicola B.|issue=73 |doi=10.1007/978-3-211-92660-4_7 |pmid=20411770 }}</ref>  CNET is a hypothesized neural signaling mechanism in the SNc and LC (which are catecholaminergic neurons), that could assist with action selection by routing energy between neurons in each group as part of action selection, to help one or more neurons in each group to reach [[Biology:Action potential|action potential]].<ref>{{Cite journal |last=Rourk |first=Christopher John |date=September 2018 |title=Ferritin and neuromelanin "quantum dot" array structures in dopamine neurons of the substantia nigra pars compacta and norepinephrine neurons of the locus coeruleus |journal=Biosystems |volume=171 |pages=48–58 |doi=10.1016/j.biosystems.2018.07.008 |pmid=30048795 |s2cid=51722018 |issn=0303-2647|doi-access=free }}</ref><ref>{{Citation |last=Rourk |first=Christopher J. |title=Functional neural electron transport |date=2020 |url=http://dx.doi.org/10.1016/bs.aiq.2020.08.001 |work=Quantum Boundaries of Life |series=Advances in Quantum Chemistry |volume=82 |pages=25–111 |publisher=Elsevier |doi=10.1016/bs.aiq.2020.08.001 |isbn=9780128226391 |s2cid=229230562 |access-date=2022-11-13}}</ref> It was first proposed in 2018, and is based on a number of physical parameters of those neurons, which can be broken down into three major components:

1) [[Biology:Ferritin|Ferritin]] and [[Biology:Neuromelanin|neuromelanin]] are present in high concentrations in those neurons, but it was unknown in 2018 whether they formed structures that would be capable of transmitting electrons over relatively long distances on the scale of microns between the largest of those neurons, which had not been previously proposed or observed.<ref>{{Cite journal |last1=Tribl |first1=Florian |last2=Asan |first2=Esther |last3=Arzberger |first3=Thomas |last4=Tatschner |first4=Thomas |last5=Langenfeld |first5=Elmar |last6=Meyer |first6=Helmut E. |last7=Bringmann |first7=Gerhard |last8=Riederer |first8=Peter |last9=Gerlach |first9=Manfred |last10=Marcus |first10=Katrin |date=August 2009 |title=Identification of L-ferritin in Neuromelanin Granules of the Human Substantia Nigra |journal=Molecular & Cellular Proteomics |volume=8 |issue=8 |pages=1832–1838 |doi=10.1074/mcp.m900006-mcp200 |pmid=19318681 |pmc=2722774 |s2cid=23650245 |issn=1535-9476|doi-access=free }}</ref> Those structures would also need to provide a routing or switching function, which had also not previously been proposed or observed.  Evidence of the presence of ferritin and neuromelanin structures in those neurons and their ability to both conduct electrons by sequential [[Physics:Quantum tunnelling|tunneling]] and to route/switch the path of the neurons was subsequently obtained.<ref>{{Cite journal |last=Rourk |first=Christopher J. |date=May 2019 |title=Indication of quantum mechanical electron transport in human substantia nigra tissue from conductive atomic force microscopy analysis |url=http://dx.doi.org/10.1016/j.biosystems.2019.02.003 |journal=Biosystems |volume=179 |pages=30–38 |doi=10.1016/j.biosystems.2019.02.003 |pmid=30826349 |s2cid=73509918 |issn=0303-2647}}</ref><ref>{{Cite journal |last1=Rourk |first1=Christopher |last2=Huang |first2=Yunbo |last3=Chen |first3=Minjing |last4=Shen |first4=Cai |date=2021-06-16 |title=Indication of Highly Correlated Electron Transport in Disordered Multilayer Ferritin Structures |url=http://dx.doi.org/10.31219/osf.io/7gqmt |access-date=2022-11-13 |doi=10.31219/osf.io/7gqmt |s2cid=241118606 }}</ref><ref>{{Cite journal |last1=Friedrich |first1=I. |last2=Reimann |first2=K. |last3=Jankuhn |first3=S. |last4=Kirilina |first4=E. |last5=Stieler |first5=J. |last6=Sonntag |first6=M. |last7=Meijer |first7=J. |last8=Weiskopf |first8=N. |last9=Reinert |first9=T. |last10=Arendt |first10=T. |last11=Morawski |first11=M. |date=2021-03-22 |title=Cell specific quantitative iron mapping on brain slices by immuno-µPIXE in healthy elderly and Parkinson's disease |journal=Acta Neuropathologica Communications |volume=9 |issue=1 |page=47 |doi=10.1186/s40478-021-01145-2 |pmid=33752749 |pmc=7986300 |s2cid=232322739 |issn=2051-5960|doi-access=free }}</ref>

2) ) The axons of large SNc neurons were known to have extensive arbors, but it was unknown whether post-synaptic activity at the synapses of those axons would raise the [[Biology:Membrane potential|membrane potential]] of those neurons sufficiently to cause the electrons to be routed to the neuron or neurons with the most post-synaptic activity for the purpose of action selection.  At the time, prevailing explanations of the purpose of those neurons was that they did not mediate action selection and were only modulatory and non-specific.<ref>{{Cite journal |last=Schultz |first=Wolfram |date=2016-02-02 |title=Reward functions of the basal ganglia |journal=Journal of Neural Transmission |volume=123 |issue=7 |pages=679–693 |doi=10.1007/s00702-016-1510-0 |pmid=26838982 |pmc=5495848 |s2cid=3894133 |issn=0300-9564|doi-access=free }}</ref> Prof. Pascal Kaeser of Harvard Medical School subsequently obtained evidence that large SNc neurons can be temporally and spatially specific and mediate action selection.<ref>{{Cite journal |last1=Liu |first1=Changliang |last2=Goel |first2=Pragya |last3=Kaeser |first3=Pascal S. |date=2021-04-09 |title=Spatial and temporal scales of dopamine transmission |url=http://dx.doi.org/10.1038/s41583-021-00455-7 |journal=Nature Reviews Neuroscience |volume=22 |issue=6 |pages=345–358 |doi=10.1038/s41583-021-00455-7 |pmid=33837376 |pmc=8220193 |issn=1471-003X}}</ref>  Other evidence indicates that the large LC axons have similar behavior.<ref>{{Cite journal |last1=Behl |first1=Tapan |last2=Kaur |first2=Ishnoor |last3=Sehgal |first3=Aayush |last4=Singh |first4=Sukhbir |last5=Makeen |first5=Hafiz A. |last6=Albratty |first6=Mohammed |last7=Alhazmi |first7=Hassan A. |last8=Bhatia |first8=Saurabh |last9=Bungau |first9=Simona |date=July 2022 |title=The Locus Coeruleus – Noradrenaline system: Looking into Alzheimer's therapeutics with rose coloured glasses |journal=Biomedicine & Pharmacotherapy |volume=151 |pages=113179 |doi=10.1016/j.biopha.2022.113179 |pmid=35676784 |s2cid=249137521 |issn=0753-3322|doi-access=free }}</ref><ref>{{Cite journal |last1=Breton-Provencher |first1=Vincent |last2=Drummond |first2=Gabrielle T. |last3=Sur |first3=Mriganka |date=2021-06-07 |title=Locus Coeruleus Norepinephrine in Learned Behavior: Anatomical Modularity and Spatiotemporal Integration in Targets |journal=Frontiers in Neural Circuits |volume=15 |page=638007 |doi=10.3389/fncir.2021.638007 |pmid=34163331 |pmc=8215268 |issn=1662-5110|doi-access=free }}</ref>

3) Several sources of electrons or excitons to provide the energy for the mechanism were hypothesized in 2018 but had not been observed at that time.  Dioxetane cleavage (which can occur during somatic dopamine metabolism by quinone degradation of melanin) was contemporaneously proposed to generate high energy triplet state electrons by Prof. Doug Brash at Yale, which could provide a source for electrons for the CNET mechanism.<ref>{{Cite journal |last1=Brash |first1=Douglas E. |last2=Goncalves |first2=Leticia C.P. |last3=Bechara |first3=Etelvino J.H. |date=June 2018 |title=Chemiexcitation and Its Implications for Disease |url=http://dx.doi.org/10.1016/j.molmed.2018.04.004 |journal=Trends in Molecular Medicine |volume=24 |issue=6 |pages=527–541 |doi=10.1016/j.molmed.2018.04.004 |pmid=29751974 |pmc=5975183 |issn=1471-4914}}</ref><ref>{{Cite journal |last1=Sulzer |first1=David |last2=Cassidy |first2=Clifford |last3=Horga |first3=Guillermo |last4=Kang |first4=Un Jung |last5=Fahn |first5=Stanley |last6=Casella |first6=Luigi |last7=Pezzoli |first7=Gianni |last8=Langley |first8=Jason |last9=Hu |first9=Xiaoping P. |last10=Zucca |first10=Fabio A. |last11=Isaias |first11=Ioannis U. |last12=Zecca |first12=Luigi |date=2018-04-10 |title=Neuromelanin detection by magnetic resonance imaging (MRI) and its promise as a biomarker for Parkinson's disease |url=http://dx.doi.org/10.1038/s41531-018-0047-3 |journal=npj Parkinson's Disease |volume=4 |issue=1 |page=11 |doi=10.1038/s41531-018-0047-3 |pmid=29644335 |pmc=5893576 |issn=2373-8057}}</ref><ref>{{Cite journal |last1=Premi |first1=S. |last2=Wallisch |first2=S. |last3=Mano |first3=C. M. |last4=Weiner |first4=A. B. |last5=Bacchiocchi |first5=A. |last6=Wakamatsu |first6=K. |last7=Bechara |first7=E. J. H. |last8=Halaban |first8=R. |last9=Douki |first9=T. |last10=Brash |first10=D. E. |date=2015-02-19 |title=Chemiexcitation of melanin derivatives induces DNA photoproducts long after UV exposure |url=http://dx.doi.org/10.1126/science.1256022 |journal=Science |volume=347 |issue=6224 |pages=842–847 |doi=10.1126/science.1256022 |pmid=25700512 |pmc=4432913 |bibcode=2015Sci...347..842P |issn=0036-8075}}</ref>

While evidence of a number of physical predictions of the CNET hypothesis has thus been obtained, evidence of whether the hypothesis itself is correct has not been sought. One way to try to determine whether the CNET mechanism is present in these neurons would be to use quantum dot fluorophores and optical probes to determine whether electron tunneling associated with ferritin in the neurons is occurring in association with specific actions.<ref name=":0" /><ref>{{Cite journal |last1=Pisano |first1=Filippo |last2=Pisanello |first2=Marco |last3=Lee |first3=Suk Joon |last4=Lee |first4=Jaeeon |last5=Maglie |first5=Emanuela |last6=Balena |first6=Antonio |last7=Sileo |first7=Leonardo |last8=Spagnolo |first8=Barbara |last9=Bianco |first9=Marco |last10=Hyun |first10=Minsuk |last11=De Vittorio |first11=Massimo |last12=Sabatini |first12=Bernardo L. |last13=Pisanello |first13=Ferruccio |date=November 2019 |title=Depth-resolved fiber photometry with a single tapered optical fiber implant |url=http://www.nature.com/articles/s41592-019-0581-x |journal=Nature Methods |language=en |volume=16 |issue=11 |pages=1185–1192 |doi=10.1038/s41592-019-0581-x |pmid=31591577 |s2cid=203848191 |issn=1548-7091}}</ref><ref>{{Cite journal |last1=Garg |first1=Mayank |last2=Vishwakarma |first2=Neelam |last3=Sharma |first3=Amit L. |last4=Singh |first4=Suman |date=2021-07-08 |title=Amine-Functionalized Graphene Quantum Dots for Fluorescence-Based Immunosensing of Ferritin |url=http://dx.doi.org/10.1021/acsanm.1c01398 |journal=ACS Applied Nano Materials |volume=4 |issue=7 |pages=7416–7425 |doi=10.1021/acsanm.1c01398 |s2cid=237804893 |issn=2574-0970}}</ref>

== See also ==

* {{Annotated link |Action description language}}
* {{Annotated link |Artificial intelligence in video games}}
* {{Annotated link |Cognitive robotics}}
* {{Annotated link |Expert system}}
* {{Annotated link |Inference engine}}
* {{Annotated link |Intelligent agent}}
* {{Annotated link |OPS5}}
* {{Annotated link |Production system (computer science)|Production system}}
* {{Annotated link |Reinforcement learning}}
* {{Annotated link |Rete algorithm}}
* {{Annotated link |Utility system}}

== References ==
{{reflist}}

== Further reading ==
* Bratman, M.: Intention, plans, and practical reason. Cambridge, Mass: Harvard University Press (1987)
* Brom, C., Lukavský, J., Šerý, O., Poch, T., Šafrata, P.: [https://archive.today/20070803004108/http://urtax.ms.mff.cuni.cz/ive Affordances and level-of-detail AI for virtual humans]. In: Proceedings of Game Set and Match 2, Delft (2006)
* Bryson, J.: [https://web.archive.org/web/20170706112447/ftp://publications.ai.mit.edu/ai-publications/2002/AITR-2002-003.pdf Intelligence by Design: Principles of Modularity and Coordination for Engineering Complex Adaptive Agents]. PhD thesis, [http://publications.csail.mit.edu/ai/ Massachusetts Institute of Technology] (2001)
* Champandard, A. J.: [https://books.google.com/books?id=ZpuR8GnBSGcC&q=%22action+selection%22 AI Game Development: Synthetic Creatures with learning and Reactive Behaviors]. New Riders, USA (2003)
* Grand, S., Cliff, D., Malhotra, A.: [https://www.researchgate.net/profile/Dave_Cliff/publication/220794226_Creatures_Artificial_Life_Autonomous_Software_Agents_for_Home_Entertainment/links/00463514cc5d6b6550000000/Creatures-Artificial-Life-Autonomous-Software-Agents-for-Home-Entertainment.pdf Creatures: Artificial life autonomous software-agents for home entertainment]. In: Johnson, W. L. (eds.): Proceedings of the First International Conference on Autonomous Agents. ACM press (1997) 22-29
* Huber, M. J.: [http://www.marcush.net/IRS/irs_downloads.html JAM: A BDI-theoretic mobile agent architecture]. In: Proceedings of the Third International Conference on Autonomous Agents (Agents'99). Seattle (1999) 236-243
* Isla, D.: [http://www.gamasutra.com/gdc2005/features/20050311/isla_pfv.htm Handling complexity in Halo 2]. In: Gamastura online, 03/11 (2005) {{Webarchive|url=https://web.archive.org/web/20060108113445/http://www.gamasutra.com/gdc2005/features/20050311/isla_pfv.htm |date=2006-01-08 }}
* Maes, P.: [https://dl.acm.org/citation.cfm?id=122367 The agent network architecture (ANA)]. In: SIGART Bulletin, 2 (4), pages 115–120 (1991)
* Nareyek, A. [http://www.ai-center.com/projects/excalibur/index.html Excalibur project]
* Reynolds, C. W. [https://www.researchgate.net/profile/Craig_Reynolds2/publication/2797343_Flocks_Herds_and_Schools_A_Distributed_Behavioral_Model/links/0fcfd5095a869204df000000.pdf Flocks, Herds, and Schools: A Distributed Behavioral Model]. In: Computer Graphics, 21(4) (SIGGRAPH '87 Conference Proceedings) (1987) 25–34.
* de Sevin, E. Thalmann, D.:[http://edesevin.free.fr/recherche/articles/CGI2005.pdf A motivational Model of Action Selection for Virtual Humans]. In: Computer Graphics International (CGI), IEEE Computer SocietyPress, New York (2005)
* Tyrrell, T.: [https://www.cs.cmu.edu/afs/cs.cmu.edu/project/oz/ftp/Tyrell/as.1.ps.Z Computational Mechanisms for Action Selection]. Ph.D. Dissertation. Centre for Cognitive Science, University of Edinburgh (1993)
* van Waveren, J. M. P.: The Quake III Arena Bot. Master thesis. Faculty ITS, University of Technology Delft (2001)
* Wooldridge, M. [https://books.google.com/books?id=X3ZQ7yeDn2IC&q=%22action+selection%22&pg=PR13 An Introduction to MultiAgent Systems]. John Wiley & Sons (2002)

== External links ==
* The University of Memphis: [http://www.msci.memphis.edu/~classweb/comp7990/fall2002/action.htm Agents by action selection] {{Webarchive|url=https://web.archive.org/web/20060418021500/http://www.msci.memphis.edu/~classweb/comp7990/fall2002/action.htm |date=2006-04-18 }}
* Michael Wooldridge: [https://web.archive.org/web/20051202035140/http://www.csc.liv.ac.uk/~mjw/pubs/mas99.pdf Introduction to agents and their action selection mechanisms]
* Cyril Brom: [http://ksvi.mff.cuni.cz/~brom/teaching.html#umelebytosti Slides on a course on action selection of artificial beings]
* [https://web.archive.org/web/20060507174634/http://sitemaker.umich.edu/soar Soar project]. University of Michigan.
* [https://archive.today/20071030065012/http://publishing.royalsoc.ac.uk/natural-action Modelling natural action selection], a special issue published by The Royal Society - [[Philosophy:Philosophical Transactions of the Royal Society|Philosophical Transactions of the Royal Society]]

{{Differentiable computing}}

[[Category:Artificial intelligence]]
[[Category:Functional analysis]]
[[Category:Motor control]]

{{Sourceattribution|Action selection}}