HEXQ

HEXQ is a reinforcement learning algorithm created by Bernhard Hengst, which attempts to solve a Markov Decision Process by decomposing it hierarchically.

Bernhard Hengst (2002). Discovering Hierarchy in Reinforcement Learning with HEXQ. http://citeseer.ist.psu.edu/hengst02discovering.html.