View source for Shunting-yard algorithm

In [[Computer science|computer science]], the '''shunting-yard algorithm''' is a method for parsing mathematical expressions specified in [[Infix notation|infix notation]]. It can produce either a postfix notation string, also known as [[Reverse Polish notation]] (RPN), or an [[Abstract syntax tree|abstract syntax tree]] (AST).<ref>{{cite web|access-date=2020-12-28|title=Parsing Expressions by Recursive Descent|url=http://www.engr.mun.ca/~theo/Misc/exp_parsing.htm|website=www.engr.mun.ca|author=Theodore Norvell|date=1999}}</ref> The [[Algorithm|algorithm]] was invented by Edsger Dijkstra and named the "shunting yard" algorithm because its operation resembles that of a railroad shunting yard. Dijkstra first described the Shunting Yard Algorithm in the Mathematisch Centrum report [https://ir.cwi.nl/pub/9251 MR 34/61].

Like the evaluation of RPN, the shunting yard algorithm is stack-based. Infix expressions are the form of mathematical notation most people are used to, for instance {{nowrap|"3 + 4"}} or {{nowrap|"3 + 4 × (2 − 1)"}}. For the conversion there are two text variables ([[String (computer science)|strings]]), the input and the output. There is also a stack that holds operators not yet added to the output queue. To convert, the program reads each symbol in order and does something based on that symbol. The result for the above examples would be (in [[Reverse Polish notation]]) {{nowrap|"3 4 +"}} and {{nowrap|"3 4 2 1 − × +"}}, respectively.

The shunting yard algorithm will correctly parse all valid infix expressions, but does not reject all invalid expressions. For example, {{nowrap|"1 2 +"}} is not a valid infix expression, but would be parsed as {{nowrap|"1 + 2"}}. The algorithm can however reject expressions with mismatched parentheses.

The shunting-yard algorithm was later generalized into [[Operator-precedence parser|operator-precedence parsing]].

==A simple conversion==

#Input: {{nowrap|3 + 4}}
#Push 3 to the output queue (whenever a number is read it is pushed to the output)
#Push + (or its ID) onto the operator stack
#Push 4 to the output queue
#After reading the expression, pop the operators off the stack and add them to the output. 
#:In this case there is only one, "+".
#Output: {{nowrap|3 4 +}}

This already shows a couple of rules:
* All numbers are pushed to the output when they are read.
* At the end of reading the expression, pop all operators off the stack and onto the output.

==Graphical illustration==

[[File:Shunting yard.svg|frameless|border|center|500px|]]
Graphical illustration of algorithm, using a three-way railroad junction. The input is processed one symbol at a time: if a variable or number is found, it is copied directly to the output a), c), e), h). If the symbol is an operator, it is pushed onto the operator stack b), d), f). If the operator's precedence is lower than that of the operators at the top of the stack or the precedents are equal and the operator is left associative, then that operator is popped off the stack and added to the output g). Finally, any remaining operators are popped off the stack and added to the output i).

==The algorithm in detail==


 {{font color|blue|/* This implementation does not implement composite functions, functions with variable number of arguments, and unary operators. */}}
 
 '''while''' there are tokens to be read:
     read a token
     '''if''' the token is:
     - a ''number'':
         put it into the output queue
     - a ''[[Function (mathematics)|function]]'':
         push it onto the operator stack 
     - an ''operator'' ''o''<sub>1</sub>:
         '''while''' (
             there is an operator ''o''<sub>2</sub> other than the left parenthesis at the top
             of the operator stack, '''and''' (''o''<sub>2</sub> has greater [[Order of operations|precedence]] than ''o''<sub>1</sub>
             '''or''' they have the same precedence and ''o''<sub>1</sub> is left-associative)
         ):
             pop ''o''<sub>2</sub> from the operator stack into the output queue
         push ''o''<sub>1</sub> onto the operator stack
     - a ''left parenthesis'' (i.e. "("):
         push it onto the operator stack
     - a ''right parenthesis'' (i.e. ")"):
         '''while''' the operator at the top of the operator stack is not a left parenthesis:
             {'''assert''' the operator stack is not empty}
             {{font color|blue|/* If the stack runs out without finding a left parenthesis, then there are mismatched parentheses. */}}
             pop the operator from the operator stack into the output queue
         {'''assert''' there is a left parenthesis at the top of the operator stack}
         pop the left parenthesis from the operator stack and discard it
         '''if''' there is a function token at the top of the operator stack, '''then''':
             pop the function from the operator stack into the output queue
 {{font color|blue|/* After the while loop, pop the remaining items from the operator stack into the output queue. */}}
 '''while''' there are tokens on the operator stack:
     {{font color|blue|/* If the operator token on the top of the stack is a parenthesis, then there are mismatched parentheses. */}}
     {'''assert''' the operator on top of the stack is not a (left) parenthesis}
     pop the operator from the operator stack onto the output queue

To analyze the running time complexity of this algorithm, one has only to note that each token will be read once, each number, function, or operator will be printed once, and each function, operator, or parenthesis will be pushed onto the stack and popped off the stack once—therefore, there are at most a constant number of operations executed per token, and the running time is thus O(''n'')—linear in the size of the input.

The shunting yard algorithm can also be applied to produce prefix notation (also known as [[Polish notation]]). To do this one would simply start from the end of a string of tokens to be parsed and work backwards, reverse the output queue (therefore making the output queue an output stack), and flip the left and right parenthesis behavior (remembering that the now-left parenthesis behavior should pop until it finds a now-right parenthesis). And changing the [[Operator associativity|associativity]] condition to right.

==Detailed example==
Input: {{nowrap|3 + 4 × 2 ÷ ( 1 − 5 ) ^ 2 ^ 3}}

:{| class="wikitable"
! Operator !! Precedence !! Associativity
|- || align="center"
| ^ || 4 || Right 
|- || align="center"
| × || 3 || Left
|- || align="center"
| ÷ || 3 || Left
|- || align="center"
| + || 2 || Left
|- || align="center"
| − || 2 || Left
|}
The symbol ^ represents the [[Exponentiation|power operator]].
:{| class="wikitable"
! Token !! Action !! Output<br>(in RPN) !! Operator<br>stack !! Notes
|-
| align="center" | 3 || Add token to output || 3 || ||
|-
| align="center" | + || Push token to stack || 3 || align="right" | + ||
|-
| align="center" | 4 || Add token to output || 3 4 || align="right" | + ||
|-
| align="center" | × || Push token to stack || 3 4 || align="right" | × + || × has higher precedence than +
|-
| align="center" | 2 || Add token to output || 3 4 2 || align="right" | × + ||
|-
| align="center" rowspan="2" | ÷ || Pop stack to output || 3 4 2 × || align="right" | + || ÷ and × have same precedence
|-
| Push token to stack || 3 4 2 × || align="right" | ÷ + || ÷ has higher precedence than +
|-
| align="center" | ( || Push token to stack || 3 4 2 × || align="right" | ( ÷ + ||
|-
| align="center" | 1 || Add token to output || 3 4 2 × 1 || align="right" | ( ÷ + ||
|-
| align="center" | − || Push token to stack || 3 4 2 × 1 || align="right" | − ( ÷ + ||
|-
| align="center" | 5 || Add token to output || 3 4 2 × 1 5 || align="right" | − ( ÷ + ||
|-
| align="center" rowspan="2" | ) || Pop stack to output || 3 4 2 × 1 5 − || align="right" | ( ÷ + || Repeated until "(" found
|-
| Pop stack || 3 4 2 × 1 5 − || align="right" | ÷ + || Discard matching parenthesis
|-
| align="center" | ^ || Push token to stack || 3 4 2 × 1 5 − || align="right" | ^ ÷ + || ^ has higher precedence than ÷
|-
| align="center" | 2 || Add token to output || 3 4 2 × 1 5 − 2 || align="right" | ^ ÷ + ||
|-
| align="center" | ^ || Push token to stack || 3 4 2 × 1 5 − 2 || align="right" | ^ ^ ÷ + || ^ is evaluated right-to-left
|-
| align="center" | 3 || Add token to output || 3 4 2 × 1 5 − 2 3 || align="right" | ^ ^ ÷ + ||
|-
| align="center" | ''end'' || Pop entire stack to output || 3 4 2 × 1 5 − 2 3 ^ ^ ÷ + || ||
|}

Input: {{nowrap|sin ( max ( 2, 3 ) ÷ 3 × {{pi}} )}}<!-- While the result is right, the steps taken are wrong.  If the equation was, for instance, sin(max(-2, 3)÷3*π), the result in RPN would be 2 3 (-) max 3 ÷ π * sin.  (Assuming (-) is the negation function).  Generally, any time you put in a function without a left parenthesis inside a multivariable function, ignoring the comma will result in a wrong answer. -->

:{| class="wikitable"
! Token !! Action !! Output =<br>(in RPN) !! Operator<br>stack !! Notes
|-
| align="center" | sin || Push token to stack ||  || align="right" | sin ||
|-
| align="center" | ( || Push token to stack ||  || align="right" | ( sin ||
|-
| align="center" | max || Push token to stack ||  || align="right" | max ( sin ||
|-
| align="center" | ( || Push token to stack ||  || align="right" | ( max ( sin ||
|-
| align="center" | 2 || Add token to output || 2 || align="right" | ( max ( sin ||
|-
| align="center" | , || ignore || 2 || align="right" | ( max ( sin ||
|-
| align="center" | 3 || Add token to output || 2 3 || align="right" | ( max ( sin || 
|-
| align="center" rowspan="2' | ) || pop stack to output || 2 3 || align="right" | ( max ( sin || Repeated until "(" is at the top of the stack
|-
| Pop stack || 2 3 || align="right" | max ( sin ||Discarding matching parentheses
|-
| align="center" rowspan="2" | ÷ || Pop stack to output || 2 3 max || align="right" | ( sin ||
|-
| Push token to stack  || 2 3 max || align="right" | ÷ ( sin ||
|-
| align="center" | 3 || Add token to output || 2 3 max 3 || align="right" | ÷ ( sin ||
|-
| align="center" rowspan="2" | × || Pop stack to output || 2 3 max 3 ÷ || align="right" | ( sin ||
|-
| Push token to stack || 2 3 max 3 ÷ || align="right" | × ( sin || 
|-
| align="center" | {{pi}} || Add token to output || 2 3 max 3 ÷ {{pi}} || align="right" | × ( sin ||
|-
| align="center" rowspan="2" | ) || Pop stack to output || 2 3 max 3 ÷ {{pi}} × || align="right" | ( sin ||Repeated until "(" is at the top of the stack
|-
| Pop stack || 2 3 max 3 ÷ {{pi}} × || align="right" | sin ||Discarding matching parentheses
|-
| align="center" | ''end'' || Pop entire stack to output || 2 3 max 3 ÷ {{pi}} × sin || ||
|}

==See also==
*[[Operator-precedence parser]]
*[[Stack-sortable permutation]]

==References==
{{Reflist}}

==External links==
*[http://www.cs.utexas.edu/~EWD/MCReps/MR35.PDF Dijkstra's original description of the Shunting yard algorithm]
*[https://web.archive.org/web/20110718214204/http://en.literateprograms.org/Shunting_yard_algorithm_(C) Literate Programs implementation in C] 
*[http://www.chris-j.co.uk/parsing.php Java Applet demonstrating the Shunting yard algorithm]
*[http://www.codeding.com/?article=11 Silverlight widget demonstrating the Shunting yard algorithm and evaluation of arithmetic expressions]
*[http://www.engr.mun.ca/~theo/Misc/exp_parsing.htm Parsing Expressions by Recursive Descent] Theodore Norvell © 1999–2001. Access date September 14, 2006.
*[https://nl.mathworks.com/matlabcentral/fileexchange/68458-evaluation Matlab code, evaluation of arithmetic expressions using the shunting yard algorithm]

{{Parsers}}

[[Category:Parsing algorithms]]

{{Sourceattribution|Shunting-yard algorithm|1}}