Trees In this lecture we upgrade our discussion of self-referential structures from linked lists to trees, by creating TN: a class that includes a value and two references to other objects from the TN class (or None). What seems like a trivial extension turns out to be profound: like going from a 1-dimensional space to a 2-dimensional space. There are entire books written about trees (in both computer science and mathematics), but no books written about linked lists. Over the next two lectures we will examine a few applications for trees. In this lecture, we will discuss binary order trees (search trees) and structure trees (expression trees) and discuss various recursive functions that operate on them. Both use the same defintion of the TN (tree node) class shown below class TN: def __init__(self,value,left=None,right=None): self.value = value self.left = left self.right = right ------------------------------------------------------------------------------ Binary Search Trees A binary tree has a structure and order property. Its structure property dictates that every parent node has 0, 1, or 2 children nodes (called left and right). We draw them with their root on the top, their left and right children below, and their leaves at the bottom (a leaf is a node with 0 children; an internal node has at least one chile). Its order property dictates that all values in the left subtree of any node are less than that node, and all values in the right subtree of any node is greater than that node. Typically binary search trees have unique values. Structurally, binary trees are more interesting than linked lists: structurally (ignoring values) there is only one linked list of length 4: +---+---+ +---+---+ +---+---+ +---+---+ +---+---+ | ? | --+--->| ? | --+--->| ? | --+--->| ? | --+--->| ? | / | +---+---+ +---+---+ +---+---+ +---+---+ +---+---+ But there are 14 different binary trees with four nodes. Here is a list, each has a mirror image (below/above it). ? ? ? / \ / \ / ? ? ? ? ? / \ / \ ? ? ? ? ? ? ? / \ / \ \ ? ? ? ? ? \ / / \ ? ? ? ? ? ? ? ? / / / / ? ? ? ? / / \ \ ? ? ? ? / \ / \ ? ? ? ? ? ? ? ? \ \ \ \ ? ? ? ? \ \ / / ? ? ? ? \ / \ / ? ? ? ? There is just one standard metric for linked lists: length. For trees there are two standard metrics: size and height. Size counts the number of nodes in a tree (therefore it is similar to length for linked lists). It can easily be computed recursively by a function similar to a recursive computation of length. def size(atree): if atree == None: return 0 else: return 1 + size(atree.left) + size(atree.right) There is no simple way to compute size with a loop: for every node in the tree we must visit both its left and right subtrees, so every time that we go left we must also save the right for future references; it can be done with an extra list of nodes, but the code is not simple nor easy to understand. def size_i(atree) nodes = [] size = 0 nodes.append(atree) while len(nodes) > 0: next = nodes.pop(0) if next != None: size += 1; nodes.append(next.left) nodes.append(next.righ) return size The second metric for trees is height. The standard definition of the height of a node is a bit strange: it is the number of steps needed to get from the node to the deepest leaf in either of the node's subtrees. So the height of a leaf (the base case) is 0 and the height of a tree is the height of its root. We can directly translate this definition into the following code. Again there are at most two recursive calls, in the case of a node with two children. def height(atree) if atree.left == None and atree.right == None: # leaf check return 0 elif atree.left == None: return 1 + height(atree.right); elif atree.right == None: return 1 + height(atree.left); else return 1 + max(height(atree.left),height(atree.right)); This function deals with all the necessary cases: a leaf node, an internal node with only a left (or only a right) subtree, and an internal node with both left and right subtrees. This function does not work on empty trees, which have no directly defined height from the previous definition. But, this code is much more complicated than the code for computing size. The problem is with using a leaf node as the base case. Let us simplify this code by using an empty tree as a base case and defining the height of an empty tree to be -1. This might seem like a very strange approach, but it seems reasonable too: an empty tree should have a height that is one less than a leaf node (whose height is 0). By using this definition (and no others), we can simplify the height function dramatically (as well as defining it for all possible trees, even empty ones). def height(atree): if atree == None: return -1 else: return 1+ max(height(atree.left),height(atree.right)) Mathematicians generalize definitions such as this one all the time. For any value a, a**0 is defined as 1. There are many ways to justify this definition (some quite complicated, using limits and calculus); the simplest way is to note the algebraic law a**x * a**y = a **(x+y). By this law (a quite useful one to have) a**0 * a**x = a**(0+x) = a**x; which means that a**00 must be equal to 1 for this identity to hold. Next we will look at methods that convert between trees and lists, showing that there is a standard way to represent a tree as a nested list of values. We represent every TN as a 3-list containing the value, left, and right subtrees (each subtree is itself a 3-list). So, we represent the tree 5 / \ 3 8 by the list [5 [3 None None] [8 None None]]. Note that each list present has 3 values. There are simple recursive functions to translate from a tree to a list and a list to a tree. Again, each uses two recursive calls def list_to_tree(alist): if alist == None: return None else: return TN(alist[0],list_to_tree(alist[1]),list_to_tree(alist[2])) Each recursive call on a non-empty list builds a TN with the value, and then uses the two sublists to build other (eventually empty) subtrees. def tree_to_list(atree): if atree == None: return None else: return [atree.value,tree_to_list(atree.left),tree_to_list(atree.right)] Each recursive call on a non-empty tree builds a list with the value, followed by the two sublists built from the (eventually empty) subtrees. The following function prints a tree rotated 90 degree counter-clockwise. So the tree we would show as 30 / \ 15 50 / \ / \ 10 25 35 70 / 20 prints as ....70 ..50 ....35 30 ....25 ......20 ..15 ....10 This function declares print_tree_1, a helper function that does all the work, and then calls print_tree_1 with an initial identation of 0. The helper function either does nothing (for printing an empty tree), or prints all values in its right subtree (with more indentation), its own value, and then all values in its left subtree (with more indentation). def print_tree(atree,indent_char =' ',indent_delta=2): def print_tree_1(indent,atree): if atree == None: return None else: print_tree_1(indent+indent_delta, atree.right) print(indent*indent_char+str(atree.value)) print_tree_1(indent+indent_delta, atree.left) print_tree_1(0,atree) At this point we have dealt with the structure of trees, but not their values. In a binary search trees, we can use the value property to search for, add a value, and remove a value. We can use the following iterative function to search for a value; unlike the other functions written above, this one goes only one way (left or right) for each tree node. We know if the value we are searching for is less than a node's value, it must be in the left subtree; if the value we are searching for is greater than a node's value, it must be in the right subtree. def search_i(atree,value): while atree != None and atree.value != value: atree = atree.left if value < atree.value else atree.right return atree # either None or the TN storing value We can also write this function recursively. def search(atree,value): if atree == None: return None if value == atree.value: return atree elif value < atree.value: return search(atree.left,value) else: # value > atree.value return search(atree.right,value) and we can shorten this function to the following def search(atree,value): if atree == None or atree.value == value return atree else: return search(atree.left if value < atree.value else atree.right ,value) In the function above, the "base" case is an empty tree or the node storing the value; the same recursive call is executed for subtrees, with the first "smaller" tree (having fewer nodes) being either atree.left or atree.right. There is a similar (to the top) function to add a value to a tree. We call it like: atree = add(atree,value). def add(atree,value): if atree == None: return TN(value) if value < atree.value: atree.left = add(atree.left,value) return atree elif value > atree.value: atree.right = add(atree.right,value) return atree else: return atree # already in tree In all cases, this function returns a tree to which a TN with value has been added. Note that if the root is None, it returns a reference to a TN with value; if node is not empty, it returns that node, but either its left of right subtree (as appropriate) now refers to a tree into which value has been added. Note that the structure of a tree is not determined by the values it contains. It is determined by the order those values are added. Adding values in increasing order, decreasing order, at random, will all lead to different shaped trees. I will defer showing the remove method, but I will describe it here and you should use this description to practice deleting values from trees. Use the following simple tree for a first 30 / \ 15 50 / \ / \ 10 25 35 70 / 20 Here are the rules: 1) To remove a leaf, make its parent refer to None 2) To remove a node with one child, make its parent refer to its child 3) To remove a node with 2 children: (a) Find the biggest node less than it (or smallest node greater than it) that node will have 0 or 1 children (why) (b) Remove that node by rule 1 or 2 (c) Take its value and move it to the node being removed So the node being removed isn't really removed (another one is): its value is replaced The first two rules are very simple. If we remove the root, 30, we would (a) find the node 25, (b) remove it by making 15's right refer to 20, (c) move the value 25 to 30. Note the order property is preserved: all values to the left of the node that used to store 30 are less than what it now stores, 25 (25 was the biggest of the nodes < 30); all values to the right of the node that used to store 30 are greater than what it now stores, 25 (25 is < 30). The module contains simple recursive functions for copying a tree and determining whether two trees are equal (not only store the same values, but store trees that look identical). Finally, the generator function generates all the values (from lowest to highest) in the tree. In ICS-46 we will study traversal orders and recognize this as in-order traversal. We can use binary search trees easily to create a dictionaries whose values are always iterated over in a sorted order. The penalty for this constraint is the amount of time to search, add, and remove are a bit higher than for dict (but still much less than using lists). It is easy to imagine how a binary search tree can store key-value pairs. We can search a well-balanced tree must faster than a linked list (even an ordered one). The amount of time it takes to search a reasonable binary search tree is proportional to its height, which for a N node tree must be at least Log2 N (log base 2 of the number of nodes in the tree), but is typically not more than twice that number. Try the following experiemnt values = [i for i in range(1000)] random.shuffle(values) print(height(add_all(None,values))) The Log2 1000 is about 10, so the typical height of such a tree is about 20, which means it takes 20 comparisons to find a value: much better than the average of about 500 if the values are in an unordered list or linked-list. We will learn a bit more about this performance in ICS-33, but will learn much more about it in ICS-46, and look at tree processing in more depth there. ------------------------------------------------------------------------------ Expression Trees We can also use binary trees to represent expressions. In these trees, leaf nodes represent values (either literals or names bound to values), and the internal nodes represent binary operators or unary operators or unary functions (whose operands will be in the right subtree). For example, the expression (-b + sqrt(b**2 - 4*a*c))/(2*a) would be represented by the expression tree. '/' / \ + * / \ / \ - sqrt 2 a \ \ b - / \ ** * / \ / \ b 2 * c / \ 4 a Here I wrote '/' for the divide operator, since / means a left subtree. Note that the structure of the tree determines how the subexpressions are computed. There is no need for operator precedence rules or parenthese: the structureof the tree embodies these rules. There is an algorithm that people can follow to construct such a tree: find the last operator or function call the computer would evaluate and put that at the root of the tree; now find the root of its one/two subtrees that are subexpressions, and keep repeating finding the root of these until there are no more operators or functions (names and literals stand for themselves). In the expression above, the division between the numerator and denominator is evaluated last: on the leftr side the division is evaluated last; on the right side there is only the multiplications, so that is done last. Continue this process. If we call print_tree on this tree, it would print ....2 ..* ....2 / ..........c ........* ............a ..........* ............4 ......- ..........2 ........** ..........b ....sqrt ..+ ......b ....- Once we have such a tree, we can peform many operations on it. The first and most important is evaluating the tree. We can do this recursively (evaluating subexpressions) by (1) evaluating leaves as themselves (2) evaluating either unary operators on their evaluated operand or unary functions on their evaluated argument (3) evaluating binary operators on their evaluated arguments The code for this method follows this outline def evaluate(etree): #name/literal if etree.left == None and etree.right == None: return eval(str(etree.value)) #unary operator/function call elif etree.left == None: if etree.value in ['+','-','*','/','//','**']: return eval(etree.value + str(evaluate(etree.right))) else: return eval(etree.value+'('+str(evaluate(etree.right))+')') #binary operator else: return eval(str(evaluate(etree.left)) + etree.value + str(evaluate(etree.right))) If we set a=1, b=2, c=1, the calcuated value is -1. We can translate this tree into infix (but overparenthesized) and postfix form: in the postfix form, each operator is proceeded by its two operands: a + 1 (infix form) translates to 1 a +. Using postfix notation (also called Polish notation because it was invented by Polish logicians right before World War II), we can write expressions unambiguously without any parentheses or knowledge of operator precedence! Here are the functions to perform these translations, and their results. def infix(etree): if etree.left == None and etree.right == None: return '('+str(etree.value)+')' elif etree.left == None: return '('+etree.value+str(infix(etree.right))+')' else: return '('+str(infix(etree.left))+etree.value+str(infix(etree.right))+')' which produces: (((-(b))+(sqrt(((b)**(2))-(((4)*(a))*(c)))))/((2)*(a))) def postfix(etree): if etree.left == None and etree.right == None: return str(etree.value) elif etree.left == None: return str(postfix(etree.right)) + ' ' + etree.value else: return str(postfix(etree.left)) + ' ' + str(postfix(etree.right)) + ' ' + etree.value which produces: b - b 2 ** 4 a * c * - sqrt + 2 a * / If you have never seen Polish notation this is difficult to read, but if you have studied this notation, it is easy. To understand which operators apply to which data, start on the left and circle each operand: when you get to an operand circle it and the number of operands it takes (before it). Finally, I have defined a parse_infix function that takes a string argument and produces a tree representing the string. It is limited in the follolwing ways: all tokens must be separated by spaces; it assumes all operators are binary, and that all operators are left-associative (which ** is not). ------------------------------------------------------------------------------ In the second lecture on trees we will see how to use sets and dictionaries to store N-ary trees (trees with an arbitrary number of children).