Graph Algorithms I


Topological Sorting

First, we will discuss Topological Sorting. Here the domain is a graph where
an edge from A to B means node A "comes before" node B (or, if you will, node
A is less than node B). Unlike normal types where the law of trichotomy holds
(for any two values, one is <, =, or > than the other) two values might be
"uncomparable" meaning their order is not directly specified: there is no edge
from A to B or from B to A. The edges in graph supply a partial order of the
nodes. A topological sort of a graph is a list of nodes such that all the
contstraints (edges saying which nodes come before which  others) are
satisified in the resulting list. In the edge list a, b, c, ... then either 
there is an edge a->b or no edge between a and b,; there is an edge b->c or no
edge between b and c; ...

A simple algorithm for doing topological sorting follows. It produces a list
of nodes satisfying all the edge constraints (a List<String>).

  while there is a node with in-degree 0
    remove it from the graph
    add it to (put it next in) the answer list

This process continues until (a) all nodes are removed from graph or (b) some
nodes still remain n the graph, but no node with in-degree = 0 can be found. In
the latter case, the graph is cyclic: there are nodes a, b, c, ... ,x, y such
that a must come before b, b must come before c, ... x must come before y, and
y must come before a (which is impossible). If no node has degree 0, then every
node is the destination of another (only possible if there is a cycle).

We can characterize this algorithm as O(N^2): finding an in-degree 0 node is
O(N) (iterate through all the nodes), and in a non-cyclic graph this will
happen N times before the graph is empty. Note that in the worst case, a graph
will have O(N^2) edges, each of which must be removed by the algorithm above
before getting to the empty graph.

We will apply this algorithm to generate one (of a few different) topological
orders (we have lattitude to pick an in-degree 0 node; different picks lead
to different final orders, but all satisifying the required constraints) for
the graph below (draw a picture of this graph before starting).

A -> D
A -> E
B -> D
C -> E
C -> H
D -> F
D -> G
D -> H
E -> G

One order is: A, B, C, D, E, F, G, H
Another is  : B, A, C, D, F, H, E, G


Minimum Spanning Tree (MST):

A spanning tree of a undirected graph is a subgraph with no redundant edges,
such that every node is reachable from every other node. That means in a graph
with N nodes, will have exactly N-1 edges in its spanning tree. The graph must
be connected to have a spanning tree. We can prove we need only N-1 edges by
induction. In all 1 node graphs, we need only 0 edges to have the graph be
connected (satisfying the N-1 requirement); if we take any connected N node
graph and add another node, all we must do is add one more edge (from any of
the N nodes to the new one) to have the graph be connected again.

Note that spanning trees are acyclic graphs, because any cyclic structure has a
redundant edge (remove any edge in the cycle and the graph is still a spanning
tree).

We call this graph a spanning "tree" because we can look at each graph as
a tree: chose any node as the root; all the nodes reachable from it become
the root's children; the nodes reachable from these (those not already reached)
become the grandchildren, etc. Choosing a different root will yield a different
tree.

A minimum spanning tree (MST) is a spanning tree whose cost (sum of its edges)
is the minimum possible.

If we wanted to connect various cities with a phone network, and we knew the
cost of laying the fiber-optic cable between any two cities (it might not just
be proportional to the the linear distance between the cities: this cost is the
value on an edge between two cities), finding the MST would solve the problem.

Kruskal's algorithm is one efficient way to find a MST. To run this algorithm,
we must be able to tell whether an edge connects two nodes that do not yet (in
the MST we are building) have a path between them. We can easily "see" this
property on small graphs, so for now we will assume that we can perfrom this
operation efficiently (O(1)). In the next lecture we will learn about a new
data type called "equivalence classes" and about the union/find problem, and
see how to solve this problem on the computer efficiently (and in Quiz #8 you
will implement a HashEquivalence class).

Kruskal's Algorithm for MST

  1) Put each node in own cluster (we will just visually see clusters)
     We also call these clusters equivalence classes: nodes in the same
       equivalence class are all reachable from each other

  2) Put all the edges in priority queue (edge with the smallest value has
       highest priority)

  3) Start with empty graph T that will represent the answer

  4) While T's # edges < N-1 (see above: N node graphs have N-1 edge MSTs)
       a) Remove the next edge from the priority queue (smallest edge value)
            If there are no more edges, the graph isn't connected and there is
            no [minimal] spanning tree 
       b) If it connects two disconnected clusters (e.g., if its origin and
            destination node are in different equivalence classes), add the
            edge to T and merge the clusters containing origin and destination
            node (put those two nodes and all the nodes in their equivalence
            classes into one big equivalence class).

In a dense graph, this algorithm is O(N^2 Log2 N^2), which is O(N^2 Log2 N):
we can put all edges in the priority queue using an offline algorithm in 
O(N^2). We might have to remove each of these edges from the priority queus as
well (which would take O(N^2 Log2 N^2) which is O(N^2 Log2 N) -imagine a graph
with the longest edge going to a node that is connected to no other node, all
other nodes would have to be processed first. In a sparse graph, with O(N)
edges, this algorithm is just O(N Log2 N), therefore "fast" (like fast
sorting). In fact, we might best state the complexity of this algorithm in
terms of the number of edges E: E Log2 E.

Kruskal's algorithm is known as a "Greedy algorithm". In a greedy algorithm a 
locally correct choice (here the next smallest edge to process) is guaranteed
to be in the solution. So, once we choose this edge to be part of a MST, we
will never have to "unchoose" it. Recall in the standard backtracking search
algorithms (like N-queens) sometimes we make choices that we have to "unmake",
choosing something else instead (so such an algorithm is not greedy). If a
problem is solvable by a greedy algorithm, the choices the algorithm makes
never need to be un-made (and typically its complexity class can be lower than
for a backtracking search).

An example of a problem where a greedy algorithm fails is making change using
US currency. Normally you'd first find the number of quarters needed, then
find the number of dimes, then nickels, then pennies. But suppose you want to
make 30 cents change but in the cash register has only 1 quarter, 3 dimes, and
no nickels or pennies. A greedy algorithm would choose the quarter first, but
be unable to choose any dimes, and fail to solve the problem. A backtracking
search algorithm would undo the choice of 1 quarter, then choose 3 dimes to
solve the problem.

There is a dual algorithm to Kruskals, which starts with the original connected
graph and removes edges (starting with the longest), so long as the graph
remains connected when the edge is removed. The original algorithm works best
if there are many more edges than those required in the MST; the dual algorithm
works best if the graph is so sparse that almost all the edges in the graph are
in the spanning tree (so there are very few to remove before getting to just
n-1 edges).

Applying Kruskal's algorithm (or its dual) to a graph with unique edge costs
will produce a unique MST. If some edge costs are duplicates, then there might
be multipl MSTs with the same total cost.

We applied this algorithm to generate an MST for the graph below. Here
A<-(4)->B means the cost of the undirected edge between A and B is 4.

A<-(4)->B
A<-(1)->C
A<-(4)->D
B<-(5)->C
B<-(7)->G
B<-(9)->E
B<-(9)->F
C<-(3)->D
D<-(10)->G
D<-(18)->J
E<-(2)->F
E<-(4)->H
E<-(6)->I
F<-(8)->G
F<-(2)->H
G<-(9)->H
G<-(8)->J
H<-(3)->I
H<-(9)->J
I<-(9)->J

If we put these values in the priority queue, one order (there are multiple
edges with the same value). Also, initially each node is in its own separate
cluster/equivalence class

A<-(1)->C
E<-(2)->F
F<-(2)->H
C<-(3)->D
H<-(3)->I
A<-(4)->B
A<-(4)->D
E<-(4)->H
B<-(5)->C
E<-(6)->I
B<-(7)->G
F<-(8)->G
G<-(8)->J
B<-(9)->E
B<-(9)->F
G<-(9)->H
H<-(9)->J
I<-(9)->J
D<-(10)->G
D<-(18)->J

Note that this graph has 10 nodes so the MST will have 9 edges, so we can
terminate the algorithm when we reach 9 edges.

 1) We remove A<-(1)->C; A and C are in different clusters, so we put this edge
in the MST (it now has 1 edge) and update the clusters so that now {A,C} are in
the same cluster.

 2) We remove E<-(2)->F; E and F are in different clusters, so we put this edge
in the MST (it now has 2 edges) and update the clusters so that now {E,F} are
in the same cluster.

 3) We remove F<-(2)->H; F and H are in different clusters, so we put this edge
in the MST (it now has 3 edges) and update the clusters so that now {E,F,H} are
in the same cluster.

 4) We remove C<-(3)->D; C and D are in different clusters, so we put this edge
in the MST (it now has 4 edges) and update the clusters so that now {A,C,D} are
in the same cluster.

 5) We remove H<-(3)->I; H and I are in different clusters, so we put this edge
in the MST (it now has 5 edges) and update the clusters so that now {E,F,H,I}
are in the same cluster.

 6) We remove A<-(4)->B; A and B are in different clusters, so we put this edge
in the MST (it now has 6 edges) and update the clusters so that now {A,B,C,D}
are in the same cluster.

 7) We remove A<-(4)->D; A and D are already in the same cluster (see 6) so we
don't include this edge in the MST. This is the first time that we are
discarding an edge (not using it in the result graph).

 8) We remove E<-(4)->H; E and H are already in the same cluster (see 5) so we
don't include this edge in the MST.

 9) We remove B<-(5)->C; B and C are already in the same cluster (see 6) so we
don't include this edge in the MST.

 9) We remove E<-(6)->I; E and I are already in the same cluster (see 5) so we
don't include this edge in the MST.

10) We remove B<-(7)->G; B and G are in different clusters, so we put this edge
in the MST (it now has 7 edges) and update the clusters so that now {A,B,C,D,G}
are in the same cluster.

11) We remove F<-(8)->G; F and G are in different clusters, so we put this edge
in the MST (it now has 8 edges) and update the clusters so that now
{A,B,C,D,E,F,G,H,I} are in the same cluster. This edge connects two formerly
unconnected components.

11) We remove G<-(8)->J; G and J are in different clusters, so we put this edge
in the MST (it now has 9 edges) and update the clusters so that now
{A,B,C,D,E,F,G,H,I,J} are in the same cluster.

Note, we now have the needed 9 edges, and we see that all the nodes are in the
same cluster (all reachable from each other). So, the following nodes are in
the MST

A<-(1)->C
E<-(2)->F
F<-(2)->H
C<-(3)->D
H<-(3)->I
A<-(4)->B
B<-(7)->G
F<-(8)->G
G<-(8)->J

With a total cost of 1+2+2+3+3+4+7+8+8 = 38.