Cherry-pick from https://gerrit.onos.onlab.us/#/c/339/

Cherry-pick from https://gerrit.onos.onlab.us/#/c/67

Implemented bypassing function to FlowPusher so that FlowSynchronizer can
push flows while queue is suspended (ONOS-967).

In this commit I modified FlowPusher to have two (or more, in future)
queues to prioritize messages.
High priority queue is for flow synchronization. This queue won't be
blocked even in SUSPENDED status.
Low priority queue is for normal OpenFlow message installation. This
queue works only in READY status, but blocked in SUSPENDED status.
Messages in high priority queue are always installed before messages in
low priority queue. In other words, messages in low priority queue are
installed only if high priority queue is empty.

Moved timing of FlowEntryAdded notification.

Change-Id: Ic7ee4ce002c1f6b50af4f160fbf98eb82658ee8b

Implemented bypassing function to FlowPusher.

Change-Id: Ia3ae86242ceed34257c58e9aaffcdd8f72534bf3

Fixed a bug FlowPusher#hasMessageToSend doesn't work with SUSPENDED status.

Conflicts:
        src/main/java/net/onrc/onos/ofcontroller/flowprogrammer/FlowPusher.java

NOTE: The above conflict was resolved by hand.

Change-Id: I79446dced030ecd91a6e81333cb93c1c4b637914
diff --git a/src/main/java/net/onrc/onos/ofcontroller/flowprogrammer/FlowSynchronizer.java b/src/main/java/net/onrc/onos/ofcontroller/flowprogrammer/FlowSynchronizer.java
index 17ef14c..c1571c8 100644
--- a/src/main/java/net/onrc/onos/ofcontroller/flowprogrammer/FlowSynchronizer.java
+++ b/src/main/java/net/onrc/onos/ofcontroller/flowprogrammer/FlowSynchronizer.java
@@ -29,6 +29,7 @@
 import net.onrc.onos.ofcontroller.core.INetMapTopologyObjects.IFlowEntry;
 import net.onrc.onos.ofcontroller.core.INetMapTopologyObjects.ISwitchObject;
 import net.onrc.onos.ofcontroller.flowmanager.FlowDatabaseOperation;
+import net.onrc.onos.ofcontroller.flowprogrammer.IFlowPusherService.MsgPriority;
 import net.onrc.onos.ofcontroller.util.Dpid;
 import net.onrc.onos.ofcontroller.util.FlowEntry;
 import net.onrc.onos.ofcontroller.util.FlowEntryId;
@@ -97,8 +98,7 @@
 	double graphIDTime, switchTime, compareTime, graphEntryTime, extractTime, pushTime, totalTime;
 	@Override
 	public SyncResult call() {
-	    // TODO: stop adding other flow entries while synchronizing
-	    //pusher.suspend(sw);
+	    pusher.suspend(sw);
 	    long start = System.nanoTime();
 	    Set<FlowEntryWrapper> graphEntries = getFlowEntriesFromGraph();
 	    long step1 = System.nanoTime();
@@ -115,7 +115,7 @@
 	    compareTime = (step3 - step2);
 	    totalTime = (step3 - start);
 	    outputTime();
-	    //pusher.resume(sw);
+	    pusher.resume(sw);
 	    
 	    return result;
 	}
@@ -301,7 +301,7 @@
 	    extractTime = System.nanoTime() - startExtract;
 
 	    double startPush = System.nanoTime();
-	    pusher.pushFlowEntry(sw, flowEntry);
+	    pusher.pushFlowEntry(sw, flowEntry, MsgPriority.HIGH);
 	    pushTime = System.nanoTime() - startPush;
 	}
 
@@ -325,7 +325,7 @@
 	    fm.setPriority(statisticsReply.getPriority());
 	    fm.setOutPort(OFPort.OFPP_NONE);
 
-	    pusher.add(sw, fm);
+	    pusher.add(sw, fm, MsgPriority.HIGH);
 	}
 
 	/**