TestON/logs/OnosScale4nodes_21_Nov_2013_13_20_21/OnosScale4nodes_21_Nov_2013_13_20_21.log

                                +----------------+
------------------------------ { Script And Files }  ------------------------------
                                +----------------+

	Script Log File : /home/admin/ONLabTest/TestON/logs/OnosScale4nodes_21_Nov_2013_13_20_21/OnosScale4nodes_21_Nov_2013_13_20_21.log
	Report Log File : /home/admin/ONLabTest/TestON/logs/OnosScale4nodes_21_Nov_2013_13_20_21/OnosScale4nodes_21_Nov_2013_13_20_21.rpt
	ONOS4 Session Log : /home/admin/ONLabTest/TestON/logs/OnosScale4nodes_21_Nov_2013_13_20_21/ONOS4.session
	Cassandra3 Session Log : /home/admin/ONLabTest/TestON/logs/OnosScale4nodes_21_Nov_2013_13_20_21/Cassandra3.session
	ONOS2 Session Log : /home/admin/ONLabTest/TestON/logs/OnosScale4nodes_21_Nov_2013_13_20_21/ONOS2.session
	ONOS3 Session Log : /home/admin/ONLabTest/TestON/logs/OnosScale4nodes_21_Nov_2013_13_20_21/ONOS3.session
	ONOS1 Session Log : /home/admin/ONLabTest/TestON/logs/OnosScale4nodes_21_Nov_2013_13_20_21/ONOS1.session
	Cassandra1 Session Log : /home/admin/ONLabTest/TestON/logs/OnosScale4nodes_21_Nov_2013_13_20_21/Cassandra1.session
	Cassandra2 Session Log : /home/admin/ONLabTest/TestON/logs/OnosScale4nodes_21_Nov_2013_13_20_21/Cassandra2.session
	Mininet1 Session Log : /home/admin/ONLabTest/TestON/logs/OnosScale4nodes_21_Nov_2013_13_20_21/Mininet1.session
	Cassandra4 Session Log : /home/admin/ONLabTest/TestON/logs/OnosScale4nodes_21_Nov_2013_13_20_21/Cassandra4.session
	Zookeeper1 Session Log : /home/admin/ONLabTest/TestON/logs/OnosScale4nodes_21_Nov_2013_13_20_21/Zookeeper1.session
	Test Script :/home/admin/ONLabTest/TestON/Tests/OnosScale4nodes.py
	Test Params : /home/admin/ONLabTest/TestON/Tests/OnosScale4nodes.params
	Topology : /home/admin/ONLabTest/TestON/Tests/OnosScale4nodes.tpl
                              +------------------+
---------------------------  { Script Exec Params }  ---------------------------
                              +------------------+

	
	'TargetTime': '50'
	 'testcases': '1
	2
	3
	4'
	 'WaitTime': '50'
	 'NR_Links': '198'
	 'RestIP': '10.128.100.4'
	 'NR_Switches': '100'
	 'Iterations': '2'
	

                               +---------------+
----------------------------- { Components Used }  -----------------------------
                               +---------------+
	ONOS4
	Cassandra3
	ONOS2
	ONOS3
	ONOS1
	Cassandra1
	Cassandra2
	Mininet1
	Cassandra4
	Zookeeper1


                              +--------+
---------------------------- { Topology }  ----------------------------
                              +--------+

	
	'ONOS4': 
	'connect_order': '10'
	 'password': None
	 'type': 'OnosCliDriver'
	 'host': '10.128.100.6'
	 'user': 'admin'
	
	 'Cassandra3': 
	'connect_order': '5'
	 'password': None
	 'type': 'CassandraCliDriver'
	 'host': '10.128.100.5'
	 'user': 'admin'
	
	 'ONOS2': 
	'connect_order': '8'
	 'password': None
	 'type': 'OnosCliDriver'
	 'host': '10.128.100.4'
	 'user': 'admin'
	
	 'ONOS3': 
	'connect_order': '9'
	 'password': None
	 'type': 'OnosCliDriver'
	 'host': '10.128.100.5'
	 'user': 'admin'
	
	 'ONOS1': 
	'connect_order': '7'
	 'password': None
	 'type': 'OnosCliDriver'
	 'host': '10.128.100.1'
	 'user': 'admin'
	
	 'Cassandra1': 
	'connect_order': '3'
	 'password': None
	 'type': 'CassandraCliDriver'
	 'host': '10.128.100.1'
	 'user': 'admin'
	
	 'Cassandra2': 
	'connect_order': '4'
	 'password': None
	 'type': 'CassandraCliDriver'
	 'host': '10.128.100.4'
	 'user': 'admin'
	
	 'Mininet1': 
	'connect_order': '25'
	 'host': '10.128.100.11'
	 'user': 'admin'
	 'COMPONENTS': 
	'arg1': ''
	 'arg2': '--topo linear
	100 --arp'
	 'controller': 'remote
	ip=10.128.100.1
	port=6633'
	
	 'password': None
	 'type': 'MininetCliDriver'
	
	 'Cassandra4': 
	'connect_order': '6'
	 'password': None
	 'type': 'CassandraCliDriver'
	 'host': '10.128.100.6'
	 'user': 'admin'
	
	 'Zookeeper1': 
	'connect_order': '2'
	 'host': '10.128.100.1'
	 'user': 'admin'
	 'COMPONENTS': ''
	 'password': None
	 'type': 'ZookeeperCliDriver'
	
	
------------------------------------------------------------


******************************
 CASE INIT 
******************************

2013-11-21 13:20:21,419 - OnosScale4nodes - INFO - Creating component Handle: Zookeeper1
2013-11-21 13:20:21,488 - OnosScale4nodes - INFO - Password not required logged in
2013-11-21 13:20:21,592 - OnosScale4nodes - INFO - Starting Zookeeper
2013-11-21 13:20:24,319 - OnosScale4nodes - WARNING - zookeeper ... already running
2013-11-21 13:20:24,320 - OnosScale4nodes - INFO - Creating component Handle: Cassandra1
2013-11-21 13:20:24,380 - OnosScale4nodes - INFO - Password not required logged in
2013-11-21 13:20:24,484 - OnosScale4nodes - INFO - Creating component Handle: Cassandra2
2013-11-21 13:20:24,544 - OnosScale4nodes - INFO - Password not required logged in
2013-11-21 13:20:24,648 - OnosScale4nodes - INFO - Creating component Handle: Cassandra3
2013-11-21 13:20:24,706 - OnosScale4nodes - INFO - Password not required logged in
2013-11-21 13:20:24,810 - OnosScale4nodes - INFO - Creating component Handle: Cassandra4
2013-11-21 13:20:24,867 - OnosScale4nodes - INFO - Password not required logged in
2013-11-21 13:20:24,970 - OnosScale4nodes - INFO - Creating component Handle: ONOS1
2013-11-21 13:20:25,024 - OnosScale4nodes - INFO - Password not required logged in
2013-11-21 13:20:25,127 - OnosScale4nodes - INFO - Creating component Handle: ONOS2
2013-11-21 13:20:25,188 - OnosScale4nodes - INFO - Password not required logged in
2013-11-21 13:20:25,292 - OnosScale4nodes - INFO - Creating component Handle: ONOS3
2013-11-21 13:20:25,351 - OnosScale4nodes - INFO - Password not required logged in
2013-11-21 13:20:25,454 - OnosScale4nodes - INFO - Creating component Handle: ONOS4
2013-11-21 13:20:25,525 - OnosScale4nodes - INFO - Password not required logged in
2013-11-21 13:20:25,629 - OnosScale4nodes - INFO - Creating component Handle: Mininet1
2013-11-21 13:20:25,696 - OnosScale4nodes - INFO - Password not required logged in
2013-11-21 13:20:25,799 - OnosScale4nodes - INFO - Clearing any residual state or processes
2013-11-21 13:20:28,821 - OnosScale4nodes - INFO - Clean
2013-11-21 13:20:28,822 - OnosScale4nodes - INFO - building fresh mininet
2013-11-21 13:20:29,061 - OnosScale4nodes - INFO -  Creating network
2013-11-21 13:20:29,062 - OnosScale4nodes - INFO -  Adding controller
2013-11-21 13:20:29,628 - OnosScale4nodes - INFO -  Adding hosts:
2013-11-21 13:20:30,766 - OnosScale4nodes - INFO -  Adding switches:
2013-11-21 13:20:34,623 - OnosScale4nodes - INFO -  Adding links:
2013-11-21 13:21:04,960 - OnosScale4nodes - INFO -  Configuring hosts
2013-11-21 13:22:07,995 - OnosScale4nodes - INFO -  Starting controller
2013-11-21 13:22:07,996 - OnosScale4nodes - INFO -  Starting 100 switches
2013-11-21 13:23:07,249 - OnosScale4nodes - INFO -  Starting CLI:
2013-11-21 13:23:07,262 - OnosScale4nodes - INFO - mininet built


******************************
 Result summary for Testcase1
******************************

2013-11-21 13:23:08,102 - OnosScale4nodes - WARNING - ONOS wasn't running
2013-11-21 13:23:08,896 - OnosScale4nodes - WARNING - ONOS wasn't running
2013-11-21 13:23:09,732 - OnosScale4nodes - WARNING - ONOS wasn't running
2013-11-21 13:23:11,393 - OnosScale4nodes - WARNING - ONOS wasn't running
2013-11-21 13:23:11,394 - OnosScale4nodes - INFO - Starting Cassandra
2013-11-21 13:23:16,974 - OnosScale4nodes - INFO - Cassandra Started 
2013-11-21 13:23:16,975 - OnosScale4nodes - INFO - Starting Cassandra
2013-11-21 13:23:22,633 - OnosScale4nodes - INFO - Cassandra Started 
2013-11-21 13:23:22,634 - OnosScale4nodes - INFO - Starting Cassandra
2013-11-21 13:23:28,233 - OnosScale4nodes - INFO - Cassandra Started 
2013-11-21 13:23:28,234 - OnosScale4nodes - INFO - Starting Cassandra
2013-11-21 13:23:33,943 - OnosScale4nodes - INFO - Cassandra Started 
2013-11-21 13:23:56,788 - OnosScale4nodes - INFO - Keyspace dropped
2013-11-21 13:24:06,374 - OnosScale4nodes - INFO - ONOS Started 
2013-11-21 13:24:27,167 - OnosScale4nodes - INFO - ONOS Started 
2013-11-21 13:24:38,356 - OnosScale4nodes - INFO - ONOS Started 
2013-11-21 13:24:49,759 - OnosScale4nodes - INFO - ONOS Started 
2013-11-21 13:24:52,184 - OnosScale4nodes - INFO - Executed :~/ONOS/start-rest.sh start 		 Expected Prompt '\$' Found
2013-11-21 13:24:52,186 - OnosScale4nodes - INFO - Rest Server Started Successfully
2013-11-21 13:25:02,379 - OnosScale4nodes - INFO - Executed :~/ONOS/start-rest.sh status  		 Expected Prompt 'running' Found
2013-11-21 13:25:02,380 - OnosScale4nodes - INFO - Rest Server is running
2013-11-21 13:25:02,482 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found

[2013-11-21 13:25:02.599301] [OnosScale4nodes] [CASE]  Checking if the startup was clean...

[2013-11-21 13:25:02.602258] [OnosScale4nodes] [STEP]  INIT : Initializing the test case :Checking if the startup was clean...

[2013-11-21 13:25:02.605514] [OnosScale4nodes] [STEP]  1.1: Testing startup Zookeeper
2013-11-21 13:25:02,815 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:25:03,689 - OnosScale4nodes - INFO - Executed :~/zookeeper-3.4.5/bin/zkServer.sh status  		 Expected Prompt 'Mode' Found
2013-11-21 13:25:03,693 - OnosScale4nodes - INFO - Verifying the Expected is equal to the actual or not using assert_equal
2013-11-21 13:25:03,694 - OnosScale4nodes - INFO - Zookeeper is up!
2013-11-21 13:25:03,695 - OnosScale4nodes - INFO - Assertion Passed

[2013-11-21 13:25:03.697230] [OnosScale4nodes] [STEP]  1.2: Testing startup Cassandra
2013-11-21 13:25:03,700 - OnosScale4nodes - INFO - trying 5 times
2013-11-21 13:25:03,887 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:25:05,682 - OnosScale4nodes - INFO - Verifying the Expected is equal to the actual or not using assert_equal
2013-11-21 13:25:05,683 - OnosScale4nodes - INFO - Cassandra is up!
2013-11-21 13:25:05,684 - OnosScale4nodes - INFO - Assertion Passed

[2013-11-21 13:25:05.686144] [OnosScale4nodes] [STEP]  1.3: Testing startup ONOS
2013-11-21 13:25:05,790 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:25:06,732 - OnosScale4nodes - INFO - Executed :~/ONOS/start-onos.sh status  		 Expected Prompt 'running' Found
2013-11-21 13:25:06,834 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:25:06,937 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:25:37,070 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:25:37,172 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:25:37,174 - OnosScale4nodes - INFO - ONOS process is running...
2013-11-21 13:25:37,174 - OnosScale4nodes - INFO - Assertion Passed
2013-11-21 13:25:37,185 - OnosScale4nodes - INFO - Onos log is moving! It's looking good!
2013-11-21 13:25:37,390 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:25:38,037 - OnosScale4nodes - INFO - Executed :~/ONOS/start-onos.sh status  		 Expected Prompt 'running' Found
2013-11-21 13:25:38,139 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:25:38,241 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:26:08,372 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:26:08,475 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:26:08,476 - OnosScale4nodes - INFO - ONOS process is running...
2013-11-21 13:26:08,477 - OnosScale4nodes - INFO - Assertion Passed
2013-11-21 13:26:08,495 - OnosScale4nodes - INFO - Onos log is moving! It's looking good!
2013-11-21 13:26:08,685 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:26:09,432 - OnosScale4nodes - INFO - Executed :~/ONOS/start-onos.sh status  		 Expected Prompt 'running' Found
2013-11-21 13:26:09,535 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:26:09,636 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:26:39,769 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:26:39,871 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:26:39,872 - OnosScale4nodes - INFO - ONOS process is running...
2013-11-21 13:26:39,872 - OnosScale4nodes - INFO - Assertion Passed
2013-11-21 13:26:39,892 - OnosScale4nodes - INFO - Onos log is moving! It's looking good!
2013-11-21 13:26:40,129 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:26:40,815 - OnosScale4nodes - INFO - Executed :~/ONOS/start-onos.sh status  		 Expected Prompt 'running' Found
2013-11-21 13:26:40,917 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:26:41,019 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:27:11,136 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:27:11,238 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:27:11,240 - OnosScale4nodes - INFO - ONOS process is running...
2013-11-21 13:27:11,240 - OnosScale4nodes - INFO - Assertion Passed
2013-11-21 13:27:11,251 - OnosScale4nodes - INFO - Onos log is moving! It's looking good!
2013-11-21 13:27:11,252 - OnosScale4nodes - INFO - Verifying the Expected is equal to the actual or not using assert_equal
2013-11-21 13:27:11,253 - OnosScale4nodes - INFO - ONOS is up and running!
2013-11-21 13:27:11,253 - OnosScale4nodes - INFO - Assertion Passed


*****************************
 Result: Pass 
*****************************



******************************
 Result summary for Testcase2
******************************


[2013-11-21 13:27:11.258715] [OnosScale4nodes] [CASE]  Timing Onos Convergence for switch -> a single ONOS node in the cluster

[2013-11-21 13:27:11.261028] [OnosScale4nodes] [STEP]  INIT : Initializing the test case :Timing Onos Convergence for switch -> a single ONOS node in the cluster

[2013-11-21 13:27:11.265212] [OnosScale4nodes] [STEP]  2.1: Bringing ONOS down...
2013-11-21 13:27:11,267 - OnosScale4nodes - INFO - all switch no controllers
2013-11-21 13:27:23,475 - OnosScale4nodes - INFO - bringing ONOS down
2013-11-21 13:27:24,525 - OnosScale4nodes - WARNING - ONOS wasn't running
2013-11-21 13:27:25,536 - OnosScale4nodes - WARNING - ONOS wasn't running
2013-11-21 13:27:26,404 - OnosScale4nodes - WARNING - ONOS wasn't running
2013-11-21 13:27:27,181 - OnosScale4nodes - WARNING - ONOS wasn't running
2013-11-21 13:27:27,181 - OnosScale4nodes - INFO - Dropping keyspace...
2013-11-21 13:27:29,328 - OnosScale4nodes - INFO - Keyspace dropped
2013-11-21 13:27:34,334 - OnosScale4nodes - INFO - Bringing ONOS up
2013-11-21 13:27:43,754 - OnosScale4nodes - INFO - ONOS Started 
2013-11-21 13:27:58,959 - OnosScale4nodes - INFO - ONOS Started 
2013-11-21 13:28:09,372 - OnosScale4nodes - INFO - ONOS Started 
2013-11-21 13:28:19,871 - OnosScale4nodes - INFO - ONOS Started 
2013-11-21 13:28:20,150 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:28:21,083 - OnosScale4nodes - INFO - Executed :~/ONOS/start-onos.sh status  		 Expected Prompt 'running' Found
2013-11-21 13:28:21,185 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:28:21,287 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:28:51,403 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:28:51,504 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:28:51,506 - OnosScale4nodes - INFO - ONOS process is running...
2013-11-21 13:28:51,506 - OnosScale4nodes - INFO - Assertion Passed
2013-11-21 13:28:51,533 - OnosScale4nodes - INFO - Onos log is moving! It's looking good!
2013-11-21 13:28:51,702 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:28:52,441 - OnosScale4nodes - INFO - Executed :~/ONOS/start-onos.sh status  		 Expected Prompt 'running' Found
2013-11-21 13:28:52,543 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:28:52,645 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:29:22,773 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:29:22,875 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:29:22,876 - OnosScale4nodes - INFO - ONOS process is running...
2013-11-21 13:29:22,876 - OnosScale4nodes - INFO - Assertion Passed
2013-11-21 13:29:22,892 - OnosScale4nodes - INFO - Onos log is moving! It's looking good!
2013-11-21 13:29:23,079 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:29:23,708 - OnosScale4nodes - INFO - Executed :~/ONOS/start-onos.sh status  		 Expected Prompt 'running' Found
2013-11-21 13:29:23,811 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:29:23,912 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:29:54,020 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:29:54,122 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:29:54,123 - OnosScale4nodes - INFO - ONOS process is running...
2013-11-21 13:29:54,123 - OnosScale4nodes - INFO - Assertion Passed
2013-11-21 13:29:54,135 - OnosScale4nodes - INFO - Onos log is moving! It's looking good!
2013-11-21 13:29:54,377 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:29:55,029 - OnosScale4nodes - INFO - Executed :~/ONOS/start-onos.sh status  		 Expected Prompt 'running' Found
2013-11-21 13:29:55,131 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:29:55,233 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:30:25,337 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:30:25,438 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:30:25,439 - OnosScale4nodes - INFO - ONOS process is running...
2013-11-21 13:30:25,440 - OnosScale4nodes - INFO - Assertion Passed
2013-11-21 13:30:25,451 - OnosScale4nodes - INFO - Onos log is moving! It's looking good!
2013-11-21 13:30:25,452 - OnosScale4nodes - INFO - Making some rest calls...
2013-11-21 13:30:26,774 - OnosScale4nodes - INFO - 
-----
switch: network 1 : 0 switches 0 active
switch fail: network 1 should have 100 switches but has 0
switch fail: network 1 should have 100 active switches but has 0
link: total 0 links (correct : 198)
link fail
-----

2013-11-21 13:30:26,777 - OnosScale4nodes - INFO - Pointing the Switches at ONE controller... then BEGIN time
2013-11-21 13:30:38,642 - OnosScale4nodes - INFO - Executed :sh ~/ONOS/scripts/test-ctrl-local.sh  		 Expected Prompt 'mininet' Found
2013-11-21 13:30:38,643 - OnosScale4nodes - INFO - Making some rest calls...
2013-11-21 13:30:44,768 - OnosScale4nodes - INFO - 
-----
switch: network 1 : 24 switches 24 active
switch fail: network 1 should have 100 switches but has 24
switch fail: network 1 should have 100 active switches but has 24
link: total 44 links (correct : 198)
link fail
-----

2013-11-21 13:30:45,771 - OnosScale4nodes - INFO - Making some rest calls...
2013-11-21 13:30:53,806 - OnosScale4nodes - INFO - 
-----
switch: network 1 : 51 switches 51 active
switch fail: network 1 should have 100 switches but has 51
switch fail: network 1 should have 100 active switches but has 51
link: total 45 links (correct : 198)
link fail
-----

2013-11-21 13:30:54,810 - OnosScale4nodes - INFO - Making some rest calls...
2013-11-21 13:31:06,811 - OnosScale4nodes - INFO - 
-----
switch: network 1 : 99 switches 99 active
switch fail: network 1 should have 100 switches but has 99
switch fail: network 1 should have 100 active switches but has 99
link: total 198 links (correct : 198)
-----

2013-11-21 13:31:07,815 - OnosScale4nodes - INFO - Making some rest calls...
2013-11-21 13:31:17,355 - OnosScale4nodes - INFO - 
-----
switch: network 1 : 100 switches 100 active
link: total 198 links (correct : 198)
-----

2013-11-21 13:31:17,359 - OnosScale4nodes - INFO - Making some rest calls...
2013-11-21 13:31:25,792 - OnosScale4nodes - INFO - Time is less then supplied target time
2013-11-21 13:31:25,794 - OnosScale4nodes - INFO - Verifying the Expected is equal to the actual or not using assert_equal
2013-11-21 13:31:25,795 - OnosScale4nodes - INFO - None
2013-11-21 13:31:25,797 - OnosScale4nodes - INFO - Assertion Passed


*****************************
 Result: Pass 
*****************************



******************************
 Result summary for Testcase3
******************************


[2013-11-21 13:31:25.805590] [OnosScale4nodes] [CASE]  Timing Onos Convergence for switch -> all ONOS nodes in cluster

[2013-11-21 13:31:25.808631] [OnosScale4nodes] [STEP]  INIT : Initializing the test case :Timing Onos Convergence for switch -> all ONOS nodes in cluster

[2013-11-21 13:31:25.813595] [OnosScale4nodes] [STEP]  3.1: Bringing ONOS down...
2013-11-21 13:31:25,815 - OnosScale4nodes - INFO - all switch no controllers
2013-11-21 13:31:37,810 - OnosScale4nodes - INFO - bringing ONOS down
2013-11-21 13:31:38,684 - OnosScale4nodes - WARNING - ONOS wasn't running
2013-11-21 13:31:39,430 - OnosScale4nodes - WARNING - ONOS wasn't running
2013-11-21 13:31:40,185 - OnosScale4nodes - WARNING - ONOS wasn't running
2013-11-21 13:31:41,250 - OnosScale4nodes - WARNING - ONOS wasn't running
2013-11-21 13:31:46,257 - OnosScale4nodes - INFO - Bringing ONOS up
2013-11-21 13:31:55,996 - OnosScale4nodes - INFO - ONOS Started 
2013-11-21 13:32:11,247 - OnosScale4nodes - INFO - ONOS Started 
2013-11-21 13:32:13,562 - OnosScale4nodes - INFO - Executed :~/ONOS/start-rest.sh start 		 Expected Prompt '\$' Found
2013-11-21 13:32:13,565 - OnosScale4nodes - INFO - Rest Server Started Successfully
2013-11-21 13:32:28,991 - OnosScale4nodes - INFO - ONOS Started 
2013-11-21 13:32:39,496 - OnosScale4nodes - INFO - ONOS Started 
2013-11-21 13:32:39,645 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:32:40,296 - OnosScale4nodes - INFO - Executed :~/ONOS/start-onos.sh status  		 Expected Prompt 'running' Found
2013-11-21 13:32:40,399 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:32:40,501 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:33:10,633 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:33:10,735 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:33:10,736 - OnosScale4nodes - INFO - ONOS process is running...
2013-11-21 13:33:10,736 - OnosScale4nodes - INFO - Assertion Passed
2013-11-21 13:33:10,761 - OnosScale4nodes - INFO - Onos log is moving! It's looking good!
2013-11-21 13:33:11,061 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:33:11,714 - OnosScale4nodes - INFO - Executed :~/ONOS/start-onos.sh status  		 Expected Prompt 'running' Found
2013-11-21 13:33:11,817 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:33:11,920 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:33:42,053 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:33:42,155 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:33:42,157 - OnosScale4nodes - INFO - ONOS process is running...
2013-11-21 13:33:42,157 - OnosScale4nodes - INFO - Assertion Passed
2013-11-21 13:33:42,174 - OnosScale4nodes - INFO - Onos log is moving! It's looking good!
2013-11-21 13:33:42,572 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:33:43,307 - OnosScale4nodes - INFO - Executed :~/ONOS/start-onos.sh status  		 Expected Prompt 'running' Found
2013-11-21 13:33:43,408 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:33:43,510 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:34:13,615 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:34:13,717 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:34:13,718 - OnosScale4nodes - INFO - ONOS process is running...
2013-11-21 13:34:13,719 - OnosScale4nodes - INFO - Assertion Passed
2013-11-21 13:34:13,735 - OnosScale4nodes - INFO - Onos log is moving! It's looking good!
2013-11-21 13:34:13,985 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:34:14,678 - OnosScale4nodes - INFO - Executed :~/ONOS/start-onos.sh status  		 Expected Prompt 'running' Found
2013-11-21 13:34:14,780 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:34:14,882 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:34:45,014 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:34:45,116 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:34:45,117 - OnosScale4nodes - INFO - ONOS process is running...
2013-11-21 13:34:45,117 - OnosScale4nodes - INFO - Assertion Passed
2013-11-21 13:34:45,128 - OnosScale4nodes - INFO - Onos log is moving! It's looking good!
2013-11-21 13:34:45,128 - OnosScale4nodes - INFO - Making some rest calls...
2013-11-21 13:34:52,755 - OnosScale4nodes - INFO - 
-----
switch: network 1 : 100 switches 0 active
switch fail: network 1 should have 100 active switches but has 0
link: total 0 links (correct : 198)
link fail
-----

2013-11-21 13:34:52,759 - OnosScale4nodes - INFO - Pointing the Switches at ALL controllers... then BEGIN time
2013-11-21 13:35:06,330 - OnosScale4nodes - INFO - Executed :sh ~/ONOS/scripts/test-ctrl-add-ext.sh 		 Expected Prompt 'mininet' Found
2013-11-21 13:35:06,333 - OnosScale4nodes - INFO - Making some rest calls...
2013-11-21 13:35:14,978 - OnosScale4nodes - INFO - 
-----
switch: network 1 : 100 switches 99 active
switch fail: network 1 should have 100 active switches but has 99
link: total 198 links (correct : 198)
-----

2013-11-21 13:35:15,981 - OnosScale4nodes - INFO - Making some rest calls...
2013-11-21 13:35:24,693 - OnosScale4nodes - INFO - 
-----
switch: network 1 : 100 switches 100 active
link: total 198 links (correct : 198)
-----

2013-11-21 13:35:24,697 - OnosScale4nodes - INFO - Making some rest calls...
2013-11-21 13:35:32,340 - OnosScale4nodes - INFO - Time is less then supplied target time
2013-11-21 13:35:32,342 - OnosScale4nodes - INFO - Verifying the Expected is equal to the actual or not using assert_equal
2013-11-21 13:35:32,342 - OnosScale4nodes - INFO - None
2013-11-21 13:35:32,343 - OnosScale4nodes - INFO - Assertion Passed


*****************************
 Result: Pass 
*****************************



******************************
 Result summary for Testcase4
******************************


[2013-11-21 13:35:32.349674] [OnosScale4nodes] [CASE]  Timing Onos Convergence for even single controller distribution

[2013-11-21 13:35:32.352154] [OnosScale4nodes] [STEP]  INIT : Initializing the test case :Timing Onos Convergence for even single controller distribution

[2013-11-21 13:35:32.357204] [OnosScale4nodes] [STEP]  4.1: Bringing ONOS down...
2013-11-21 13:35:32,359 - OnosScale4nodes - INFO - all switch no controllers
2013-11-21 13:35:46,036 - OnosScale4nodes - INFO - bringing ONOS down
2013-11-21 13:35:47,002 - OnosScale4nodes - WARNING - ONOS wasn't running
2013-11-21 13:35:47,798 - OnosScale4nodes - WARNING - ONOS wasn't running
2013-11-21 13:35:48,838 - OnosScale4nodes - WARNING - ONOS wasn't running
2013-11-21 13:35:49,640 - OnosScale4nodes - WARNING - ONOS wasn't running
2013-11-21 13:35:54,646 - OnosScale4nodes - INFO - Bringing ONOS up
2013-11-21 13:36:04,360 - OnosScale4nodes - INFO - ONOS Started 
2013-11-21 13:36:19,065 - OnosScale4nodes - INFO - ONOS Started 
2013-11-21 13:36:21,376 - OnosScale4nodes - INFO - Executed :~/ONOS/start-rest.sh start 		 Expected Prompt '\$' Found
2013-11-21 13:36:21,377 - OnosScale4nodes - INFO - Rest Server Started Successfully
2013-11-21 13:36:36,672 - OnosScale4nodes - INFO - ONOS Started 
2013-11-21 13:36:47,590 - OnosScale4nodes - INFO - ONOS Started 
2013-11-21 13:36:47,765 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:36:48,544 - OnosScale4nodes - INFO - Executed :~/ONOS/start-onos.sh status  		 Expected Prompt 'running' Found
2013-11-21 13:36:48,645 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:36:48,747 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:37:18,872 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:37:18,975 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:37:18,976 - OnosScale4nodes - INFO - ONOS process is running...
2013-11-21 13:37:18,976 - OnosScale4nodes - INFO - Assertion Passed
2013-11-21 13:37:19,001 - OnosScale4nodes - INFO - Onos log is moving! It's looking good!
2013-11-21 13:37:19,234 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:37:19,875 - OnosScale4nodes - INFO - Executed :~/ONOS/start-onos.sh status  		 Expected Prompt 'running' Found
2013-11-21 13:37:19,977 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:37:20,079 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:37:50,200 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:37:50,302 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:37:50,304 - OnosScale4nodes - INFO - ONOS process is running...
2013-11-21 13:37:50,304 - OnosScale4nodes - INFO - Assertion Passed
2013-11-21 13:37:50,321 - OnosScale4nodes - INFO - Onos log is moving! It's looking good!
2013-11-21 13:37:50,476 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:37:51,182 - OnosScale4nodes - INFO - Executed :~/ONOS/start-onos.sh status  		 Expected Prompt 'running' Found
2013-11-21 13:37:51,284 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:37:51,386 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:38:21,506 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:38:21,608 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:38:21,610 - OnosScale4nodes - INFO - ONOS process is running...
2013-11-21 13:38:21,610 - OnosScale4nodes - INFO - Assertion Passed
2013-11-21 13:38:21,622 - OnosScale4nodes - INFO - Onos log is moving! It's looking good!
2013-11-21 13:38:21,817 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:38:22,469 - OnosScale4nodes - INFO - Executed :~/ONOS/start-onos.sh status  		 Expected Prompt 'running' Found
2013-11-21 13:38:22,571 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:38:22,673 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:38:52,790 - OnosScale4nodes - INFO - Executed :
 		 Expected Prompt '\$' Found
2013-11-21 13:38:52,893 - OnosScale4nodes - INFO - Executed :tail ~/ONOS/onos-logs/onos.*.log 		 Expected Prompt '\$' Found
2013-11-21 13:38:52,894 - OnosScale4nodes - INFO - ONOS process is running...
2013-11-21 13:38:52,894 - OnosScale4nodes - INFO - Assertion Passed
2013-11-21 13:38:52,906 - OnosScale4nodes - INFO - Onos log is moving! It's looking good!
2013-11-21 13:38:52,906 - OnosScale4nodes - INFO - Making some rest calls...
2013-11-21 13:39:00,163 - OnosScale4nodes - INFO - 
-----
switch: network 1 : 100 switches 0 active
switch fail: network 1 should have 100 active switches but has 0
link: total 0 links (correct : 198)
link fail
-----

2013-11-21 13:39:00,165 - OnosScale4nodes - INFO - Pointing the Switches to alternating controllers... then BEGIN time
2013-11-21 13:39:12,655 - OnosScale4nodes - INFO - Executed :sh ~/ONOS/scripts/ctrl-divide.sh  		 Expected Prompt 'mininet' Found
2013-11-21 13:39:12,657 - OnosScale4nodes - INFO - Making some rest calls...
2013-11-21 13:39:20,814 - OnosScale4nodes - INFO - 
-----
switch: network 1 : 100 switches 98 active
switch fail: network 1 should have 100 active switches but has 98
link: total 191 links (correct : 198)
link fail
-----

2013-11-21 13:39:21,819 - OnosScale4nodes - INFO - Making some rest calls...
2013-11-21 13:39:29,109 - OnosScale4nodes - INFO - 
-----
switch: network 1 : 100 switches 100 active
link: total 198 links (correct : 198)
-----

2013-11-21 13:39:29,111 - OnosScale4nodes - INFO - Making some rest calls...
2013-11-21 13:39:36,040 - OnosScale4nodes - INFO - Time is less then supplied target time
2013-11-21 13:39:36,043 - OnosScale4nodes - INFO - Verifying the Expected is equal to the actual or not using assert_equal
2013-11-21 13:39:36,044 - OnosScale4nodes - INFO - None
2013-11-21 13:39:36,045 - OnosScale4nodes - INFO - Assertion Passed


*****************************
 Result: Pass 
*****************************




*************************************
	Test Execution Summary

************************************* 

 Test Start           : 21 Nov 2013 13:20:21
 Test End             : 21 Nov 2013 13:39:36
 Execution Time       : 0:19:14.634494
 Total tests planned  : 4
 Total tests RUN      : 4
 Total Pass           : 4
 Total Fail           : 0
 Total No Result      : 0
 Success Percentage   : 100%
 Execution Result     : 100%
2013-11-21 13:39:37,936 - OnosScale4nodes - INFO - Disconnecting mininet...
2013-11-21 13:39:38,039 - OnosScale4nodes - INFO - Executed :exit 		 Expected Prompt '(.*)' Found
2013-11-21 13:39:38,141 - OnosScale4nodes - INFO - Executed :exit 		 Expected Prompt '(.*)' Found