blob: e8e10afb61b65207c4ecdb6631f37288d6b5b35f [file] [log] [blame]
*** Settings ***
Documentation ONOS Sanity Test Suite
Suite Setup ONOS Suite Setup ${CONTROLLER_IP} ${CONTROLLER_USER} ##Things to be done prior to running tests
Suite Teardown ONOS Suite Teardown ##Things to be done after test suite is finished.
Library SSHLibrary
Library Collections
Library OperatingSystem
Library String
Library RequestsLibrary
Library HttpLibrary.HTTP
*** Variables ***
##Grab the environment variables sources from your "cell"
${CONTROLLER_IP} %{OC1}
${MININET_IP} %{OCN}
${CONTROLLER_USER} %{ONOS_USER}
${MININET_USER} %{ONOS_USER}
##USER_HOME used for public key
${USER_HOME} /home/fedora
##ONOS_HOME is where the onos dist will be deployed on the controller vm
${ONOS_HOME} /opt/onos
${RESTCONFPORT} 8181
${LINUX_PROMPT} $
##SWITCHES_RESULT_FILE and JENKINS_WORKSPACE can be configurable...read overriding variables in README
${SWITCHES_RESULT_FILE} ${USER_HOME}/workspace/tools/switches.csv
${JENKINS_WORKSPACE} ${USER_HOME}/workspace/ONOS-Stable/
${prompt_timeout} 30s
${start} 10
${end} 100
${increments} 10
##Number of nodes in cluster. To add more nodes, create CONTROLLER_IP2/3/4 etc. variables above and change this cluster variable
${cluster} 1
*** Test Cases ***
Find Max Switches By Scaling
[Documentation] Find the max number of switches from ${start} until reaching ${end} in steps of ${increments}. The following checks are made through REST APIs:
... -\ Verify device count is correct
... -\ Verify device status is available
... -\ Verify device roles are MASTER (default role in case of standalone controller)
... -\ Verify topology recognizes corret number of devices (Through api "/topology")
... -\ Observe each device individually
... -\ Observe links, hosts, and flows through the controller
... -\ Observe device info at lower level on mininet (Written for PoC). Shows flows, links, and ports. Checks can be easily implemented at that level as well
... -\ STOP Mininet Topo
... -\ Verify device count is zero
... -\ Verify topology sees no devices (Through api "/topology")
[Tags] done
Append To File ${SWITCHES_RESULT_FILE} Max Switches Linear Topo\n
${max}= Find Max Switches ${start} ${end} ${increments}
Log ${max}
Append To File ${SWITCHES_RESULT_FILE} ${max}\n
*** Keywords ***
ONOS Suite Setup
[Arguments] ${controller} ${user}
[Documentation] Transfers the ONOS dist over to the test vm and start the controller. We will leverage the bash script, "onos-install" to do this.
Create Controller IP List
${rc}= Run and Return RC onos-package
Should Be Equal As Integers ${rc} 0
: FOR ${ip} IN @{controller_list}
\ ${rc}= Run and Return RC onos-install -f ${ip}
\ Should Be Equal As Integers ${rc} 0
Create HTTP Sessions
Wait Until Keyword Succeeds 60s 2s Verify All Controller are up
#If creating a cluster, create a keyword and call it here
ONOS Suite Teardown
[Documentation] Stop ONOS on Controller VMs and grabs karaf logs on each controller (put them in /tmp)
${rc}= Run and Return RC onos-kill
#Should Be Equal As Integers ${rc} 0
${rc}= Run and Return RC cp ${SWITCHES_RESULT_FILE} ${JENKINS_WORKSPACE}
Should Be Equal As Integers ${rc} 0
${rc}= Run and Return RC rm ${SWITCHES_RESULT_FILE}
Should Be Equal As Integers ${rc} 0
Clean Mininet System
: FOR ${ip} IN @{controller_list}
\ Get Karaf Logs ${ip}
Create Controller IP List
[Documentation] Creates a list of controller ips for a cluster. When creating a cluster, be sure to set each variable to %{OC} env vars in the variables section
@{controller_list}= Create List ${CONTROLLER_IP}
Set Suite Variable @{controller_list}
Create HTTP Sessions
[Documentation] Creates an http session with all controllers in the cluster. Session names are set to respective ips.
${HEADERS}= Create Dictionary Content-Type application/json
: FOR ${ip} IN @{controller_list}
\ Create Session ${ip} http://${ip}:${RESTCONFPORT} headers=${HEADERS}
Find Max Switches
[Arguments] ${start} ${stop} ${step}
[Documentation] Will find out max switches starting from ${start} till reaching ${stop} and in steps defined by ${step}
${max-switches} Set Variable ${0}
${start} Convert to Integer ${start}
${stop} Convert to Integer ${stop}
${step} Convert to Integer ${step}
: FOR ${switches} IN RANGE ${start} ${stop+1} ${step}
\ Start Mininet Linear ${switches}
\ ${status} ${result} Run Keyword And Ignore Error Verify Controllers are Not Dead
\ Exit For Loop If '${status}' == 'FAIL'
\ ${status} ${result} Run Keyword And Ignore Error Wait Until Keyword Succeeds ${switches*2} 2s
\ ... Ensure Switch Count ${switches}
\ Exit For Loop If '${status}' == 'FAIL'
\ ${status} ${result} Run Keyword And Ignore Error Wait Until Keyword Succeeds ${switches*2} 2s
\ ... Ensure Switches are Available
\ Exit For Loop If '${status}' == 'FAIL'
\ ${status} ${result} Run Keyword And Ignore Error Wait Until Keyword Succeeds ${switches*2} 2s
\ ... Ensure Switch Role MASTER
\ Exit For Loop If '${status}' == 'FAIL'
\ ${status} ${result} Run Keyword And Ignore Error Wait Until Keyword Succeeds ${switches*2} 2s
\ ... Ensure Topology ${switches} ${cluster}
\ Exit For Loop If '${status}' == 'FAIL'
\ ${status} ${result} Run Keyword And Ignore Error Wait Until Keyword Succeeds ${switches*2} 2s
\ ... Experiment Links, Hosts, and Flows
\ ${status} ${result} Run Keyword And Ignore Error Wait Until Keyword Succeeds ${switches*2} 2s
\ ... Check Each Switch Individually
\ ${status} ${result} Run Keyword And Ignore Error Wait Until Keyword Succeeds ${switches*2} 2s
\ ... Check Mininet at Lower Level
\ Stop Mininet
\ ${status} ${result} Run Keyword And Ignore Error Wait Until Keyword Succeeds ${switches*2} 2s
\ ... Ensure No Switches are Available
\ Exit For Loop If '${status}' == 'FAIL'
\ ${status} ${result} Run Keyword And Ignore Error Wait Until Keyword Succeeds ${switches*2} 2s
\ ... Ensure No Switches in Topology ${cluster}
\ Exit For Loop If '${status}' == 'FAIL'
\ ${max-switches} Convert To String ${switches}
[Return] ${max-switches}
Run Command On Remote System
[Arguments] ${remote_system} ${cmd} ${user}=${CONTROLLER_USER} ${prompt}=${LINUX_PROMPT} ${prompt_timeout}=30s
[Documentation] Reduces the common work of running a command on a remote system to a single higher level robot keyword,
... taking care to log in with a public key and the command given is written and the output returned. No test conditions
... are checked.
Log Attempting to execute ${cmd} on ${remote_system}
${conn_id}= SSHLibrary.Open Connection ${remote_system} prompt=${prompt} timeout=${prompt_timeout}
Login With Public Key ${user} ${USER_HOME}/.ssh/id_rsa any
SSHLibrary.Write ${cmd}
${output}= SSHLibrary.Read Until ${LINUX_PROMPT}
SSHLibrary.Close Connection
Log ${output}
[Return] ${output}
Start Mininet Linear
[Arguments] ${switches}
[Documentation] Start mininet linear topology with ${switches} nodes
Log To Console \n
Log To Console Starting mininet linear ${switches}
${mininet_conn_id}= Open Connection ${MININET_IP} prompt=${LINUX_PROMPT} timeout=${switches*3}
Set Suite Variable ${mininet_conn_id}
Login With Public Key ${MININET_USER} ${USER_HOME}/.ssh/id_rsa any
Write sudo mn --controller=remote,ip=${CONTROLLER_IP} --topo linear,${switches} --switch ovsk,protocols=OpenFlow13
Read Until mininet>
Sleep 6
Stop Mininet
[Documentation] Stop mininet topology
Log To Console Stopping Mininet
Switch Connection ${mininet_conn_id}
Read
Write exit
Read Until ${LINUX_PROMPT}
Close Connection
Check Mininet at Lower Level
[Documentation] PoC for executing mininet commands at the lower level
Switch Connection ${mininet_conn_id}
Read
Write dpctl dump-flows -O OpenFlow13
${output}= Read Until mininet>
Log ${output}
Write dump
${output}= Read Until mininet>
Log ${output}
Write links
${output}= Read Until mininet>
Log ${output}
Write ports
${output}= Read Until mininet>
Log ${output}
Clean Mininet System
[Arguments] ${mininet_system}=${MININET_IP}
[Documentation] Cleans the mininet environment (sudo mn -c)
Run Command On Remote System ${mininet_system} sudo mn -c ${CONTROLLER_USER} ${LINUX_PROMPT} 600s
Verify All Controller are up
[Documentation] Verifies each controller is up by issuing a rest call and getting a 200
: FOR ${ip} IN @{controller_list}
\ ${resp}= ONOS Get ${ip} devices
\ Should Be Equal As Strings ${resp.status_code} 200
Verify Controllers are Not Dead
[Documentation] Verifies each controller is not dead by making sure karaf log does not contain "OutOfMemoryError" and rest call still returns 200
: FOR ${ip} IN @{controller_list}
\ Verify Controller Is Not Dead ${ip}
Verify Controller Is Not Dead
[Arguments] ${controller}
${response}= Run Command On Remote System ${controller} grep java.lang.OutOfMemoryError /opt/onos/log/karaf.log
Should Not Contain ${response} OutOfMemoryError
${resp} RequestsLibrary.Get ${controller} /onos/v1/devices
Log ${resp.content}
Should Be Equal As Strings ${resp.status_code} 200
Experiment Links, Hosts, and Flows
[Documentation] Currently this only returns the information the controller has on links, hosts, and flows. Checks can easily be implemented
: FOR ${ip} IN @{controller_list}
\ ${resp}= ONOS Get ${ip} links
\ ${jsondata} To JSON ${resp.content}
\ ${length}= Get Length ${jsondata['links']}
\ Log ${resp.content}
\ ${resp}= ONOS Get ${ip} flows
\ ${jsondata} To JSON ${resp.content}
\ ${length}= Get Length ${jsondata['flows']}
\ Log ${resp.content}
\ ${resp}= ONOS Get ${ip} hosts
\ ${jsondata} To JSON ${resp.content}
\ ${length}= Get Length ${jsondata['hosts']}
\ Log ${resp.content}
Ensure Topology
[Arguments] ${device_count} ${cluster_count}
[Documentation] Verifies the devices count through /topoplogy api. Currently, cluster count is inconsistent (Possible bug, will look into it), so the check on that is ignored, but logged
: FOR ${ip} IN @{controller_list}
\ ${resp}= ONOS Get ${ip} topology
\ Log ${resp.content}
\ ${devices}= Get Json Value ${resp.content} /devices
\ ${clusters}= Get Json Value ${resp.content} /clusters
\ Should Be Equal As Strings ${devices} ${device_count}
\ #Should Be Equal As Strings ${clusters} ${cluster_count}
Ensure No Switches in Topology
[Arguments] ${cluster_count}
[Documentation] Verifies the topology sees no devices
: FOR ${ip} IN @{controller_list}
\ ${resp}= ONOS Get ${ip} topology
\ Log ${resp.content}
\ ${devices}= Get Json Value ${resp.content} /devices
\ ${clusters}= Get Json Value ${resp.content} /clusters
\ Should Be Equal As Strings ${devices} 0
\ #Should Be Equal As Strings ${clusters} ${cluster_count}
ONOS Get
[Arguments] ${session} ${noun}
[Documentation] Common keyword to issue GET requests to the controller. Arguments are the session (controller ip) and api path
${resp} RequestsLibrary.Get ${session} /onos/v1/${noun}
Log ${resp.content}
[Return] ${resp}
Ensure Switch Count
[Arguments] ${switch_count}
[Documentation] Verfies the device count (passed in as arg) on each controller is accurate.
: FOR ${ip} IN @{controller_list}
\ ${resp} ONOS Get ${ip} devices
\ Log ${resp.content}
\ Should Be Equal As Strings ${resp.status_code} 200
\ Should Not Be Empty ${resp.content}
\ ${jsondata} To JSON ${resp.content}
\ ${length}= Get Length ${jsondata['devices']}
\ Should Be Equal As Integers ${length} ${switch_count}
Ensure Switches are Available
[Documentation] Verifies that the switch's availability state is true on all switches through all controllers
: FOR ${ip} IN @{controller_list}
\ ${resp} ONOS Get ${ip} devices
\ Log ${resp.content}
\ Should Be Equal As Strings ${resp.status_code} 200
\ Should Not Be Empty ${resp.content}
\ ${jsondata} To JSON ${resp.content}
\ #Robot tweak to do a nested for loop
\ Check Each Switch Status ${jsondata} True
Check Each Switch Status
[Arguments] ${jdata} ${bool}
${length}= Get Length ${jdata['devices']}
: FOR ${INDEX} IN RANGE 0 ${length}
\ ${data}= Get From List ${jdata['devices']} ${INDEX}
\ ${status}= Get From Dictionary ${data} available
\ Should Be Equal As Strings ${status} ${bool}
Ensure No Switches are Available
[Documentation] Verifies that the switch's availability state is false on all switches through all controllers
: FOR ${ip} IN @{controller_list}
\ ${resp} ONOS Get ${ip} devices
\ Log ${resp.content}
\ Should Be Equal As Strings ${resp.status_code} 200
\ Should Not Be Empty ${resp.content}
\ ${jsondata} To JSON ${resp.content}
\ Check Each Switch Status ${jsondata} False
Check Each Switch Individually
[Documentation] Currently just observe each information the device has. Checks can easily be implemented
: FOR ${ip} IN @{controller_list}
\ ${resp} ONOS Get ${ip} devices
\ Should Be Equal As Strings ${resp.status_code} 200
\ ${jsondata} To JSON ${resp.content}
\ #Robot tweak to do a nested for loop
\ Check Each Switch ${jsondata}
Check Each Switch
[Arguments] ${jdata}
${length}= Get Length ${jdata['devices']}
@{dpid_list}= Create List
: FOR ${INDEX} IN RANGE 0 ${length}
\ ${devicedata}= Get From List ${jdata['devices']} ${INDEX}
\ ${id}= Get From Dictionary ${devicedata} id
\ Append To List ${dpid_list} ${id}
\ Log ${dpid_list}
${length}= Get Length ${dpid_list}
: FOR ${i} IN @{dpid_list}
\ ${resp} ONOS Get ${ip} devices/${i}
\ Log ${resp.content}
Ensure Switch Role
[Arguments] ${role}
[Documentation] Verifies that the controller's role for each switch is MASTER (default in standalone mode)
: FOR ${ip} IN @{controller_list}
\ ${resp} ONOS Get ${ip} devices
\ Log ${resp.content}
\ Should Be Equal As Strings ${resp.status_code} 200
\ Should Not Be Empty ${resp.content}
\ ${jsondata} To JSON ${resp.content}
\ Ensure Role ${jsondata} ${role}
Ensure Role
[Arguments] ${jdata} ${role}
${length}= Get Length ${jdata['devices']}
: FOR ${INDEX} IN RANGE 0 ${length}
\ ${data}= Get From List ${jdata['devices']} ${INDEX}
\ ${status}= Get From Dictionary ${data} role
\ Should Be Equal As Strings ${status} ${role}
Get Karaf Logs
[Arguments] ${controller}
[Documentation] Compresses all the karaf log files on each controler and puts them on your pybot execution machine (in /tmp)
Run Command On Remote System ${controller} tar -zcvf /tmp/${SUITE NAME}.${controller}.tar.gz ${ONOS_HOME}/log
SSHLibrary.Open Connection ${controller} prompt=${LINUX_PROMPT} timeout=${prompt_timeout}
Login With Public Key ${CONTROLLER_USER} ${USER_HOME}/.ssh/id_rsa any
SSHLibrary.Get File /tmp/${SUITE NAME}.${controller}.tar.gz /tmp/
SSHLibrary.Close Connection