Gitiles
Code Review Sign In
gerrit.onosproject.org / OnosSystemTest / 80b8218959ed7f243af6fe9c088744b4794473a3 / . / TestON / RobotTests / ScalePOC.robot
blob: 9ce62a7c886dc06f81f8c398dc68872cfadd248b [file] [log] [blame]
Kailash Khalasi80b82182015-06-26 22:58:40 +0000[diff] [blame^]1*** Settings ***
2Documentation ONOS Sanity Test Suite
3Suite Setup ONOS Suite Setup ${CONTROLLER_IP} ${CONTROLLER_USER}
4Suite Teardown ONOS Suite Teardown
5Library SSHLibrary
6Library Collections
7Library OperatingSystem
8Library String
9Library RequestsLibrary
10Library HttpLibrary.HTTP
11
12*** Variables ***
13${CONTROLLER_IP} %{OC1}
14${MININET_IP} %{OCN}
15${CONTROLLER_USER} %{ONOS_USER}
16${MININET_USER} %{ONOS_USER}
17${USER_HOME} /home/fedora
18${ONOS_HOME} /opt/onos
19${RESTCONFPORT} 8181
20${LINUX_PROMPT} $
21${SWITCHES_RESULT_FILE} ${USER_HOME}/workspace/tools/switches.csv
22${JENKINS_WORKSPACE} ${USER_HOME}/workspace/ONOS-Stable/
23${prompt_timeout} 30s
24${start} 10
25${end} 100
26${increments} 10
27${cluster} 1
28
29*** Test Cases ***
30Find Max Switches By Scaling
31 [Documentation] Find the max number of switches from ${start} until reaching ${end} in steps of ${increments}. The following checks are made through REST APIs:
32 ... -\ Verify device count is correct
33 ... -\ Verify device status is available
34 ... -\ Verify device roles are MASTER (default role in case of standalone controller)
35 ... -\ Verify topology recognizes corret number of devices (Through api "/topology")
36 ... -\ Observe each device individually
37 ... -\ Observe links, hosts, and flows through the controller
38 ... -\ Observe device info at lower level on mininet (Written for PoC). Shows flows, links, and ports. Checks can be easily implemented at that level as well
39 ... -\ STOP Mininet Topo
40 ... -\ Verify device count is zero
41 ... -\ Verify topology sees no devices (Through api "/topology")
42 [Tags] done
43 Append To File ${SWITCHES_RESULT_FILE} Max Switches Linear Topo\n
44 ${max}= Find Max Switches ${start} ${end} ${increments}
45 Log ${max}
46 Append To File ${SWITCHES_RESULT_FILE} ${max}\n
47
48*** Keywords ***
49ONOS Suite Setup
50 [Arguments] ${controller} ${user}
51 [Documentation] Transfers the ONOS dist over to the test vm and start the controller. We will leverage the bash script, "onos-install" to do this.
52 Create Controller IP List
53 ${rc}= Run and Return RC onos-package
54 Should Be Equal As Integers ${rc} 0
55 : FOR ${ip} IN @{controller_list}
56 \ ${rc}= Run and Return RC onos-install -f ${ip}
57 \ Should Be Equal As Integers ${rc} 0
58 Create HTTP Sessions
59 Wait Until Keyword Succeeds 60s 2s Verify All Controller are up
60 #If creating a cluster, create a keyword and call it here
61
62ONOS Suite Teardown
63 [Documentation] Stop ONOS on Controller VMs and grabs karaf logs on each controller (put them in /tmp)
64 ${rc}= Run and Return RC onos-kill
65 #Should Be Equal As Integers ${rc} 0
66 ${rc}= Run and Return RC cp ${SWITCHES_RESULT_FILE} ${JENKINS_WORKSPACE}
67 Should Be Equal As Integers ${rc} 0
68 ${rc}= Run and Return RC rm ${SWITCHES_RESULT_FILE}
69 Should Be Equal As Integers ${rc} 0
70 Clean Mininet System
71 : FOR ${ip} IN @{controller_list}
72 \ Get Karaf Logs ${ip}
73
74Create Controller IP List
75 [Documentation] Creates a list of controller ips for a cluster. When creating a cluster, be sure to set each variable to %{OC} env vars in the variables section
76 @{controller_list}= Create List ${CONTROLLER_IP}
77 Set Suite Variable @{controller_list}
78
79Create HTTP Sessions
80 [Documentation] Creates an http session with all controllers in the cluster. Session names are set to respective ips.
81 ${HEADERS}= Create Dictionary Content-Type application/json
82 : FOR ${ip} IN @{controller_list}
83 \ Create Session ${ip} http://${ip}:${RESTCONFPORT} headers=${HEADERS}
84
85Find Max Switches
86 [Arguments] ${start} ${stop} ${step}
87 [Documentation] Will find out max switches starting from ${start} till reaching ${stop} and in steps defined by ${step}
88 ${max-switches} Set Variable ${0}
89 ${start} Convert to Integer ${start}
90 ${stop} Convert to Integer ${stop}
91 ${step} Convert to Integer ${step}
92 : FOR ${switches} IN RANGE ${start} ${stop+1} ${step}
93 \ Start Mininet Linear ${switches}
94 \ ${status} ${result} Run Keyword And Ignore Error Verify Controllers are Not Dead
95 \ Exit For Loop If '${status}' == 'FAIL'
96 \ ${status} ${result} Run Keyword And Ignore Error Wait Until Keyword Succeeds ${switches*2} 2s
97 \ ... Ensure Switch Count ${switches}
98 \ Exit For Loop If '${status}' == 'FAIL'
99 \ ${status} ${result} Run Keyword And Ignore Error Wait Until Keyword Succeeds ${switches*2} 2s
100 \ ... Ensure Switches are Available
101 \ Exit For Loop If '${status}' == 'FAIL'
102 \ ${status} ${result} Run Keyword And Ignore Error Wait Until Keyword Succeeds ${switches*2} 2s
103 \ ... Ensure Switch Role MASTER
104 \ Exit For Loop If '${status}' == 'FAIL'
105 \ ${status} ${result} Run Keyword And Ignore Error Wait Until Keyword Succeeds ${switches*2} 2s
106 \ ... Ensure Topology ${switches} ${cluster}
107 \ Exit For Loop If '${status}' == 'FAIL'
108 \ ${status} ${result} Run Keyword And Ignore Error Wait Until Keyword Succeeds ${switches*2} 2s
109 \ ... Experiment Links, Hosts, and Flows
110 \ ${status} ${result} Run Keyword And Ignore Error Wait Until Keyword Succeeds ${switches*2} 2s
111 \ ... Check Each Switch Individually
112 \ ${status} ${result} Run Keyword And Ignore Error Wait Until Keyword Succeeds ${switches*2} 2s
113 \ ... Check Mininet at Lower Level
114 \ Stop Mininet
115 \ ${status} ${result} Run Keyword And Ignore Error Wait Until Keyword Succeeds ${switches*2} 2s
116 \ ... Ensure No Switches are Available
117 \ Exit For Loop If '${status}' == 'FAIL'
118 \ ${status} ${result} Run Keyword And Ignore Error Wait Until Keyword Succeeds ${switches*2} 2s
119 \ ... Ensure No Switches in Topology ${cluster}
120 \ Exit For Loop If '${status}' == 'FAIL'
121 \ ${max-switches} Convert To String ${switches}
122 [Return] ${max-switches}
123
124Run Command On Remote System
125 [Arguments] ${remote_system} ${cmd} ${user}=${CONTROLLER_USER} ${prompt}=${LINUX_PROMPT} ${prompt_timeout}=30s
126 [Documentation] Reduces the common work of running a command on a remote system to a single higher level robot keyword,
127 ... taking care to log in with a public key and the command given is written and the output returned. No test conditions
128 ... are checked.
129 Log Attempting to execute ${cmd} on ${remote_system}
130 ${conn_id}= SSHLibrary.Open Connection ${remote_system} prompt=${prompt} timeout=${prompt_timeout}
131 Login With Public Key ${user} ${USER_HOME}/.ssh/id_rsa any
132 SSHLibrary.Write ${cmd}
133 ${output}= SSHLibrary.Read Until ${LINUX_PROMPT}
134 SSHLibrary.Close Connection
135 Log ${output}
136 [Return] ${output}
137
138Start Mininet Linear
139 [Arguments] ${switches}
140 [Documentation] Start mininet linear topology with ${switches} nodes
141 Log To Console \n
142 Log To Console Starting mininet linear ${switches}
143 ${mininet_conn_id}= Open Connection ${MININET_IP} prompt=${LINUX_PROMPT} timeout=${switches*3}
144 Set Suite Variable ${mininet_conn_id}
145 Login With Public Key ${MININET_USER} ${USER_HOME}/.ssh/id_rsa any
146 Write sudo mn --controller=remote,ip=${CONTROLLER_IP} --topo linear,${switches} --switch ovsk,protocols=OpenFlow13
147 Read Until mininet>
148 Sleep 6
149
150Stop Mininet
151 [Documentation] Stop mininet topology
152 Log To Console Stopping Mininet
153 Switch Connection ${mininet_conn_id}
154 Read
155 Write exit
156 Read Until ${LINUX_PROMPT}
157 Close Connection
158
159Check Mininet at Lower Level
160 [Documentation] PoC for executing mininet commands at the lower level
161 Switch Connection ${mininet_conn_id}
162 Read
163 Write dpctl dump-flows -O OpenFlow13
164 ${output}= Read Until mininet>
165 Log ${output}
166 Write dump
167 ${output}= Read Until mininet>
168 Log ${output}
169 Write links
170 ${output}= Read Until mininet>
171 Log ${output}
172 Write ports
173 ${output}= Read Until mininet>
174 Log ${output}
175
176Clean Mininet System
177 [Arguments] ${mininet_system}=${MININET_IP}
178 [Documentation] Cleans the mininet environment (sudo mn -c)
179 Run Command On Remote System ${mininet_system} sudo mn -c ${CONTROLLER_USER} ${LINUX_PROMPT} 600s
180
181Verify All Controller are up
182 [Documentation] Verifies each controller is up by issuing a rest call and getting a 200
183 : FOR ${ip} IN @{controller_list}
184 \ ${resp}= ONOS Get ${ip} devices
185 \ Should Be Equal As Strings ${resp.status_code} 200
186
187Verify Controllers are Not Dead
188 [Documentation] Verifies each controller is not dead by making sure karaf log does not contain "OutOfMemoryError" and rest call still returns 200
189 : FOR ${ip} IN @{controller_list}
190 \ Verify Controller Is Not Dead ${ip}
191
192Verify Controller Is Not Dead
193 [Arguments] ${controller}
194 ${response}= Run Command On Remote System ${controller} grep java.lang.OutOfMemoryError /opt/onos/log/karaf.log
195 Should Not Contain ${response} OutOfMemoryError
196 ${resp} RequestsLibrary.Get ${controller} /onos/v1/devices
197 Log ${resp.content}
198 Should Be Equal As Strings ${resp.status_code} 200
199
200Experiment Links, Hosts, and Flows
201 [Documentation] Currently this only returns the information the controller has on links, hosts, and flows. Checks can easily be implemented
202 : FOR ${ip} IN @{controller_list}
203 \ ${resp}= ONOS Get ${ip} links
204 \ ${jsondata} To JSON ${resp.content}
205 \ ${length}= Get Length ${jsondata['links']}
206 \ Log ${resp.content}
207 \ ${resp}= ONOS Get ${ip} flows
208 \ ${jsondata} To JSON ${resp.content}
209 \ ${length}= Get Length ${jsondata['flows']}
210 \ Log ${resp.content}
211 \ ${resp}= ONOS Get ${ip} hosts
212 \ ${jsondata} To JSON ${resp.content}
213 \ ${length}= Get Length ${jsondata['hosts']}
214 \ Log ${resp.content}
215
216Ensure Topology
217 [Arguments] ${device_count} ${cluster_count}
218 [Documentation] Verifies the devices count through /topoplogy api. Currently, cluster count is inconsistent (Possible bug, will look into it), so the check on that is ignored, but logged
219 : FOR ${ip} IN @{controller_list}
220 \ ${resp}= ONOS Get ${ip} topology
221 \ Log ${resp.content}
222 \ ${devices}= Get Json Value ${resp.content} /devices
223 \ ${clusters}= Get Json Value ${resp.content} /clusters
224 \ Should Be Equal As Strings ${devices} ${device_count}
225 \ #Should Be Equal As Strings ${clusters} ${cluster_count}
226
227Ensure No Switches in Topology
228 [Arguments] ${cluster_count}
229 [Documentation] Verifies the topology sees no devices
230 : FOR ${ip} IN @{controller_list}
231 \ ${resp}= ONOS Get ${ip} topology
232 \ Log ${resp.content}
233 \ ${devices}= Get Json Value ${resp.content} /devices
234 \ ${clusters}= Get Json Value ${resp.content} /clusters
235 \ Should Be Equal As Strings ${devices} 0
236 \ #Should Be Equal As Strings ${clusters} ${cluster_count}
237
238ONOS Get
239 [Arguments] ${session} ${noun}
240 [Documentation] Common keyword to issue GET requests to the controller. Arguments are the session (controller ip) and api path
241 ${resp} RequestsLibrary.Get ${session} /onos/v1/${noun}
242 Log ${resp.content}
243 [Return] ${resp}
244
245Ensure Switch Count
246 [Arguments] ${switch_count}
247 [Documentation] Verfies the device count (passed in as arg) on each controller is accurate.
248 : FOR ${ip} IN @{controller_list}
249 \ ${resp} ONOS Get ${ip} devices
250 \ Log ${resp.content}
251 \ Should Be Equal As Strings ${resp.status_code} 200
252 \ Should Not Be Empty ${resp.content}
253 \ ${jsondata} To JSON ${resp.content}
254 \ ${length}= Get Length ${jsondata['devices']}
255 \ Should Be Equal As Integers ${length} ${switch_count}
256
257Ensure Switches are Available
258 [Documentation] Verifies that the switch's availability state is true on all switches through all controllers
259 : FOR ${ip} IN @{controller_list}
260 \ ${resp} ONOS Get ${ip} devices
261 \ Log ${resp.content}
262 \ Should Be Equal As Strings ${resp.status_code} 200
263 \ Should Not Be Empty ${resp.content}
264 \ ${jsondata} To JSON ${resp.content}
265 \ #Robot tweak to do a nested for loop
266 \ Check Each Switch Status ${jsondata} True
267
268Check Each Switch Status
269 [Arguments] ${jdata} ${bool}
270 ${length}= Get Length ${jdata['devices']}
271 : FOR ${INDEX} IN RANGE 0 ${length}
272 \ ${data}= Get From List ${jdata['devices']} ${INDEX}
273 \ ${status}= Get From Dictionary ${data} available
274 \ Should Be Equal As Strings ${status} ${bool}
275
276Ensure No Switches are Available
277 [Documentation] Verifies that the switch's availability state is false on all switches through all controllers
278 : FOR ${ip} IN @{controller_list}
279 \ ${resp} ONOS Get ${ip} devices
280 \ Log ${resp.content}
281 \ Should Be Equal As Strings ${resp.status_code} 200
282 \ Should Not Be Empty ${resp.content}
283 \ ${jsondata} To JSON ${resp.content}
284 \ Check Each Switch Status ${jsondata} False
285
286Check Each Switch Individually
287 [Documentation] Currently just observe each information the device has. Checks can easily be implemented
288 : FOR ${ip} IN @{controller_list}
289 \ ${resp} ONOS Get ${ip} devices
290 \ Should Be Equal As Strings ${resp.status_code} 200
291 \ ${jsondata} To JSON ${resp.content}
292 \ #Robot tweak to do a nested for loop
293 \ Check Each Switch ${jsondata}
294
295Check Each Switch
296 [Arguments] ${jdata}
297 ${length}= Get Length ${jdata['devices']}
298 @{dpid_list}= Create List
299 : FOR ${INDEX} IN RANGE 0 ${length}
300 \ ${devicedata}= Get From List ${jdata['devices']} ${INDEX}
301 \ ${id}= Get From Dictionary ${devicedata} id
302 \ Append To List ${dpid_list} ${id}
303 \ Log ${dpid_list}
304 ${length}= Get Length ${dpid_list}
305 : FOR ${i} IN @{dpid_list}
306 \ ${resp} ONOS Get ${ip} devices/${i}
307 \ Log ${resp.content}
308
309Ensure Switch Role
310 [Arguments] ${role}
311 [Documentation] Verifies that the controller's role for each switch is MASTER (default in standalone mode)
312 : FOR ${ip} IN @{controller_list}
313 \ ${resp} ONOS Get ${ip} devices
314 \ Log ${resp.content}
315 \ Should Be Equal As Strings ${resp.status_code} 200
316 \ Should Not Be Empty ${resp.content}
317 \ ${jsondata} To JSON ${resp.content}
318 \ Ensure Role ${jsondata} ${role}
319
320Ensure Role
321 [Arguments] ${jdata} ${role}
322 ${length}= Get Length ${jdata['devices']}
323 : FOR ${INDEX} IN RANGE 0 ${length}
324 \ ${data}= Get From List ${jdata['devices']} ${INDEX}
325 \ ${status}= Get From Dictionary ${data} role
326 \ Should Be Equal As Strings ${status} ${role}
327
328Get Karaf Logs
329 [Arguments] ${controller}
330 [Documentation] Compresses all the karaf log files on each controler and puts them on your pybot execution machine (in /tmp)
331 Run Command On Remote System ${controller} tar -zcvf /tmp/${SUITE NAME}.${controller}.tar.gz ${ONOS_HOME}/log
332 SSHLibrary.Open Connection ${controller} prompt=${LINUX_PROMPT} timeout=${prompt_timeout}
333 Login With Public Key ${CONTROLLER_USER} ${USER_HOME}/.ssh/id_rsa any
334 SSHLibrary.Get File /tmp/${SUITE NAME}.${controller}.tar.gz /tmp/
335 SSHLibrary.Close Connection