Wrote part of configuration chapter
device-config
bridging-unicast
xconnect
pseudowire
Change-Id: Ib393d52e7bdc606c705e1e14dd52c9fcd77c0755
diff --git a/configuration/aaa.rst b/configuration/aaa.rst
new file mode 100644
index 0000000..9af2cfd
--- /dev/null
+++ b/configuration/aaa.rst
@@ -0,0 +1,2 @@
+AAA
+===
diff --git a/configuration/address-assignment.rst b/configuration/address-assignment.rst
new file mode 100644
index 0000000..8568152
--- /dev/null
+++ b/configuration/address-assignment.rst
@@ -0,0 +1,50 @@
+Address Assignment
+==================
+
+.. tip::
+ We strongly recommend you to setup DHCP relay and configure the hosts to **obtain address via DHCP**.
+
+DHCP Relay
+----------
+
+
+Static IP
+---------
+Although we strongly recommend to use `DHCP Relay`_ for IP assignment,
+it is also possible to statically configure the IP address and route on the host.
+
+1. **Configure the IP address and subnet mask**
+
+ Make sure the IP address and the subnet mask on the fabric network interface of the host is consistent with
+ the information in the Network Configuration section. For example, you can run
+
+ .. code-block:: console
+
+ # ip addr add 10.0.0.1/24 dev mlx0
+
+2. **Configure the default route**
+
+ Make sure you change the default route of the host to the interface IP of the leaf switch it connects to.
+ For example, you can run
+
+ .. code-block:: console
+
+ # ip route add default via 10.0.0.254
+
+ .. note::
+ In the case that you want to keep default route through the management network,
+ you need to add routes to all other subnets in the network one by one.
+
+3. **Trigger host learning**
+
+ We need to let ONOS learn the host in order to program corresponding flows and groups.
+ This is automatically done as part of the DHCP process.
+ However, we need to manually triggers it by sending an ARP or ND packet if the host is configured to use static IP.
+
+ .. code-block:: console
+
+ # arping -c 1 ${GATEWAY_IP}
+
+ .. code-block:: console
+
+ # ndsend ${HOST_IP} ${INTF}
diff --git a/configuration/bridging-unicast.rst b/configuration/bridging-unicast.rst
new file mode 100644
index 0000000..ede859d
--- /dev/null
+++ b/configuration/bridging-unicast.rst
@@ -0,0 +1,153 @@
+Bridging and Unicast Routing
+============================
+
+Access Ports
+------------
+The necessary but minimum configuration for an access port is simply a VLAN.
+
+.. code-block:: json
+ :linenos:
+
+ {
+ "ports" : {
+ "of:0000000000000204/12" : {
+ "interfaces" : [{
+ "name" : "serverA-intf",
+ "vlan-untagged": 10
+ }]
+ },
+ "of:0000000000000204/16" : {
+ "interfaces" : [{
+ "name" : "serverB-intf",
+ "vlan-untagged": 10
+ }]
+ }
+ }
+ }
+
+The example above shows two ports (12 and 16) on switch of:204 that have been assigned to VLAN 10 using the ``vlan-untagged`` keyword.
+It simply means that packets come in and leave out of these switches untagged,
+but internally they are assigned VLAN 10 and they belong to the bridging domain defined for VLAN 10.
+
+``name`` is used to associate the interface with a globally unique, user friendly name. It can be omitted.
+
+With the configuration shown above, the packets will always be bridged, but they cannot be routed out of the VLAN (e.g. to other subnets).
+To add the capability to route out of VLAN 10, we need to add a subnet/gateway IP (similar to `interface-vlans or SVIs in traditional networks <https://www.youtube.com/watch?v=bUXpmiJpGb0>`_).
+
+.. code-block:: json
+ :linenos:
+
+ {
+ "ports" : {
+ "of:0000000000000204/12" : {
+ "interfaces" : [{
+ "name" : "serverA-intf",
+ "ips" : [ "10.0.1.254/24"],
+ "vlan-untagged": 10
+ }]
+ },
+ "of:0000000000000204/16" : {
+ "interfaces" : [{
+ "name" : "serverB-intf",
+ "ips" : [ "10.0.1.254/24"],
+ "vlan-untagged": 10
+ }]
+ }
+ }
+ }
+
+In this example, VLAN 10 is associated with subnet ``10.0.1.0/24``, and the gateway IP for hosts in this subnet is ``10.0.1.254/32``.
+When the desire is to route out of a VLAN, this assignment is currently necessary on all ports configured in the same VLAN.
+
+.. note::
+
+ Typically we only expect a single subnet for a VLAN. Similar to traditional networks, for us, a subnet == VLAN. Different VLANs should be configured in different subnets.
+ In certain use-cases, it may be necessary to configure multiple subnets in the same VLAN. This is possible by adding more subnet/gateway IPs in the ``ips`` array.
+
+
+Tagged Ports
+------------
+Tagged port configuration is similar.
+
+.. code-block:: json
+ :linenos:
+
+ {
+ "ports" : {
+ "of:0000000000000204/24" : {
+ "interfaces" : [{
+ "name" : "serverA-intf",
+ "ips" : [ "10.0.2.254/24", "10.0.4.254/24" ],
+ "vlan-tagged" : [ 20, 40 ]
+ }]
+ }
+ }
+ }
+
+The configuration above for port 24 on switch of:204 shows two VLANs 20 and 40 configured on that port, with corresponding subnets and gateway IPs.
+Note that there is no specific ordering required in the ``ips`` or ``vlan-tagged`` arrays to correlate the VLANs to their corresponding subnets.
+In a future release, we will correlate VLAN and subnets configuration in a more readable way.
+
+
+Native VLAN on Tagged Ports
+---------------------------
+An additional configuration ``vlan-native`` possible on tagged ports includes the ability to specify a VLAN (and thus a bridging domain) for incoming untagged packets.
+Typically, such configuration in trunk ports in traditional networks is referred to a native VLAN.
+
+.. code-block:: json
+ :linenos:
+
+ {
+ "ports" : {
+ "of:0000000000000204/24" : {
+ "interfaces" : [ {
+ "name" : "serverA-intf",
+ "ips" : [ "10.0.2.254/24", "10.0.4.254/24", "10.0.1.254/24" ],
+ "vlan-tagged" : [ 20, 40 ],
+ "vlan-native" : 10
+ }]
+ }
+ }
+ }
+
+Note that it is also necessary to configure the subnet/gateway IP corresponding to the native VLAN if you wish to route out of that VLAN.
+
+
+Configuring interface for IPv6
+------------------------------
+It is similar to configure IPv6 routing. Simply replace the addresses in ``ips`` with IPv6 addresses. For example,
+
+.. code-block:: json
+ :linenos:
+
+ {
+ "ports" : {
+ "of:0000000000000204/24" : {
+ "interfaces" : [ {
+ "name" : "serverA-intf",
+ "ips" : [ "10.0.2.254/24", "2000::1ff/120" ],
+ "vlan-tagged" : [ 20, 40 ]
+ }]
+ }
+ }
+ }
+
+
+Notes
+-----
+There is no need to configure ports on switches that are meant to connect to other switches.
+The VLAN (untagged or tagged) configuration is only meant for ports that are connected to hosts (edge ports).
+
+.. image:: images/vlan-config.png
+
+Furthermore, note that the same VLAN can be configured on multiple ToRs - e.g. vlan 20 in the figure above.
+However this does not mean that the ports are in the same bridging domain, because in the fabric, the communication between ToRs is through a routed network. '
+In other words, a host on VLAN 20 (untagged or tagged) connected to one ToR can communicate with another host on VLAN 20 (untagged or tagged) connected to a different ToR,
+but the MAC addresses will change as the traffic goes through a routed network.
+
+Please do not use this feature to connect switches in unsupported topologies as shown in the example below.
+The fabric is not designed to be one big Ethernet fabric. The bridging domain is restricted to within one ToR.
+If the bridging domain is extended across two ToRs directly linked to each other, there is a chance of loops.
+In other words, the ToRs/Leafs are not standalone 802.1Q bridges, and should not be used as such.
+
+.. image:: images/vlan-config-invalid.png
diff --git a/configuration/device-config.rst b/configuration/device-config.rst
new file mode 100644
index 0000000..6bc7e66
--- /dev/null
+++ b/configuration/device-config.rst
@@ -0,0 +1,43 @@
+Device Configuration
+====================
+Each switch in Trellis requires a device config.
+
+.. code-block:: json
+ :linenos:
+
+ {
+ "devices" : {
+ "of:0000000000000001" : {
+ "segmentrouting" : {
+ "ipv4NodeSid" : 101,
+ "ipv4Loopback" : "192.168.0.201",
+ "ipv6NodeSid" : 111,
+ "ipv6Loopback" : "2000::c0a8:0201",
+ "routerMac" : "00:00:00:00:02:01",
+ "isEdgeRouter" : true,
+ "adjacencySids" : []
+ },
+ "basic" : {
+ "name" : "Leaf-R1",
+ "driver" : "ovs-ofdpa"
+ }
+ }
+ }
+ }
+
+- ``of:0000000000000001``: DPID of the device.
+- ``ipv4NodeSid``: IPv4 node segment ID, which is used as an MPLS label in forwarding IPv4 traffic. Can be arbitrary and should be globally unique.
+- ``ipv4Loopback``: IPv4 loopback address. Can be arbitrary, should be globally unique and should not be part of the same subnet(s) defined on the data plane ports (see port config).
+- ``ipv6NodeSid``: IPv6 node segment ID, which is used as an MPLS label in forwarding IPv6 traffic. Can be arbitrary and should be globally unique. Only required when using IPv6.
+- ``ipv6Loopback``: IPv6 loopback address. Can be arbitrary, should be globally unique and should not be part of the same subnet(s) defined on the data plane ports (see port config).
+ Only required when using IPv6.
+- ``routerMac``: Router MAC address. Can be arbitrary and should be globally unique.
+ This MAC address will be used to reply the ARP request for the loopback IP or the Interface IP that will be introduced later.
+ (We recommend using the MAC address of the device's management interface as the router MAC.)
+- ``isEdgeRouter``: True for leaf switches. False for spine switches.
+- ``adjacencySids``: Deprecated. Just put an empty array for now.
+- ``name``: Name of the device. It is an arbitrary name to identify the device easily.
+- ``driver``: This tells ONOS which OpenFlow driver should be loaded for this device. It overrides the driver selected by ONOS automatically when the device connects.
+
+ - When using OpenvSwitch, set this to ``ovs-ofdpa``
+ - When using hardware switches, set this to ``ofdpa3`` or eliminate ``driver`` entirely as the correct driver configuration will be set automatically by ONOS
diff --git a/configuration/dual-homing.rst b/configuration/dual-homing.rst
new file mode 100644
index 0000000..367152c
--- /dev/null
+++ b/configuration/dual-homing.rst
@@ -0,0 +1,2 @@
+Dual Homing
+===========
diff --git a/configuration/external-connectivity.rst b/configuration/external-connectivity.rst
new file mode 100644
index 0000000..fe91e21
--- /dev/null
+++ b/configuration/external-connectivity.rst
@@ -0,0 +1,2 @@
+External Connectivity
+=====================
diff --git a/configuration/images/pw-rest.png b/configuration/images/pw-rest.png
new file mode 100644
index 0000000..c586dab
--- /dev/null
+++ b/configuration/images/pw-rest.png
Binary files differ
diff --git a/configuration/images/vlan-config-invalid.png b/configuration/images/vlan-config-invalid.png
new file mode 100644
index 0000000..c0f8196
--- /dev/null
+++ b/configuration/images/vlan-config-invalid.png
Binary files differ
diff --git a/configuration/images/vlan-config.png b/configuration/images/vlan-config.png
new file mode 100644
index 0000000..dd93425
--- /dev/null
+++ b/configuration/images/vlan-config.png
Binary files differ
diff --git a/configuration/macsec.rst b/configuration/macsec.rst
new file mode 100644
index 0000000..91306c9
--- /dev/null
+++ b/configuration/macsec.rst
@@ -0,0 +1,2 @@
+MAC Security
+============
diff --git a/configuration/multicast.rst b/configuration/multicast.rst
new file mode 100644
index 0000000..ca8cb97
--- /dev/null
+++ b/configuration/multicast.rst
@@ -0,0 +1,2 @@
+Multicast
+=========
diff --git a/configuration/pseudowire.rst b/configuration/pseudowire.rst
new file mode 100644
index 0000000..16998db
--- /dev/null
+++ b/configuration/pseudowire.rst
@@ -0,0 +1,180 @@
+Pseudowire
+==========
+In Trellis, we assign different subnets to hosts that attach to different edge (leaf) switches.
+These hosts communicate over IP when traversing the spine switches, meaning that the layer-2 header is not preserved across transportation.
+However, in some use cases it is beneficial for hosts that connect to different leaves to be able to talk directly over L2 and also belong to the same IP subnet.
+We implemented Pseudowire (PW) in Trellis to fulfill this requirement.
+
+Pseudowires create tunnels between different connection points that transport L2 traffic between them.
+Pseudowires can handle single-tagged/double-tagged traffic and are able to modify the VLAN tags in some cases.
+
+.. note::
+ Pseudowires are only supported on OF-DPA hardware switches.
+
+Managing pseudowires via REST
+-----------------------------
+You can find the REST API documentation at ``http://${ONOS-IP}:8181/onos/v1/docs``.
+Please click the drop down list and select Segment Routing Server.
+
+.. image:: images/pw-rest.png
+ :width: 700px
+
+We configure and implement PWs through the dedicated REST API.
+
+.. code-block:: console
+
+ $ curl --user onos:rocks -X POST -H 'Content-Type:application/json' http://${ONOS-IP}:8181/onos/segmentrouting/pseudowire/ -d@pseudowire.json
+ $ curl --user onos:rocks -X GET -H 'Content-Type:application/json' http://${ONOS-IP}:8181/onos/segmentrouting/pseudowire/ -d@pseudowire.json
+ $ curl --user onos:rocks -X DELETE -H 'Content-Type:application/json' http://${ONOS-IP}:8181/onos/segmentrouting/pseudowire/ -d@pseudowire.json
+
+Below is an example configuration for a single pseudowire,
+which transfers L2 double tagged traffic from cp1 to cp2 and also changes the outer VLAN tag of the packets.
+
+
+.. code-block:: json
+
+ {
+ "pwId" : 42,
+ "cP1": "of:0000000000000205/25",
+ "cP2": "of:0000000000000206/33",
+ "cP1InnerTag": "101",
+ "cP1OuterTag": "110",
+ "cP2InnerTag": "101",
+ "cP2OuterTag": "111",
+ "mode": "RAW",
+ "sdTag": "",
+ "pwLabel": "1572"
+ }
+
+- ``cP1``: First connection point of PW
+- ``cP2``: Second connection point of PW
+- ``cP1InnerTag / cp1OuterTag``: The expected inner and outer VLAN tags for ``cp1``. If you only want to use a single incoming tag,
+ use the innerTag, and use ``None`` for the outerTag.
+- ``cP2InnerTag / cp2OuterTag``: The expected inner and outer VLAN tags for ``cp2``.
+- ``mode``: Pseudowire mode - RAW and TAGGED. Only RAW mode is supported in Trellis.
+- ``sdTag``: Service delimiting tag, only meaningful when in TAGGED mode.
+- ``pwLabel``: Unique pseudowire label.
+- ``pwId``: ID of the pseudowire.
+
+The REST API also supports bulk addition and removal of pseudowire.
+
+.. code-block:: console
+
+ $ curl --user onos:rocks -X POST -H 'Content-Type:application/json'http://<controller-ip>:8181/onos/segmentrouting/pseudowire/bulk -d@pseudowires.json
+ $ curl --user onos:rocks -X DELETE -H 'Content-Type:application/json'http://<controller-ip>:8181/onos/segmentrouting/pseudowire/bulk -d@pseudowires.json
+
+with the following example configuration:
+
+.. code-block:: json
+
+ {
+ "pseudowires": [{
+ "pwId": 43,
+ "cP1": "of:0000000000000227/15",
+ "cP2": "of:0000000000000205/33",
+ "cP1InnerTag": "101",
+ "cP1OuterTag": "",
+ "cP2InnerTag": "101",
+ "cP2OuterTag": "",
+ "mode": "RAW",
+ "sdTag": "",
+ "pwLabel": "2572"
+ },
+ {
+ "pwId": 42,
+ "cP1": "of:0000000000000201/12",
+ "cP2": "of:0000000000000204/12",
+ "cP1InnerTag": "101",
+ "cP1OuterTag": "",
+ "cP2InnerTag": "101",
+ "cP2OuterTag": "",
+ "mode": "RAW",
+ "sdTag": "",
+ "pwLabel": "1572"
+ }
+ ]
+ }
+
+
+Managing pseudowires via CLI
+----------------------------
+
+To add a pseudowire:
+
+.. code-block:: text
+
+ onos> sr-pw-add --help
+ DESCRIPTION
+ onos:sr-pw-add
+
+ Add a pseudowire to the network configuration, if it already exists update it.
+
+ SYNTAX
+ onos:sr-pw-add [options] pwId pwLabel mode sDTag cP1 cP1InnerVlan cP1OuterVlan cP2 cP2InnerVlan cP2OuterVlan
+
+ <omitted>
+
+ onos> sr-pw-add 42 1000 RAW None of:0000000000000204/12 100 10 of:0000000000000205/12 100 40
+
+To list existing pseudowires:
+
+.. code-block:: text
+
+ onos> sr-pw-list
+
+To remove a pseudowire:
+
+.. code-block:: text
+
+ onos> sr-pw-remove 42
+
+
+Type of pseudowires
+-------------------
+- **Leaf - Leaf** : As explained above we support leaf - leaf pseudowires.
+
+- **Spine - Leaf** : We further support leaf to spine pseudowires,
+ where one termination point resides in a spine switch rather than a leaf switch.
+ Please note that the encapsulated traffic for these types of pseudowires is carried **tagged** between leaf and spines,
+ the reason for this is due to some hardware limitation. The **reserved transport VLAN is 4093**.
+
+- Multi-stage
+
+ - **Leaf - Spine-1 - Spine-2**: Where Spine-1 and Spine-2 are interconnected between them and are part of distinct topologies.
+ - **Leaf_1 - Spine_1 - Spine_2 - Leaf_2**: Where leaf-1 and leaf-2 are part of different fabric topologies which are interconnected between them between two spines.
+
+
+Common errors
+-------------
+If your pseudowires do not work make sure you have done the following in the host side :
+
+- Disabled VLAN offloading in the VM interfaces and also in the compute node interfaces.
+- Set the compute node interfaces and the VM interfaces at promiscuous mode.
+- In general, do the above for all the interfaces and bridges participated in the topology (either in a VM or physical server).
+- Before programming pseudowires make sure that the topology is completely up and running and also the configuration is added to the controller and the fabric is
+ in an operational state.
+
+
+Restrictions
+------------
+We list now some restrictions of pseudowire support, some of them stem from the implementation while others from switch hardware constraints.
+
+- We only support pseudowires with the same type of tagged traffic at each end.
+ For example, if cP1 handles single tagged traffic then cP2 must handle single tagged traffic (same applies for single tagged and double tagged traffic).
+ However, for pseudowires with single tagged traffic we support changing the tag,
+ and for pseudowires with double tagged traffic we support changing the outer tag.
+ We would like to support more use cases, however we are limited by hardware pipeline restrictions.
+ In the future, and if needed, we might address this by using the egress tables of OF-DPA.
+- As of now, we only support RAW mode. We will support also TAGGED mode in the future if needed by any use case.
+ Since, we only support RAW mode the sdTag must always be empty or None.
+
+
+Implementation limits
+---------------------
+- **Link failures**: We partially support link failures.
+ Specifically, our PW implementation does not properly handle link failures when the link involved is the one used for pseudowire traffic at the spine switches towards a leaf switch.
+ This is due to an OF-DPA limitation which does not allow the use of MPLS ECMP groups in the spine.
+ We plan to address this in the next release.
+
+- **Device failures**: We do not handle device failures currently, on a device failure please remove and re-install the pseudowire.
+ We will address this soon.
diff --git a/configuration/route-service.rst b/configuration/route-service.rst
new file mode 100644
index 0000000..d776deb
--- /dev/null
+++ b/configuration/route-service.rst
@@ -0,0 +1,8 @@
+Route Service
+=============
+
+Static routes
+-------------
+
+Route blackholing
+-----------------
diff --git a/configuration/xconnect.rst b/configuration/xconnect.rst
new file mode 100644
index 0000000..6486d96
--- /dev/null
+++ b/configuration/xconnect.rst
@@ -0,0 +1,73 @@
+VLAN Cross Connect
+==================
+VLAN cross connect creates a L2 bridge between two given ports on the same device.
+Once configured, every packets arriving at one of the port with specific VLAN tag will be sent to another port directly.
+Current implementation only matches on the outermost VLAN tag. If the packet is double tagged, the S-tag will be matched and both the S-tag and C-tag will be persisted.
+
+VLAN cross connect only works on the same device.
+If you are looking for transporting L2 traffic across devices, please refer to :doc:`pseoduwire <pseudowire>`
+
+View cross connect via CLI
+--------------------------
+To view a list of cross connects configured in the system through ONOS CLI:
+
+.. code-block:: text
+
+ sr-xconnect
+
+Add cross connect via CLI
+-------------------------
+
+.. code-block:: text
+
+ sr-xconnect-add <deviceId> <vlanId> <port1> <port2>
+
+- ``deviceId``: device ID, e.g. **of:0000000000000201**
+- ``vlanId``: VLAN ID, e.g. **94**
+- ``port1``: One end of the cross connect, e.g. **2**
+- ``port2``: Another end of the cross connect, e.g. **54**
+
+Remove cross connect via CLI
+----------------------------
+
+.. code-block:: text
+
+ sr-xconnect-remove <deviceId> <vlanId>
+
+- ``deviceId``: device ID, e.g. **of:0000000000000201**
+- ``vlanId``: VLAN ID, e.g. **94**
+
+View cross connect via REST
+---------------------------
+
+.. code-block:: console
+
+ $ curl -X GET --header 'Accept: application/json' 'http://${ONOS-IP}:8181/onos/segmentrouting/xconnect'
+
+Add cross connect via REST
+--------------------------
+
+.. code-block:: console
+
+ $ curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{ \
+ "deviceId": "of:0000000000000201", \
+ "vlanId": "94", \
+ "ports": [ 1, 2 ] \
+ }' 'http://${ONOS-IP}/onos/segmentrouting/xconnect'
+
+- ``deviceId``: device ID, e.g. **of:0000000000000201**
+- ``vlanId``: VLAN ID, e.g. **94**
+- ``ports``: endpoints of the cross connect, e.g. **[1, 2]**
+
+Remove cross connect via REST
+-----------------------------
+
+.. code-block:: console
+
+ $ curl -X DELETE --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{ \
+ "deviceId": "of:0000000000000201", \
+ "vlanId": "94" \
+ }' 'http://${ONOS-IP}:8181/onos/segmentrouting/xconnect'
+
+- ``deviceId``: device ID, e.g. **of:0000000000000201**
+- ``vlanId``: VLAN ID, e.g. **94**
\ No newline at end of file
