Update Sphinx version and versioning process
- Updated to newer Sphinx version
- Changed to use doc8 to check .rst files
- Added a dictionary (dict.txt) and spellchecking
- Reformatted and fixed issues that were found in the content
Change-Id: If7b35e01ee8be25dbbd1ecd0e67b264aa6cc2a94
diff --git a/configuration/aaa.rst b/configuration/aaa.rst
index a2da53d..8d839f2 100644
--- a/configuration/aaa.rst
+++ b/configuration/aaa.rst
@@ -3,8 +3,9 @@
Introduction
------------
-In this section, we will explain how to use Trellis with AAA service, which can be used to authenticate a client host.
-We will explain how this works with a simple **single switch** topology.
+In this section, we will explain how to use Trellis with AAA service, which can
+be used to authenticate a client host. We will explain how this works with a
+simple **single switch** topology.
.. image:: ../images/config-aaa.png
@@ -13,8 +14,9 @@
Activate AAA app
^^^^^^^^^^^^^^^^
-We need to install and activate AAA app separately since it is located in a separate (CORD) repository.
-There are multiple methods to install and activate a pre-compiled app. Let's use CLI now.
+We need to install and activate AAA app separately since it is located in a
+separate (CORD) repository. There are multiple methods to install and activate
+a pre-compiled app. Let's use CLI now.
.. code-block::console
@@ -23,10 +25,10 @@
Provide network configuration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-We need to provide AAA configuration in the apps section of network configuration.
+We need to provide AAA configuration in the apps section of network
+configuration.
.. code-block:: json
- :linenos:
{
"apps": {
@@ -42,8 +44,12 @@
- ``radiusIp``: The IP address of the Radius server
-- ``radiusServerPort``: The UDP port of the Radius server. (Optional -- ONOS will use port 1812 by default).
-- ``radiusSecret``: The Radius secret. This needs to be consistent with the Radius server configuration
+
+- ``radiusServerPort``: The UDP port of the Radius server. (Optional -- ONOS
+ will use port 1812 by default).
+
+- ``radiusSecret``: The Radius secret. This needs to be consistent with the
+ Radius server configuration
Then push the JSON to ONOS:
@@ -57,10 +63,10 @@
Install FreeRadius
^^^^^^^^^^^^^^^^^^
-Technically all Radius server should work.
-However, the way to configure them are probably different case to case.
-Here we use FreeRadius on Ubuntu as an example.
-To install the Radius server, simply run:
+
+Technically, any Radius server should work. However, the way to configure them
+are probably different case to case. Here we use FreeRadius on Ubuntu as an
+example. To install the Radius server, simply run:
.. code-block:: console
@@ -71,9 +77,11 @@
Add a user
""""""""""
-We usually connect Radius server to a database where we store the user information.
-In this section, we statically configure a user to simplify the setup.
-To add a user ``admin`` with password ``cord_test``, edit ``/etc/freeradius/users`` and add following lines:
+
+We usually connect Radius server to a database where we store the user
+information. In this section, we statically configure a user to simplify the
+setup. To add a user ``admin`` with password ``cord_test``, edit
+``/etc/freeradius/users`` and add following lines:
.. code-block:: text
@@ -109,14 +117,16 @@
+ private_key_password = onos_test
.. note::
- The key and certificates required by TLS will locate under ``/etc/freeradius/certs`` by default.
- There will be three symbolic links link to ``ca.pem``, ``server.key``, ``server.pem``.
- We only need to change the symbolic links after we generates the keys and certificates.
+ The key and certificates required by TLS will locate under
+ ``/etc/freeradius/certs`` by default. There will be three symbolic links
+ link to ``ca.pem``, ``server.key``, ``server.pem``. We only need to change
+ the symbolic links after we generates the keys and certificates.
Therefore, we don't need to change the path in ``/etc/freeradius/eap.conf``
.. note::
- Both server certificate and client certificate need to be signed by the same CA certificate.
- Also note that each key we generate below needs a unique Common Name.
+ Both server certificate and client certificate need to be signed by the
+ same CA certificate. Also note that each key we generate below needs a
+ unique Common Name.
Generate CA certificate (ca.pem) and private key (privkey.pem)
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
@@ -146,14 +156,17 @@
Deploy keys and certificates
""""""""""""""""""""""""""""
-On the server side, please link **/etc/freeradius/{ca.pem, server.key, server.pem}** to the files we just generated.
-Also copy **ca.pem, client.key, client.pem** to the client side through a secured channel.
-They will later be used when testing the Radius authentication.
+On the server side, please link **/etc/freeradius/{ca.pem, server.key,
+server.pem}** to the files we just generated. Also copy **ca.pem, client.key,
+client.pem** to the client side through a secured channel. They will later be
+used when testing the Radius authentication.
Testing
-------
-We can use the ``wpa_supplicant`` as the test client. In case ``wpa_supplicant`` has not been installed, you can run ``sudo apt-get install wpasupplicant``
+We can use the ``wpa_supplicant`` as the test client. In case
+``wpa_supplicant`` has not been installed, you can run ``sudo apt-get install
+wpasupplicant``
Compose wpa_supplicant.conf
^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -178,8 +191,10 @@
Run the test client
^^^^^^^^^^^^^^^^^^^
+
.. tip::
- If you are using a Linux VM behind a bridge to send out this authentication message, make sure the Linux kernel of your host machine is 3.2 or above.
+ If you are using a Linux VM behind a bridge to send out this authentication
+ message, make sure the Linux kernel of your host machine is 3.2 or above.
Otherwise the EAPOL messages won't go through the bridge.
.. code-block:: console
diff --git a/configuration/bridging-unicast.rst b/configuration/bridging-unicast.rst
index 2dc8c92..7a590f9 100644
--- a/configuration/bridging-unicast.rst
+++ b/configuration/bridging-unicast.rst
@@ -7,10 +7,10 @@
Access Ports
------------
+
The necessary but minimum configuration for an access port is simply a VLAN.
.. code-block:: json
- :linenos:
{
"ports" : {
@@ -29,17 +29,23 @@
}
}
-The example above shows two ports (12 and 16) on switch of:204 that have been assigned to VLAN 10 using the ``vlan-untagged`` keyword.
+The example above shows two ports (12 and 16) on switch ``of:204`` that have
+been assigned to VLAN 10 using the ``vlan-untagged`` keyword.
+
It simply means that packets come in and leave out of these switches untagged,
-but internally they are assigned VLAN 10 and they belong to the bridging domain defined for VLAN 10.
+but internally they are assigned VLAN 10 and they belong to the bridging domain
+defined for VLAN 10.
-``name`` is used to associate the interface with a globally unique, user friendly name. It can be omitted.
+``name`` is used to associate the interface with a globally unique, user
+friendly name. It can be omitted.
-With the configuration shown above, the packets will always be bridged, but they cannot be routed out of the VLAN (e.g. to other subnets).
-To add the capability to route out of VLAN 10, we need to add a subnet/gateway IP (similar to `interface-vlans or SVIs in traditional networks <https://www.youtube.com/watch?v=bUXpmiJpGb0>`_).
+With the configuration shown above, the packets will always be bridged, but
+they cannot be routed out of the VLAN (e.g. to other subnets). To add the
+capability to route out of VLAN 10, we need to add a subnet/gateway IP (similar
+to `interface-vlans or SVIs in traditional networks
+<https://www.youtube.com/watch?v=bUXpmiJpGb0>`_).
.. code-block:: json
- :linenos:
{
"ports" : {
@@ -60,15 +66,24 @@
}
}
-In this example, VLAN 10 is associated with subnet ``10.0.1.0/24``, and the gateway IP for hosts in this subnet is ``10.0.1.254/32``.
-When the desire is to route out of a VLAN, this assignment is currently necessary on all ports configured in the same VLAN.
+In this example, VLAN 10 is associated with subnet ``10.0.1.0/24``, and the
+gateway IP for hosts in this subnet is ``10.0.1.254/32``.
+
+When the desire is to route out of a VLAN, this assignment is currently
+necessary on all ports configured in the same VLAN.
.. note::
- Typically we only expect a single subnet for a VLAN. Similar to traditional networks, for us, a subnet == VLAN. Different VLANs should be configured in different subnets.
- In certain use-cases, it may be necessary to configure multiple subnets in the same VLAN. This is possible by adding more subnet/gateway IPs in the ``ips`` array.
+ Typically we only expect a single subnet for a VLAN. Similar to traditional
+ networks, for us, a subnet == VLAN. Different VLANs should be configured in
+ different subnets.
+
+ In certain use-cases, it may be necessary to configure multiple subnets in
+ the same VLAN. This is possible by adding more subnet/gateway IPs in the
+ ``ips`` array.
.. tip::
One subnet cannot be configured on multiple leaf switches.
+
We usually configure one subnet for all the ports on the same leaf switch.
Tagged Ports
@@ -76,7 +91,6 @@
Tagged port configuration is similar.
.. code-block:: json
- :linenos:
{
"ports" : {
@@ -90,18 +104,26 @@
}
}
-The configuration above for port 24 on switch of:204 shows two VLANs 20 and 40 configured on that port, with corresponding subnets and gateway IPs.
-Note that there is no specific ordering required in the ``ips`` or ``vlan-tagged`` arrays to correlate the VLANs to their corresponding subnets.
-In a future release, we will correlate VLAN and subnets configuration in a more readable way.
+The configuration above for port 24 on switch of:204 shows two VLANs 20 and 40
+configured on that port, with corresponding subnets and gateway IPs.
+Note that there is no specific ordering required in the ``ips`` or
+``vlan-tagged`` arrays to correlate the VLANs to their corresponding subnets.
+
+In a future release, we will correlate VLAN and subnets configuration in a more
+readable way.
Native VLAN on Tagged Ports
---------------------------
-An additional configuration ``vlan-native`` possible on tagged ports includes the ability to specify a VLAN (and thus a bridging domain) for incoming untagged packets.
-Typically, such configuration in trunk ports in traditional networks is referred to a native VLAN.
+
+An additional configuration ``vlan-native`` possible on tagged ports includes
+the ability to specify a VLAN (and thus a bridging domain) for incoming
+untagged packets.
+
+Typically, such configuration in trunk ports in traditional networks is
+referred to a native VLAN.
.. code-block:: json
- :linenos:
{
"ports" : {
@@ -116,15 +138,16 @@
}
}
-Note that it is also necessary to configure the subnet/gateway IP corresponding to the native VLAN if you wish to route out of that VLAN.
-
+Note that it is also necessary to configure the subnet/gateway IP corresponding
+to the native VLAN if you wish to route out of that VLAN.
Configuring interface for IPv6
------------------------------
-It is similar to configure IPv6 routing. Simply replace the addresses in ``ips`` with IPv6 addresses. For example,
+
+It is similar to configure IPv6 routing. Simply replace the addresses in
+``ips`` with IPv6 addresses. For example:
.. code-block:: json
- :linenos:
{
"ports" : {
@@ -144,36 +167,63 @@
Router Advertisement overview
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Router advertisement application is for enabling **Router Advertisement** and **Router Solicitation** functionalities supported by IPv6 routers.
+
+Router advertisement application is for enabling **Router Advertisement** and
+**Router Solicitation** functionalities supported by IPv6 routers.
+
More details are available in `RFC 4861 <https://tools.ietf.org/html/rfc4861>`_.
-Application identifies which IPv6 interfaces are currently configured in the system and it will try to send out **unsolicited Router Advertisement** (RA) messages from these interfaces.
-Each such RA message will have two mandatory options named **Source link-layer address** and **MTU**.
-Additional RA option **prefix** can be enabled using component configuration **raGlobalPrefixConfStatus**.
+Application identifies which IPv6 interfaces are currently configured in the
+system and it will try to send out **unsolicited Router Advertisement** (RA)
+messages from these interfaces.
-Application also processes **Router Solicitations** (RS) sent from hosts. Upon receiving RS on a particular interface application stops RA transmission in that interface and immediately sends RA targeted to the solicited host. After that application continues unsolicited RA transmission on that interface.
+Each such RA message will have two mandatory options named **Source link-layer
+address** and **MTU**.
+
+Additional RA option **prefix** can be enabled using component configuration
+**raGlobalPrefixConfStatus**.
+
+Application also processes **Router Solicitations** (RS) sent from hosts. Upon
+receiving RS on a particular interface application stops RA transmission in
+that interface and immediately sends RA targeted to the solicited host. After
+that application continues unsolicited RA transmission on that interface.
Activate and configure RA
^^^^^^^^^^^^^^^^^^^^^^^^^
+
RA application can be activated from CLI by running
.. code-block:: console
onos> app activate routeradvertisement
-Behavior of RA application is controlled by ONOS component configuration subsystem and following are possible configuration options.
+Behavior of RA application is controlled by ONOS component configuration
+subsystem and following are possible configuration options.
- ``raThreadDelay``: Delay between consecutive RA transmissions
+
- ``raPoolSize``: Capacity of thread pool to be used for RA transmissions
-- ``raFlagMbitStatus``: RA flag “Managed address configuration” enabled/disabled
+
+- ``raFlagMbitStatus``: RA flag “Managed address configuration”
+ enabled/disabled
+
- ``raFlagObitStatus``: RA flag “Other configuration” enabled/disabled
-- ``raOptionPrefixStatus``: RA Option “prefix” is enabled/disabled. Router prefixes will be available in RA only if this flag is “true”
-- ``raGlobalPrefixConfStatus``: Enable switch level global prefix configuration.
- Once “raGlobalPrefixConfStatus” is enabled, RA prefix option is generated from port configuration of device, see for more details.
+
+- ``raOptionPrefixStatus``: RA Option “prefix” is enabled/disabled. Router
+ prefixes will be available in RA only if this flag is “true”
+
+- ``raGlobalPrefixConfStatus``: Enable switch level global prefix
+ configuration.
+
+ Once “raGlobalPrefixConfStatus” is enabled, RA prefix option is generated
+ from port configuration of device, see for more details.
Prefix details are picked up from network interface configuration.
+
RA app will filter out link-local IPs while preparing prefixes.
-For example, in following configuration, Prefix will include only **2001:0558:FF10:04C9::2:1ff/120**.
+
+For example, in following configuration, Prefix will include only
+**2001:0558:FF10:04C9::2:1ff/120**.
.. code-block:: json
@@ -192,8 +242,12 @@
Global prefix configuration
^^^^^^^^^^^^^^^^^^^^^^^^^^^
-In some cases, users may want to have a set of global prefix **advertised on all edge interfaces**.
-Such prefixes can be configured in **devices** section of network configuration in the following way.
+
+In some cases, users may want to have a set of global prefix **advertised on
+all edge interfaces**.
+
+Such prefixes can be configured in **devices** section of network configuration
+in the following way.
.. code-block:: json
@@ -208,23 +262,42 @@
}
.. note::
- When global prefix is configured, RA app will ignore any prefixes configured on switch interfaces.
+ When global prefix is configured, RA app will ignore any prefixes
+ configured on switch interfaces.
Notes about interface config
----------------------------
-There is no need to configure ports on switches that are meant to connect to other switches.
-The VLAN (untagged or tagged) configuration is only meant for ports that are connected to hosts (edge ports).
+
+There is no need to configure ports on switches that are meant to connect to
+other switches.
+
+The VLAN (untagged or tagged) configuration is only meant for ports that are
+connected to hosts (edge ports).
.. image:: ../images/config-vlan.png
-Furthermore, note that the same VLAN can be configured on multiple ToRs - e.g. vlan 20 in the figure above.
-However this does not mean that the ports are in the same bridging domain, because in the fabric, the communication between ToRs is through a routed network. '
-In other words, a host on VLAN 20 (untagged or tagged) connected to one ToR can communicate with another host on VLAN 20 (untagged or tagged) connected to a different ToR,
-but the MAC addresses will change as the traffic goes through a routed network.
+Furthermore, note that the same VLAN can be configured on multiple ToRs - e.g.
+vlan 20 in the figure above.
-Please do not use this feature to connect switches in unsupported topologies as shown in the example below.
-The fabric is not designed to be one big Ethernet fabric. The bridging domain is restricted to within one ToR.
-If the bridging domain is extended across two ToRs directly linked to each other, there is a chance of loops.
-In other words, the ToRs/Leafs are not standalone 802.1Q bridges, and should not be used as such.
+However this does not mean that the ports are in the same bridging domain,
+because in the fabric, the communication between ToRs is through a routed
+network.
+
+In other words, a host on VLAN 20 (untagged or tagged) connected to one ToR can
+communicate with another host on VLAN 20 (untagged or tagged) connected to a
+different ToR, but the MAC addresses will change as the traffic goes through a
+routed network.
+
+Please do not use this feature to connect switches in unsupported topologies as
+shown in the example below.
+
+The fabric is not designed to be one big Ethernet fabric. The bridging domain
+is restricted to within one ToR.
+
+If the bridging domain is extended across two ToRs directly linked to each
+other, there is a chance of loops.
+
+In other words, the ToRs/Leafs are not standalone 802.1Q bridges, and should
+not be used as such.
.. image:: ../images/config-vlan-invalid.png
diff --git a/configuration/device-config.rst b/configuration/device-config.rst
index 6da9acd..9a55895 100644
--- a/configuration/device-config.rst
+++ b/configuration/device-config.rst
@@ -3,7 +3,6 @@
Each switch in Trellis requires a device config.
.. code-block:: json
- :linenos:
{
"devices" : {
@@ -26,30 +25,55 @@
}
- ``of:0000000000000001``: DPID of the device.
-- ``ipv4NodeSid``: IPv4 node segment ID, which is used as an MPLS label in forwarding IPv4 traffic. Can be arbitrary and should be globally unique.
-- ``ipv4Loopback``: IPv4 loopback address. Can be arbitrary, should be globally unique and should not be part of the same subnet(s) defined on the data plane ports (see port config).
-- ``ipv6NodeSid``: IPv6 node segment ID, which is used as an MPLS label in forwarding IPv6 traffic. Can be arbitrary and should be globally unique. Only required when using IPv6.
-- ``ipv6Loopback``: IPv6 loopback address. Can be arbitrary, should be globally unique and should not be part of the same subnet(s) defined on the data plane ports (see port config).
- Only required when using IPv6.
-- ``routerMac``: Router MAC address. Can be arbitrary and should be globally unique.
- This MAC address will be used to reply the ARP request for the loopback IP or the Interface IP that will be introduced later.
- (We recommend using the MAC address of the device's management interface as the router MAC.)
+
+- ``ipv4NodeSid``: IPv4 node segment ID, which is used as an MPLS label in
+ forwarding IPv4 traffic. Can be arbitrary and should be globally unique.
+
+- ``ipv4Loopback``: IPv4 loopback address. Can be arbitrary, should be globally
+ unique and should not be part of the same subnet(s) defined on the data plane
+ ports (see port config).
+
+- ``ipv6NodeSid``: IPv6 node segment ID, which is used as an MPLS label in
+ forwarding IPv6 traffic. Can be arbitrary and should be globally unique. Only
+ required when using IPv6.
+
+- ``ipv6Loopback``: IPv6 loopback address. Can be arbitrary, should be globally
+ unique and should not be part of the same subnet(s) defined on the data plane
+ ports (see port config). Only required when using IPv6.
+
+- ``routerMac``: Router MAC address. Can be arbitrary and should be globally
+ unique. This MAC address will be used to reply the ARP request for the
+ loopback IP or the Interface IP that will be introduced later. (We recommend
+ using the MAC address of the device's management interface as the router
+ MAC.)
+
- ``isEdgeRouter``: True for leaf switches. False for spine switches.
+
- ``adjacencySids``: Deprecated. Just put an empty array for now.
+
- ``name``: Name of the device. It is an arbitrary name to identify the device easily.
-- ``driver``: This tells ONOS which OpenFlow driver should be loaded for this device. It overrides the driver selected by ONOS automatically when the device connects.
+
+- ``driver``: This tells ONOS which OpenFlow driver should be loaded for this
+ device. It overrides the driver selected by ONOS automatically when the
+ device connects.
- When using OpenvSwitch, set this to ``ovs-ofdpa``
- - When using hardware switches, set this to ``ofdpa3`` or eliminate ``driver`` entirely as the correct driver configuration will be set automatically by ONOS
+
+ - When using hardware switches, set this to ``ofdpa3`` or eliminate
+ ``driver`` entirely as the correct driver configuration will be set
+ automatically by ONOS
.. caution::
- We should avoid using reserved MPLS labels for ``ipv4NodeSid`` and ``ipv6NodeSid``.
- Please check here for the reserved values: http://www.iana.org/assignments/mpls-label-values/mpls-label-values.xhtml
+ We should avoid using reserved MPLS labels for ``ipv4NodeSid`` and
+ ``ipv6NodeSid``. Please check here for the reserved values:
+ http://www.iana.org/assignments/mpls-label-values/mpls-label-values.xhtml
.. note::
Most of the Trellis configurations support dynamic configuration updates.
- Unfortunately, Trellis currently **do not support dynamic device configuration updates**.
- You will have to restart the device when if corresponding device configuration changes.
+ Unfortunately, Trellis currently **do not support dynamic device
+ configuration updates**. You will have to restart the device when if
+ corresponding device configuration changes.
Having said that, when introducing a completely new device in the network,
- the device configurations pushed before the device's connection should apply correctly.
+ the device configurations pushed before the device's connection should
+ apply correctly.
diff --git a/configuration/dhcp-relay.rst b/configuration/dhcp-relay.rst
index efebaa9..c9e68ff 100644
--- a/configuration/dhcp-relay.rst
+++ b/configuration/dhcp-relay.rst
@@ -2,20 +2,29 @@
==========
.. tip::
- We strongly recommend you to setup DHCP relay and configure the hosts to **obtain address via DHCP**.
- See `Alternative: Configure static IP`_ if you want to statically configure IP address on each host.
+ We strongly recommend you to setup DHCP relay and configure the hosts to
+ **obtain address via DHCP**.
+ See `Alternative: Configure static IP`_ if you want to statically configure
+ IP address on each host.
Overview
--------
The DHCP relay app used in Trellis is an L3 relay.
-That is, it support relaying DHCP packets from/to a server that's not in the same subnet of the client.
+
+That is, it support relaying DHCP packets from/to a server that's not in the
+same subnet of the client.
Here's a list of features supported:
- DHCPv4 and DHCPv6
-- DHCP server directly attached to fabric leaves, or indirectly connected via upstream router
-- DHCP client directly attached to fabric leaves, or indirectly connected via `LDRA (Light-weight DHCP Relay Agent) <https://tools.ietf.org/html/rfc6221>`_
+
+- DHCP server directly attached to fabric leaves, or indirectly connected via
+ upstream router
+
+- DHCP client directly attached to fabric leaves, or indirectly connected via
+ `LDRA (Light-weight DHCP Relay Agent) <https://tools.ietf.org/html/rfc6221>`_
+
- Multiple DHCP servers for HA
.. note::
@@ -30,8 +39,11 @@
.. image:: ../images/config-dhcp.png
-In this case, the configuration involves first configuring the switch interface with the vlan/subnet the DHCP service is part of.
-For example, if I have a switch ``of:205`` with a DHCP server on port 24 on vlan 20, the port config looks like:
+In this case, the configuration involves first configuring the switch interface
+with the vlan/subnet the DHCP service is part of.
+
+For example, if I have a switch ``of:205`` with a DHCP server on port 24 on
+vlan 20, the port config looks like:
.. code-block:: json
@@ -47,7 +59,8 @@
}
}
-A second part of the configuration for the DHCP relay app requires a json configuration under the key apps:
+A second part of the configuration for the DHCP relay app requires a json
+configuration under the key apps:
.. code-block:: json
@@ -64,16 +77,24 @@
}
}
-Note that the dhcprelay app is configured with location of the DHCP server (the switch port to which it is connected to the fabric).
-It is also configured with the DHCP server IP, but it is no longer necessary to configure the MAC address of the server.
+Note that the dhcprelay app is configured with location of the DHCP server (the
+switch port to which it is connected to the fabric).
+
+It is also configured with the DHCP server IP, but it is no longer necessary to
+configure the MAC address of the server.
+
ONOS will automatically learn the MAC and VLAN corresponding to the serverIP.
Server reachable via external router
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-In this case, it is actually the external router that is directly connected to the fabric.
-This external router is already configured in the ports section of network-config (for vRouter functionality).
-For example, if the external router is connected to switch of:205 on port 1
+In this case, it is actually the external router that is directly connected to
+the fabric.
+
+This external router is already configured in the ports section of
+network-config (for vRouter functionality).
+
+For example, if the external router is connected to switch ``of:205`` on port 1
.. code-block:: json
@@ -89,8 +110,12 @@
}
}
-As before the ``ips`` and ``mac`` configured on port 1, actually correspond to the addresses configured in Quagga.
-The app config in this case, includes an additional field necessary to inform the dhcp-relay app of the gatewayIP through which the DHCP server can be reached.
+As before the ``ips`` and ``mac`` configured on port 1, actually correspond to
+the addresses configured in Quagga.
+
+The app config in this case, includes an additional field necessary to inform
+the dhcp-relay app of the gatewayIP through which the DHCP server can be
+reached.
.. code-block:: json
@@ -109,13 +134,15 @@
}
.. note::
- Note that the dhcpserverConnectPoint should now be the switchport to which the external router is connected to the fabric.
+ Note that the ``dhcpserverConnectPoint`` should now be the switchport to
+ which the external router is connected to the fabric.
Setup DHCP server
-----------------
Install DHCP server
^^^^^^^^^^^^^^^^^^^
+
Modern DHCP servers should support relayed DHCP request.
However, the way to configure them are probably different case to case.
Here we use **isc-dhcp-server** on Ubuntu as an example.
@@ -128,6 +155,7 @@
Configure DHCP Server
^^^^^^^^^^^^^^^^^^^^^
+
Two configuration files are required by DHCP server.
First, we need to specify which network interface the DHCP server should listen on.
@@ -141,7 +169,9 @@
To do that, we need to modify ``/etc/dhcp/dhcpd.conf`` and add the following lines.
Note that the subnet of ``eth1`` needs to be included.
-Otherwise, the DHCP server will not listen to the interface even though we have specified that in ``/etc/default/isc-dhcp-server``.
+
+Otherwise, the DHCP server will not listen to the interface even though we have
+specified that in ``/etc/default/isc-dhcp-server``.
.. code-block:: text
@@ -188,6 +218,7 @@
Testing
-------
+
The host should be able to obtain an IP address from the pool we specified.
Try to run ``dhclient`` and see if the host can get an IP address.
@@ -206,19 +237,28 @@
sudo dhclient -6 -P -r eth1 # for releasing prefix
-If something goes wrong, check ``/var/log/syslog`` for DHCP server log and run ``tcpdump`` on DHCP server to see if the DHCP packets from the host reach the server correctly.
-
+If something goes wrong, check ``/var/log/syslog`` for DHCP server log and run
+``tcpdump`` on DHCP server to see if the DHCP packets from the host reach the
+server correctly.
Additional Features
-------------------
DHCP Relay store
^^^^^^^^^^^^^^^^
-DHCP relay application stores information from DHCP packet which processed by the app, administrator can use CLI command ``dhcp-relay`` to query these information.
+
+DHCP relay application stores information from DHCP packet which processed by
+the app, administrator can use CLI command ``dhcp-relay`` to query these
+information.
+
The store provides these functionality:
-- Latest state of DHCP client (e.g. client location, last seen time, DHCP type...), for debugging purpose
-- For direct host, ONOS can find location and vlan from relay agent option, however, for indirect host, ONOS need to query last state from the store to find correct destination.
+- Latest state of DHCP client (e.g. client location, last seen time, DHCP
+ type...), for debugging purpose
+
+- For direct host, ONOS can find location and vlan from relay agent option,
+ however, for indirect host, ONOS need to query last state from the store to
+ find correct destination.
DHCPv6 Relay counter
@@ -226,7 +266,9 @@
There are two DHCPv6 packet counters which are Host basis counters and Global counters.
Host basis counters count and record DHCPv6 packets received on this host.
-It can be displayed by ``dhcp-relay counter``. These counters can be reset by typing ``dhcp-relay counter reset``.
+
+It can be displayed by ``dhcp-relay counter``. These counters can be reset by
+typing ``dhcp-relay counter reset``.
.. code-block:: console
@@ -249,7 +291,9 @@
onos> dhcp-relay counter reset
Global counters counts and records all DHCPv6 packets received in ONOS.
-It can be displayed by ``dhcp-relay-agg-counters``. These counters can be reset by typing ``dhcp-relay-agg-counters reset``.
+
+It can be displayed by ``dhcp-relay-agg-counters``. These counters can be reset
+by typing ``dhcp-relay-agg-counters reset``.
.. code-block:: console
@@ -269,26 +313,47 @@
Indirect client support
^^^^^^^^^^^^^^^^^^^^^^^
DHCP relay can support hosts which do not directly connect to Trellis fabric.
-These hosts usually connected to another LDRA, the LDRA will forward DHCP packet to/from Trellis network.
-For **DHCPv4**, packets from the LDRA includes a valid DHCP relay agent option (option 82).
-DHCP Relay application checks relay agent option and determine the DHCP packet comes from direct or indirect host.
+These hosts usually connected to another LDRA, the LDRA will forward DHCP
+packet to/from Trellis network.
+
+For **DHCPv4**, packets from the LDRA includes a valid DHCP relay agent option
+(option 82).
+
+DHCP Relay application checks relay agent option and determine the DHCP packet
+comes from direct or indirect host.
.. image:: ../images/config-dhcp-indirect.jpg
-ONOS uses circuit id option in relay agent option with specific format if DHCP packet comes without relay agent option, the format of circuit will be: ``ConnectPoint:VlanId``
-For example, the DHCP request/discover packet comes from ``of:000000000000001/1`` with ``VLAN 100``, the circuit ONOS put will be ``of:000000000000001/1:100`` and send DHCP packet to DHCP server.
-Indirect host won't put into host store. DHCP relay app will put IP address of indirect host to the route store, and use IP address of relay agent as next hop.
+ONOS uses circuit id option in relay agent option with specific format if DHCP
+packet comes without relay agent option, the format of circuit will be:
+``ConnectPoint:VlanId``
+
+For example, the DHCP request/discover packet comes from
+``of:000000000000001/1`` with ``VLAN 100``, the circuit ONOS put will be
+``of:000000000000001/1:100`` and send DHCP packet to DHCP server.
+
+Indirect host won't put into host store. DHCP relay app will put IP address of
+indirect host to the route store, and use IP address of relay agent as next
+hop.
**DHCPv6** clients will be handled similar to DHCPv4.
-One major difference is that DHCPv6 supports ``RELAY-FORWARD`` message type and ``InterfaceId`` option natively, so we utilize those fields to encode information.
+One major difference is that DHCPv6 supports ``RELAY-FORWARD`` message type and
+``InterfaceId`` option natively, so we utilize those fields to encode
+information.
Overwrite relay agent IP
^^^^^^^^^^^^^^^^^^^^^^^^
-The DHCP relay can overwrite the relay agent address (``giaddr`` in **DHCPv4**, ``link-addr`` in **DHCPv6**) in DHCP message for different device.
-If ``relayAgentIps`` is configured, the app will overwrite ``giaddr`` or ``link-addr`` before it forward the DHCP message to the server.
+
+The DHCP relay can overwrite the relay agent address (``giaddr`` in **DHCPv4**,
+``link-addr`` in **DHCPv6**) in DHCP message for different device.
+
+If ``relayAgentIps`` is configured, the app will overwrite ``giaddr`` or
+``link-addr`` before it forward the DHCP message to the server.
+
Otherwise, it will retain the original relay agent IP.
+
An example configuration is shown below:
.. code-block:: json
@@ -318,8 +383,13 @@
Configure multiple servers
^^^^^^^^^^^^^^^^^^^^^^^^^^
-DHCP server HA can be achieved by specifying additional server configuration objects.
-Client initiated packets like ``SOLICIT`` or ``REBIND`` shall be replicated and sent to all server objects.
+
+DHCP server HA can be achieved by specifying additional server configuration
+objects.
+
+Client initiated packets like ``SOLICIT`` or ``REBIND`` shall be replicated and
+sent to all server objects.
+
Below is an example of multiple server configuration:
.. code-block:: json
@@ -402,20 +472,28 @@
}
- ``dhcpServerConnectPoint``: represent the location of DHCP server
+
- ``serverIps``: IP address of the DHCP server, contains at least one IP address of DHCP server.
IP address can be IPv4 or IPv6 for different version of DHCP.
Will use first address if multiple IPv4 or IPv6 address configured.
-- ``gatewayIps``: Optional. Should be configured if the DHCP server is not directly connected to the Trellis network
- . It tells which gateway we need to send to reach the server.
+
+- ``gatewayIps``: Optional. Should be configured if the DHCP server is not
+ directly connected to the Trellis network. It tells which gateway we need to
+ send to reach the server.
.. note::
- - If ``indirect`` server configuration is not configured, the app will use ``default`` configuration for all cases.
+ - If ``indirect`` server configuration is not configured, the app will use
+ ``default`` configuration for all cases.
Ignoring DHCP relay on a particular VLAN
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-In some cases, it may be necessary to avoid punting DHCP packets to the controller, and letting them be forwarded normally through the data plane.
-In such cases, the DHCP relay application can be configured to avoid punting DHCP packets on a particular VLAN on a particular switch.
+
+In some cases, it may be necessary to avoid punting DHCP packets to the
+controller, and letting them be forwarded normally through the data plane.
+
+In such cases, the DHCP relay application can be configured to avoid punting
+DHCP packets on a particular VLAN on a particular switch.
.. code-block:: json
@@ -430,8 +508,8 @@
}
}
-In the example shown above, DHCP packets on vlan 24 are not punted to the controller from switches of:205 and of:206
-
+In the example shown above, DHCP packets on vlan 24 are not punted to the
+controller from switches of:205 and of:206
DHCPv6 Prefix Delegation (PD) Pushing
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -439,8 +517,12 @@
.. note::
This feature requires both ``dhcprelay`` and ``fpm`` apps to be activated
-PD pushing allows IPv6 prefixes from DhcpRelay to be sent over the FPM connection to Quagga where they will be configured as a static route.
-Prior to PD Pushing, the FPM connection was only used by Quagga in one direction to push routes to FPM. PD pushing is disabled by default in DHCP Relay and FPM.
+PD pushing allows IPv6 prefixes from DhcpRelay to be sent over the FPM
+connection to Quagga where they will be configured as a static route.
+
+Prior to PD Pushing, the FPM connection was only used by Quagga in one
+direction to push routes to FPM. PD pushing is disabled by default in DHCP
+Relay and FPM.
To enable in DHCP relay:
@@ -454,9 +536,16 @@
onos> dhcp-fpm-routes
-When PD pushing is enabled in FPM, by default the next-hop to be used for all prefixes pushed to Quagga will be retrieved from the first interface with ``RUR`` in the name in ONOS.
-Next-hop may also be configured using FPM component config. This will override a ``RUR`` interface if present.
-If there is no interface with ``RUR`` in the name and the next-hop is not configured, no prefixes can be pushed to Quagga even if PD pushing is enabled. For DhcpRelay, only the IPv6 next-hop is needed.
+When PD pushing is enabled in FPM, by default the next-hop to be used for all
+prefixes pushed to Quagga will be retrieved from the first interface with
+``RUR`` in the name in ONOS.
+
+Next-hop may also be configured using FPM component config. This will override
+a ``RUR`` interface if present.
+
+If there is no interface with ``RUR`` in the name and the next-hop is not
+configured, no prefixes can be pushed to Quagga even if PD pushing is enabled.
+For DhcpRelay, only the IPv6 next-hop is needed.
To enable in FPM:
@@ -482,12 +571,17 @@
.. note::
Quagga requires a patch to be able to receive Netlink Messages from FPM.
-
Clean up expired address and PD prefix
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-DHCPv6 relay cleans up stale IP address and pd prefix based on timer whose default interval is 24 hours (24 * 3600 secs = 86400 secs).
-If the preferred life time of ip address or pd prefix exceeds 1/2 of poll interval, they will be removed from ONOS.
-The poll interval can be modified by ``cfg set org.onosproject.dhcprelay.DhcpRelayManager dhcpPollInterval <newVal>``
+
+DHCPv6 relay cleans up stale IP address and pd prefix based on timer whose
+default interval is 24 hours (24 * 3600 secs = 86400 secs).
+
+If the preferred life time of ip address or pd prefix exceeds 1/2 of poll
+interval, they will be removed from ONOS.
+
+The poll interval can be modified by ``cfg set
+org.onosproject.dhcprelay.DhcpRelayManager dhcpPollInterval <newVal>``
.. code-block:: console
@@ -504,13 +598,15 @@
Alternative: Configure static IP
--------------------------------
-Although we strongly recommend to use `DHCP Relay`_ for IP assignment,
-it is also possible to statically configure the IP address and route on the host.
+
+Although we strongly recommend to use `DHCP Relay`_ for IP assignment, it is
+also possible to statically configure the IP address and route on the host.
1. **Configure the IP address and subnet mask**
- Make sure the IP address and the subnet mask on the fabric network interface of the host is consistent with
- the information in the Network Configuration section. For example, you can run
+ Make sure the IP address and the subnet mask on the fabric network interface
+ of the host is consistent with the information in the Network Configuration
+ section. For example, you can run
.. code-block:: console
@@ -518,8 +614,8 @@
2. **Configure the default route**
- Make sure you change the default route of the host to the interface IP of the leaf switch it connects to.
- For example, you can run
+ Make sure you change the default route of the host to the interface IP of
+ the leaf switch it connects to. For example, you can run
.. code-block:: console
@@ -531,9 +627,13 @@
3. **Trigger host learning**
- We need to let ONOS learn the host in order to program corresponding flows and groups.
+ We need to let ONOS learn the host in order to program corresponding flows
+ and groups.
+
This is automatically done as part of the DHCP process.
- However, we need to manually triggers it by sending an ARP or ND packet if the host is configured to use static IP.
+
+ However, we need to manually triggers it by sending an ARP or ND packet if
+ the host is configured to use static IP.
.. code-block:: console
@@ -543,7 +643,6 @@
# ndsend ${HOST_IP} ${INTF}
-
Reference
---------
- https://www.cisco.com/c/en/us/support/docs/security/adaptive-security-appliance-asa-software/116265-configure-product-00.html
diff --git a/configuration/dual-homing.rst b/configuration/dual-homing.rst
index 3e5f729..318dc02 100644
--- a/configuration/dual-homing.rst
+++ b/configuration/dual-homing.rst
@@ -8,31 +8,36 @@
The dual-homing feature includes several sub components
-- **Use of "paired" ToRs**: Each rack of compute nodes have exactly two Top-of-Rack switches (ToRs),
- that are linked to each other via a single link - such a link is referred to as a **pair link**.
- This pairing should NOT be omitted.
- Currently there is support for only a single link between paired ToRs.
- In future releases, we may include dual pair links.
- Note that the pair link is only used in failure scenarios, and not in normal operation.
+- **Use of "paired" ToRs**: Each rack of compute nodes have exactly two
+ Top-of-Rack switches (ToRs), that are linked to each other via a single link
+ - such a link is referred to as a **pair link**. This pairing should NOT be
+ omitted.
-- **Dual-homed servers (compute-nodes)**: Each server is connected to both ToRs.
- The links to the paired ToRs are (Linux) bonded
+ Currently there is support for only a single link between paired ToRs. In
+ future releases, we may include dual pair links. Note that the pair link is
+ only used in failure scenarios, and not in normal operation.
-- **Dual-homed upstream routers**: The upstream routers MUST be connected to the two ToRs that are part of a leaf-pair.
- You cannot connect them to leafs that are not paired. This feature also requires two Quagga instances.
+- **Dual-homed servers (compute-nodes)**: Each server is connected to both
+ ToRs. The links to the paired ToRs are (Linux) bonded
+
+- **Dual-homed upstream routers**: The upstream routers MUST be connected to
+ the two ToRs that are part of a leaf-pair. You cannot connect them to leafs
+ that are not paired. This feature also requires two Quagga instances.
- **Dual-homed access devices**. This component will be added in the future.
Paired ToRs
-----------
-The reasoning behind two ToR (leaf) switches is simple.
-If you only have a single ToR switch, and you lose it, the entire rack goes down.
-Using two ToR switches increases your odds for continued connectivity for dual homed servers.
-The reasoning behind pairing the two ToR switches is more involved, as is explained in the Usage section below.
+The reasoning behind two ToR (leaf) switches is simple. If you only have a
+single ToR switch, and you lose it, the entire rack goes down. Using two ToR
+switches increases your odds for continued connectivity for dual homed servers.
+The reasoning behind pairing the two ToR switches is more involved, as is
+explained in the Usage section below.
Configure pair ToRs
^^^^^^^^^^^^^^^^^^^
-Configuring paired-ToRs involves device configuration. Assume switches of:205 and of:206 are paired ToRs.
+Configuring paired-ToRs involves device configuration. Assume switches of:205
+and of:206 are paired ToRs.
.. code-block:: json
@@ -71,15 +76,21 @@
There are two new pieces of device configuration.
-Each device in the ToR pair needs to specify the **deviceId of the leaf it is paired to**, in the ``pairDeviceId`` field.
-For example, in of:205 configuration the pairDeviceid is specified as of:206, and similarly in of:206 configuration the pairDeviceId is of:205
-Each device in the ToR pair needs to specify the **port on the device used for the pair link** in the ``pairLocalPort`` field.
-For example, the pair link in the config above show that port 20 on of:205 is connected to port 30 on of:206.
+Each device in the ToR pair needs to specify the **deviceId of the leaf it is
+paired to**, in the ``pairDeviceId`` field. For example, in ``of:205``
+configuration the ``pairDeviceId`` is specified as ``of:206``, and similarly in ``of:206``
+configuration the ``pairDeviceId`` is ``of:205``. Each device in the ToR pair needs to
+specify the **port on the device used for the pair link** in the
+``pairLocalPort`` field. For example, the pair link in the config above show
+that port 20 on of:205 is connected to port 30 on of:206.
-In addition, there is one crucial piece of config that needs to **match for both ToRs** – the ``routerMac`` address.
-The paired-ToRs MUST have the same routerMac - in the example above, they both have identical 00:00:02:05:00:01 routerMacs.
+In addition, there is one crucial piece of config that needs to **match for
+both ToRs** – the ``routerMac`` address. The paired-ToRs MUST have the same
+routerMac - in the example above, they both have identical 00:00:02:05:00:01
+routerMacs.
-All other fields are the same as before, as explained in :doc:`Device Configuration <device-config>` section.
+All other fields are the same as before, as explained in :doc:`Device
+Configuration <device-config>` section.
Usage of pair link
@@ -90,19 +101,26 @@
Dual-Homed Servers
------------------
+
There are a number of things to note when connecting dual-homed servers to paired-ToRs.
-- The switch ports on the two ToRs have to be configured the same way, when connecting a dual-homed server to the two ToRs.
+- The switch ports on the two ToRs have to be configured the same way, when
+ connecting a dual-homed server to the two ToRs.
+
- The server ports have to be Linux-bonded in a particular mode.
Configure Switch Ports
^^^^^^^^^^^^^^^^^^^^^^
-The way to configure ports are similar as described in :doc:`Bridging and Unicast <bridging-unicast>`.
-However, there are a couple of things to note.
-**First**, dual-homed servers should have the **identical configuration on each switch port they connect to on the ToR pairs**.
-The example below shows that the ``vlans`` and ``ips`` configured are the same on both switch ports ``of:205/12`` and ``of:206/29``.
-They are both configured to be access ports in ``VLAN 20``, the subnet ``10.0.2.0/24`` is assigned to these ports, and the gateway-IP is ``10.0.2.254/32``.
+The way to configure ports are similar as described in :doc:`Bridging and
+Unicast <bridging-unicast>`. However, there are a couple of things to note.
+
+**First**, dual-homed servers should have the **identical configuration on each
+switch port they connect to on the ToR pairs**. The example below shows that
+the ``vlans`` and ``ips`` configured are the same on both switch ports
+``of:205/12`` and ``of:206/29``. They are both configured to be access ports
+in ``VLAN 20``, the subnet ``10.0.2.0/24`` is assigned to these ports, and the
+gateway-IP is ``10.0.2.254/32``.
.. code-block:: json
@@ -125,17 +143,24 @@
}
}
-It is worth noting the meaning behind the configuration above from a routing perspective.
-Simply put, by configuring the same subnets on these switch ports, the fabric now believes that the entire subnet ``10.0.2.0/24`` is reachable by BOTH ToR switches ``of:205`` and ``of:206``.
+It is worth noting the meaning behind the configuration above from a routing
+perspective. Simply put, by configuring the same subnets on these switch
+ports, the fabric now believes that the entire subnet ``10.0.2.0/24`` is
+reachable by BOTH ToR switches ``of:205`` and ``of:206``.
.. caution::
- Configuring different VLANs, or different subnets, or mismatches like "vlan-untagged" in one switch port and "vlan-tagged" in the corresponding switch port facing the dual-homed server, will result in incorrect behavior.
+ Configuring different VLANs, or different subnets, or mismatches like
+ "vlan-untagged" in one switch port and "vlan-tagged" in the corresponding
+ switch port facing the dual-homed server, will result in incorrect
+ behavior.
-**Second**, we need to configure the **pair link ports on both ToR switches to be trunk (vlan-tagged) ports that contains all dual-homed VLANs and subnets**.
-This is an extra piece of configuration, the need for which will be removed in future releases.
-In the example above, a dual-homed server connects to the ToR pair on port 12 on of:205 and port 29 on of:206.
-Assume that the pair link between the two ToRs is connected to port 5 of both of:205 and of:206.
-The config for these switch ports is shown below:
+**Second**, we need to configure the **pair link ports on both ToR switches to
+be trunk (vlan-tagged) ports that contains all dual-homed VLANs and subnets**.
+This is an extra piece of configuration, the need for which will be removed in
+future releases. In the example above, a dual-homed server connects to the ToR
+pair on port 12 on of:205 and port 29 on of:206. Assume that the pair link
+between the two ToRs is connected to port 5 of both of:205 and of:206. The
+config for these switch ports is shown below:
.. code-block:: json
@@ -159,13 +184,18 @@
}
.. note::
- Even though the ports ``of:205/12`` and ``of:206/`` facing the dual-homed server are configured as ``vlan-untagged``,
- the same vlan MUST be configured as ``vlan-tagged`` on the pair-ports.
- If additional subnets and VLANs are configured facing other dual-homed servers, they need to be similarly added to the ``ips`` and ``vlan-tagged`` arrays in the pair port config.
+ Even though the ports ``of:205/12`` and ``of:206/`` facing the dual-homed
+ server are configured as ``vlan-untagged``, the same vlan MUST be
+ configured as ``vlan-tagged`` on the pair-ports.
+
+ If additional subnets and VLANs are configured facing other dual-homed
+ servers, they need to be similarly added to the ``ips`` and ``vlan-tagged``
+ arrays in the pair port config.
Configure Servers
^^^^^^^^^^^^^^^^^
+
Assuming the interfaces we are going to use for bonding are ``eth1`` and ``eth2``.
- Bring down interfaces
@@ -235,19 +265,24 @@
.. caution::
**Dual-homed host should not be statically configured.**
- Currently in ONOS, configured hosts are not updated when the connectPoint is lost.
- This is not a problem with single-homed hosts because there is no other way to reach them anyway if their connectPoint goes down.
- But in dual-homed scenarios, the controller should take corrective action if one of the connectPoints go down –
- the trigger for this event does not happen when the dual-homed host's connect points are configured (not discovered).
+ Currently in ONOS, configured hosts are not updated when the connectPoint
+ is lost. This is not a problem with single-homed hosts because there is no
+ other way to reach them anyway if their connectPoint goes down. But in
+ dual-homed scenarios, the controller should take corrective action if one
+ of the connectPoints go down – the trigger for this event does not happen
+ when the dual-homed host's connect points are configured (not discovered).
.. note::
- We also support static routes with dual-homed next hop.
- The way to configure it is exactly the same as regular single-homed next hop, as described in :doc:`External Connectivity <external-connectivity>`.
+ We also support static routes with dual-homed next hop. The way to
+ configure it is exactly the same as regular single-homed next hop, as
+ described in :doc:`External Connectivity <external-connectivity>`.
- ONOS will automatically recognize when the next-hop IP resolves to a dual-homed host and program both switches (the host connects to) accordingly.
+ ONOS will automatically recognize when the next-hop IP resolves to a
+ dual-homed host and program both switches (the host connects to)
+ accordingly.
- The failure recovery mechanism for dual-homed hosts also applies to static routes that point to the host as their next hop.
-
+ The failure recovery mechanism for dual-homed hosts also applies to static
+ routes that point to the host as their next hop.
Dual External Routers
---------------------
@@ -257,22 +292,25 @@
.. image:: ../images/config-dh-vr-logical.png
:width: 200px
-In addition to what we describe in :doc:`External Connectivity <external-connectivity>`,
-Trellis also supports dual external routers, which view the Trellis fabric as 2 individual routers, as shown above.
+In addition to what we describe in :doc:`External Connectivity
+<external-connectivity>`, Trellis also supports dual external routers, which
+view the Trellis fabric as 2 individual routers, as shown above.
As before the vRouter control plane is implemented as a combination of Quagga,
-which peers with the upstream routers, and ONOS which listens to Quagga (via FPM) and programs the underlying fabric.
-**In dual-router scenarios, there are two instances of Quagga required**.
+which peers with the upstream routers, and ONOS which listens to Quagga (via
+FPM) and programs the underlying fabric. **In dual-router scenarios, there are
+two instances of Quagga required**.
-As before the hardware fabric serves as the data-plane of vRouter.
-In dual-router scenarios, the **external routers MUST be connected to paired-ToRs**.
-
+As before the hardware fabric serves as the data-plane of vRouter. In
+dual-router scenarios, the **external routers MUST be connected to
+paired-ToRs**.
ToR connects to one upstream
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Lets consider the simpler case where the external routers are each connected to a single leaf in a ToR pair.
-The figure on the left below shows the logical view. The figure on the right shows the physical connectivity.
+Lets consider the simpler case where the external routers are each connected to
+a single leaf in a ToR pair. The figure on the left below shows the logical
+view. The figure on the right shows the physical connectivity.
.. image:: ../images/config-dh-vr-logical-simple.png
:width: 200px
@@ -280,19 +318,22 @@
.. image:: ../images/config-dh-vr-physical-simple.png
:width: 400px
-One of the upstream routers is connected to ``of:205`` and the other is connected to ``of:206``.
-Note that ``of:205`` and ``of:206`` are paired ToRs.
+One of the upstream routers is connected to ``of:205`` and the other is
+connected to ``of:206``. Note that ``of:205`` and ``of:206`` are paired ToRs.
-The ToRs are connected via a physical port to separate Quagga VMs or containers.
-These Quagga instances can be placed in any compute node. They do not need to be in the same server, and are only shown to be co-located for simplicity.
+The ToRs are connected via a physical port to separate Quagga VMs or
+containers. These Quagga instances can be placed in any compute node. They do
+not need to be in the same server, and are only shown to be co-located for
+simplicity.
-The two Quagga instances do NOT talk to each other.
-
+The two Quagga instances do NOT talk to each other.
Switch port configuration
"""""""""""""""""""""""""
-The ToRs follow the same rules as single router case described in :doc:`External Connectivity <external-connectivity>`.
-In the example shown above, the switch port config would look like this:
+
+The ToRs follow the same rules as single router case described in
+:doc:`External Connectivity <external-connectivity>`. In the example shown
+above, the switch port config would look like this:
.. code-block:: json
@@ -333,35 +374,51 @@
}
.. note::
- In the example shown above, switch ``of:205`` uses ``VLAN 100`` for bridging the peering session between Quagga1 and ExtRouter1,
- while switch ``of:205`` uses ``VLAN 200`` to do the same for the other peering session.
- But since these vlans and bridging domains are defined on different switches, the VLAN ids could have been the same.
+ In the example shown above, switch ``of:205`` uses ``VLAN 100`` for
+ bridging the peering session between Quagga1 and ExtRouter1, while switch
+ ``of:205`` uses ``VLAN 200`` to do the same for the other peering session.
+ But since these vlans and bridging domains are defined on different
+ switches, the VLAN ids could have been the same.
- This philosophy is consistent with the fabric use of :doc:`bridging <bridging-unicast>`.
+ This philosophy is consistent with the fabric use of :doc:`bridging
+ <bridging-unicast>`.
Quagga configuration
""""""""""""""""""""
-Configuring Quagga for dual external routers are similar to what we described in :doc:`External Connectivity <external-connectivity>`. However, it is worth noting that:
+Configuring Quagga for dual external routers are similar to what we described
+in :doc:`External Connectivity <external-connectivity>`. However, it is worth
+noting that:
-- The two Zebra instances **should point to two different ONOS instances** for their FPM connections.
- For example Zebra in Quagga1 could point to ONOS instance with ``fpm connection ip 10.6.0.1 port 2620``,
- while the other Zebra should point to a different ONOS instance with ``fpm connection ip 10.6.0.2 port 2620``.
- It does not matter which ONOS instances they point to as long as they are different.
-- The two Quagga BGP sessions should appear to come from different routers but still use the same AS number –
- i.e. the two Quaggas' belong to the same AS, the one used to represent the entire Trellis infrastructure.
-- The two upstream routers can belong to the same or different AS,
- but these AS numbers should be different from the one used to represent the Trellis AS.
+- The two Zebra instances **should point to two different ONOS instances** for
+ their FPM connections. For example Zebra in Quagga1 could point to ONOS
+ instance with ``fpm connection ip 10.6.0.1 port 2620``, while the other Zebra
+ should point to a different ONOS instance with ``fpm connection ip 10.6.0.2
+ port 2620``. It does not matter which ONOS instances they point to as long
+ as they are different.
+
+- The two Quagga BGP sessions should appear to come from different routers but
+ still use the same AS number – i.e. the two Quaggas' belong to the same AS,
+ the one used to represent the entire Trellis infrastructure.
+
+- The two upstream routers can belong to the same or different AS, but these AS
+ numbers should be different from the one used to represent the Trellis AS.
+
- Typically both Quagga instances advertise the same routes to the upstream.
- These prefixes belonging to various infrastructure nodes in the deployment should be reachable from either of the leaf switches connected to the upstream routers.
-- The upstream routers may or may not advertise the same routes.
- Trellis will ensure that traffic directed to a route reachable only one upstream router is directed to the appropriate leaf.
+ These prefixes belonging to various infrastructure nodes in the deployment
+ should be reachable from either of the leaf switches connected to the
+ upstream routers.
+- The upstream routers may or may not advertise the same routes. Trellis will
+ ensure that traffic directed to a route reachable only one upstream router is
+ directed to the appropriate leaf.
ToR connects to both upstream
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Now lets consider the **more-complicated but more fault-tolerant** case of each Quagga instance peering with BOTH external routers.
-Again the logical view is shown on the left and the physical view on the right.
+
+Now lets consider the **more-complicated but more fault-tolerant** case of each
+Quagga instance peering with BOTH external routers. Again the logical view is
+shown on the left and the physical view on the right.
.. image:: ../images/config-dh-vr-logical.png
:width: 200px
@@ -379,19 +436,24 @@
- Quagga instance 2 peers with external router R1 via port 2 on switch of:206
- Quagga instance 2 peers with external router R2 via port 1 on switch of:206
-To distinguish between the two peering sessions in the same physical switch, say of:205,
-the physical ports 1 and 2 need to be configured in **different VLANs and subnets**.
-For example, port 1 on of:205 is (untagged) in VLAN 100, while port 2 is in VLAN 101.
+To distinguish between the two peering sessions in the same physical switch,
+say of:205, the physical ports 1 and 2 need to be configured in **different
+VLANs and subnets**. For example, port 1 on of:205 is (untagged) in VLAN 100,
+while port 2 is in VLAN 101.
-Note that peering for **Quagga1 and R1** happens with IPs in the ``10.0.100.0/29`` subnet,
-and for **Quagga 1 and R2** in the **10.0.101.0/29** subnet.
+Note that peering for **Quagga1 and R1** happens with IPs in the
+``10.0.100.0/29`` subnet, and for **Quagga 1 and R2** in the **10.0.101.0/29**
+subnet.
-Furthermore, **pair link** (port 48) on of:205 carries both peering sessions to Quagga1.
-Thus port 48 should now be configured as a **trunk port (vlan-tagged) with both VLANs and both subnets**.
+Furthermore, **pair link** (port 48) on of:205 carries both peering sessions to
+Quagga1. Thus port 48 should now be configured as a **trunk port (vlan-tagged)
+with both VLANs and both subnets**.
-Finally the **Quagga interface** on the VM now needs **sub-interface configuration for each VLAN ID**.
+Finally the **Quagga interface** on the VM now needs **sub-interface
+configuration for each VLAN ID**.
-Similar configuration concepts apply to IPv6 as well. Here is a look at the switch port config in ONOS for of:205
+Similar configuration concepts apply to IPv6 as well. Here is a look at the
+switch port config in ONOS for of:205
.. code-block:: json
diff --git a/configuration/external-connectivity.rst b/configuration/external-connectivity.rst
index 65fbcf9..abc3884 100644
--- a/configuration/external-connectivity.rst
+++ b/configuration/external-connectivity.rst
@@ -6,23 +6,34 @@
Physical Connectivity
^^^^^^^^^^^^^^^^^^^^^
-External routers must be physically connected to one of the fabric leaf switches.
-Currently there is a limitation that the **external/upstream router and the Quagga instance must be connected to the same fabric leaf switch**.
-Therefore it is necessary to use an additional front panel port on the leaf-switch (or at least an additional vlan) to connect to the compute node hosting Quagga.
+External routers must be physically connected to one of the fabric leaf
+switches.
+
+Currently there is a limitation that the **external/upstream router and the
+Quagga instance must be connected to the same fabric leaf switch**.
+
+Therefore it is necessary to use an additional front panel port on the
+leaf-switch (or at least an additional vlan) to connect to the compute node
+hosting Quagga.
.. image:: ../images/config-vr-physical.png
Configure vRouter
^^^^^^^^^^^^^^^^^
-The operator will need to configure a subnet between the Leaf-switch, the external/upstream router and the Quagga instance.
-There are 3 IP addresses we need to allocate - 1 on the switch port, 1 in Quagga, and 1 on the upstream router.
-This means the peering subnet **cannot be smaller than a /29**.
-BGP peering happens between the IP addresses configured on the interfaces in Quagga and the external router.
+The operator will need to configure a subnet between the Leaf-switch, the
+external/upstream router and the Quagga instance. There are 3 IP addresses we
+need to allocate - 1 on the switch port, 1 in Quagga, and 1 on the upstream
+router. This means the peering subnet **cannot be smaller than a /29**.
-Routes are advertised by Quagga to the upstream with the next-hop set to the switch port IP address.
-This means that when traffic comes to the fabric leaf switch from outside, the switch is able to distinguish peering traffic from data traffic and treat each appropriately.
+BGP peering happens between the IP addresses configured on the interfaces in
+Quagga and the external router.
+
+Routes are advertised by Quagga to the upstream with the next-hop set to the
+switch port IP address. This means that when traffic comes to the fabric leaf
+switch from outside, the switch is able to distinguish peering traffic from
+data traffic and treat each appropriately.
The following shows an ONOS interface configuration example:
@@ -52,66 +63,88 @@
}
- ``name``: An arbitrary name string for the interface. Optional.
-- ``ips``: Configure the peering subnet (10.0.1.0/24) and the switch port IP (10.0.1.2).
- Note that we use the same IP address on both the quagga and upstream interfaces.
-- ``vlan-untagged``: Configure the same VLAN ID on both interfaces.
- It doesn't matter exactly what the VLAN ID is, but it must be the same on both the Quagga-facing and upstream-facing interfaces.
+
+- ``ips``: Configure the peering subnet (10.0.1.0/24) and the switch port IP
+ (10.0.1.2). Note that we use the same IP address on both the quagga and
+ upstream interfaces.
+
+- ``vlan-untagged``: Configure the same VLAN ID on both interfaces. It doesn't
+ matter exactly what the VLAN ID is, but it must be the same on both the
+ Quagga-facing and upstream-facing interfaces.
In this case the peering subnet is ``10.0.1.0/24``.
The upstream router is using the ``10.0.1.1`` address.
Quagga is assigned ``10.0.1.3``, which is the address used for peering.
-The upstream router needs to be configured with ``10.0.1.3`` as its BGP neighbor, and the BGP peering will be established between ``10.0.1.1`` and ``10.0.1.3``. The ``10.0.1.2`` address is used by the fabric switch and for the next-hop for routes advertised by Quagga.
-Of course you are not obliged to use ``10.0.1.0/24``, you should use a subnet that makes sense for your peering environment.
+The upstream router needs to be configured with ``10.0.1.3`` as its BGP
+neighbor, and the BGP peering will be established between ``10.0.1.1`` and
+``10.0.1.3``. The ``10.0.1.2`` address is used by the fabric switch and for the
+next-hop for routes advertised by Quagga.
+
+Of course you are not obliged to use ``10.0.1.0/24``, you should use a subnet
+that makes sense for your peering environment.
.. note::
- This configuration will set up an L2 link between the two fabric switch ports, over which the Quagga and external router can communicate.
- Both Quagga and the upstream router will receive untagged packets
- (i.e they will never see packets with vlanId 4000, which is used inside the leaf switch to establish a bridging domain).
+ This configuration will set up an L2 link between the two fabric switch
+ ports, over which the Quagga and external router can communicate.
- If you need a vlan-tag in the compute node to distinguish the traffic going to Quagga, you can change the vlan assignment on the switch port "of:0000000000000001/2" to be vlan-tagged instead of vlan-untagged.
+ Both Quagga and the upstream router will receive untagged packets (i.e they
+ will never see packets with vlanId 4000, which is used inside the leaf
+ switch to establish a bridging domain).
+
+ If you need a vlan-tag in the compute node to distinguish the traffic going
+ to Quagga, you can change the vlan assignment on the switch port
+ "of:0000000000000001/2" to be vlan-tagged instead of vlan-untagged.
Deploy the Quagga Docker Image
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Trellis uses a slightly modified version of Quagga, so the easiest way to deploy this is to use the provided docker image.
+
+Trellis uses a slightly modified version of Quagga, so the easiest way to
+deploy this is to use the provided docker image.
.. code-block:: console
$ docker pull opencord/quagga
-We also need to download the **pipework** tool which will be used to connect the docker image to the physical interface that we set aside earlier.
+We also need to download the **pipework** tool which will be used to connect
+the docker image to the physical interface that we set aside earlier.
.. code-block:: console
$ wget https://raw.githubusercontent.com/jpetazzo/pipework/master/pipework
$ chmod +x pipework
-Create a directory for your Quagga configuration files, and create a bgpd.conf and zebra.conf in there.
-This folder is going to be mounted into the Quagga container.
-More on configuring Quagga later.
+Create a directory for your Quagga configuration files, and create a bgpd.conf
+and zebra.conf in there. This folder is going to be mounted into the Quagga
+container. More on configuring Quagga later.
.. code-block:: console
$ mkdir configs
$ touch zebra.conf bgpd.conf
-Now run the docker image (make sure the path the config directory matches what is on your system):
+Now run the docker image (make sure the path the config directory matches what
+is on your system):
.. code-block:: console
$ sudo docker run --privileged -d -v configs:/etc/quagga -n quagga opencord/quagga
-Finally, we can use the pipework tool to add the physical interface into the container so that Quagga can talk out over the fabric:
+Finally, we can use the pipework tool to add the physical interface into the
+container so that Quagga can talk out over the fabric:
.. code-block:: console
$ sudo ./pipework mlx1 -i eth1 quagga 10.0.1.3/24
-This will add host interface ``mlx1`` to the container with name ``quagga`` with interface name ``eth1`` inside the container.
-The newly added interface will have the IP ``10.0.1.3``.
-This IP address should be the peering subnet address that you want to assign to Quagga.
+This will add host interface ``mlx1`` to the container with name ``quagga``
+with interface name ``eth1`` inside the container. The newly added interface
+will have the IP ``10.0.1.3``. This IP address should be the peering subnet
+address that you want to assign to Quagga.
-If you need to change anything about the container (for example if you change the Quagga configuration) you can remove the original container and run a new one:
+If you need to change anything about the container (for example if you change
+the Quagga configuration) you can remove the original container and run a new
+one:
.. code-block:: console
@@ -120,19 +153,28 @@
Configure Quagga
^^^^^^^^^^^^^^^^
-At this point Quagga should have IP connectivity to the external routers, and it should be able to ping them on the peering subnet.
+
+At this point Quagga should have IP connectivity to the external routers, and
+it should be able to ping them on the peering subnet.
Now Quagga and the upstream routers can be configured to peer with one another.
-This configuration of Quagga is going to be highly dependent on the configuration of the upstream network, so it won't be possible to give comprehensive configuration examples here.
-It is recommended to consult the Quagga documentation for exhaustive information on Quagga's capabilities and configuration.
-Here I will attempt to provide a few basic examples of Quagga configuration to get you started.
-You'll have to enhance these with the features and functions that are needed in your network.
+This configuration of Quagga is going to be highly dependent on the
+configuration of the upstream network, so it won't be possible to give
+comprehensive configuration examples here.
+
+It is recommended to consult the Quagga documentation for exhaustive
+information on Quagga's capabilities and configuration. Here I will attempt to
+provide a few basic examples of Quagga configuration to get you started.
+You'll have to enhance these with the features and functions that are needed in
+your network.
Zebra configuration
"""""""""""""""""""
-Regardless of which routing protocols you are using in your network,
-it is important to configure Zebra's FPM connection to send routes to the FPM app running on ONOS.
-This feature was enabled by the patch that was applied earlier when we installed Quagga.
+
+Regardless of which routing protocols you are using in your network, it is
+important to configure Zebra's FPM connection to send routes to the FPM app
+running on ONOS. This feature was enabled by the patch that was applied
+earlier when we installed Quagga.
A minimal Zebra configuration might look like this:
@@ -145,12 +187,15 @@
fpm connection ip 10.6.0.1 port 2620
!
-The FPM connection IP address is the IP address of **one of the onos cluster instances** - does not matter which one.
-If you have other configuration that needs to go in zebra.conf you should add that here as well.
+The FPM connection IP address is the IP address of **one of the onos cluster
+instances** - does not matter which one. If you have other configuration that
+needs to go in zebra.conf you should add that here as well.
BGP configuration
"""""""""""""""""
-An example simple BGP configuration for peering with one BGP peer might look like this:
+
+An example simple BGP configuration for peering with one BGP peer might look
+like this:
.. code-block:: text
@@ -173,16 +218,22 @@
neighbor 10.0.1.1 route-map NEXTHOP out
!
-This configuration peers with one upstream router ``10.0.1.1`` and advertises one route ``192.168.0.0/16``.
-Note that Quagga (and as a result Trellis) is in a different AS ``65535`` from the upstream router AS ``65540``,
-as we are using E-BGP for this connectivity.
+This configuration peers with one upstream router ``10.0.1.1`` and advertises
+one route ``192.168.0.0/16``. Note that Quagga (and as a result Trellis) is in
+a different AS ``65535`` from the upstream router AS ``65540``, as we are using
+E-BGP for this connectivity.
.. note::
- Pay attention to the configuration to rewrite the next hop of routes that are advertised to the upstream router.
- A ``route-map`` is used to set the next hop of advertised routes to ``10.0.1.2``,
- which is **different from the address that Quagga is using to peer with the external router**.
- As mentioned above, it is important that this rewriting is done correctly so that the fabric switch is able to **distinguish data plane and control plane** traffic.
+ Pay attention to the configuration to rewrite the next hop of routes that
+ are advertised to the upstream router.
+ A ``route-map`` is used to set the next hop of advertised routes to
+ ``10.0.1.2``, which is **different from the address that Quagga is using to
+ peer with the external router**.
+
+ As mentioned above, it is important that this rewriting is done correctly
+ so that the fabric switch is able to **distinguish data plane and control
+ plane** traffic.
Route service and static route
------------------------------
@@ -192,8 +243,11 @@
View routes
"""""""""""
+
This will show routes from all sources, including static and dynamic routes.
-The example below shows routes learned from the upstream router (Source: FPM) and routes configured manually (Source: STATIC)
+
+The example below shows routes learned from the upstream router (Source: FPM)
+and routes configured manually (Source: STATIC)
.. code-block:: text
@@ -287,7 +341,8 @@
Verify routes
^^^^^^^^^^^^^
-Check the leaf switches that the route (e.g. 1.1.0.0/18) has been programmed in the routing table (table 30).
+Check the leaf switches that the route (e.g. 1.1.0.0/18) has been programmed in
+the routing table (table 30).
.. code-block:: console
@@ -299,9 +354,10 @@
Notes about next hops
^^^^^^^^^^^^^^^^^^^^^
-The next hop of a route should be resolvable to a MAC address that is known to ONOS.
-Typically the next hop is a server interface that is known to ONOS as a host learned via ARP or DHCP.
-If you are not sure, check the ``hosts`` command on the ONOS CLI.
+The next hop of a route should be resolvable to a MAC address that is known to
+ONOS. Typically the next hop is a server interface that is known to ONOS as a
+host learned via ARP or DHCP. If you are not sure, check the ``hosts`` command
+on the ONOS CLI.
.. code-block:: console
@@ -311,17 +367,24 @@
id=B2:A4:E2:72:D1:91/None, mac=B2:A4:E2:72:D1:91, location=of:0000000000000204/16, vlan=None, ip(s)=[10.0.1.20], configured=false
id=EE:22:F7:BE:86:50/None, mac=EE:22:F7:BE:86:50, location=of:0000000000000205/16, vlan=None, ip(s)=[10.0.2.15], configured=false
-If the next hop has not been resolved for any reason, it would be necessary to configure the next hop as a host (/32 prefix) together with MAC address and location.
-Learn more about how to configure a host using `Network Config Host Provider <https://wiki.onosproject.org/display/ONOS/Network+Config+Host+Provider>`_
+If the next hop has not been resolved for any reason, it would be necessary to
+configure the next hop as a host (/32 prefix) together with MAC address and
+location.
-Finally note that if you are configuring routes manually/statically and they are publicly routable IPs that should be reachable from “outside”, you would need to configure Quagga to advertise them upstream.
+Learn more about how to configure a host using `Network Config Host Provider
+<https://wiki.onosproject.org/display/ONOS/Network+Config+Host+Provider>`_
+
+Finally note that if you are configuring routes manually/statically and they
+are publicly routable IPs that should be reachable from “outside”, you would
+need to configure Quagga to advertise them upstream.
Route blackhole
---------------
-The blackhole consists of a rule on table 30 on every edge device on the fabric.
-The Table 30 rule matches on a given IP address and mask and has nothing but a clearDeferred action, practically dropping the packet.
-Every IP we want to blackhole will have it's own rule in every edge switch.
+The blackhole consists of a rule on table 30 on every edge device on the
+fabric. The Table 30 rule matches on a given IP address and mask and has
+nothing but a clearDeferred action, practically dropping the packet. Every IP
+we want to blackhole will have it's own rule in every edge switch.
An example of such rule is:
@@ -347,8 +410,10 @@
Ignore certain FPM peer
-----------------------
-The ``FpmConnectionInfo`` consists a new flag ``acceptRoutes``, indicating whether we want to accept or discard the routes advertised by certain FPM peer.
-Per current requirement, we always have the ``acceptRoutes`` flag set to ``true`` by default, meaning that we will accept routes from all peers.
+The ``FpmConnectionInfo`` consists a new flag ``acceptRoutes``, indicating
+whether we want to accept or discard the routes advertised by certain FPM peer.
+Per current requirement, we always have the ``acceptRoutes`` flag set to
+``true`` by default, meaning that we will accept routes from all peers.
We can updated the flag using REST API and CLI command as below
@@ -360,9 +425,6 @@
.. image:: ../images/config-fpm-rest.png
:width: 900px
-
-
-
CLI
^^^
diff --git a/configuration/multicast.rst b/configuration/multicast.rst
index a33fe3c..1d435b0 100644
--- a/configuration/multicast.rst
+++ b/configuration/multicast.rst
@@ -2,8 +2,14 @@
=========
Trellis supports IP multicast in an classic SDN way.
-**None of the common IP multicast protocols (e.g. IGMP, PIM) is used when calculating the multicast tree internally**.
-Instead, benefiting from SDN, Trellis calculates the multicast tree in a centralized way and program the switches accordingly. Having said that, it doesn't prevent us from using common IP multicast protocols to communicate with externals.
+
+**None of the common IP multicast protocols (e.g. IGMP, PIM) is used when
+calculating the multicast tree internally**.
+
+Instead, benefiting from SDN, Trellis calculates the multicast tree in a
+centralized way and program the switches accordingly. Having said that, it
+doesn't prevent us from using common IP multicast protocols to communicate with
+externals.
Here's a summary of multicast features Trellis supports.
@@ -16,19 +22,30 @@
:width: 1000px
.. note::
- Dual-homed sinks and multiple sources support comes with major changes in the Multicast Routing subsystem.
- **Sink and sources are now identified by an ONOS HostId (MAC/VLAN)** and has associated a number of ConnectPoint while in the past we identified the sinks and the sources just using one ConnectPoint.
- Multicast subsystem listen for Host events and according to them modify properly the ConnectPoint associated to the HostId.
- In particular, it leverages the Host Location which is a collection of ONOS ConnectPoint where the Host is attached.
- This information is maintained updated by ONOS thanks to several discovery mechanisms implemented in the Host subsystem.
- Following image try to summarize the work flow of the Multicast Routing subsystem after the aforementioned changes:
+ Dual-homed sinks and multiple sources support comes with major changes in
+ the Multicast Routing subsystem.
+
+ **Sink and sources are now identified by an ONOS HostId (MAC/VLAN)** and
+ has associated a number of ConnectPoint while in the past we identified the
+ sinks and the sources just using one ConnectPoint.
+
+ Multicast subsystem listen for Host events and according to them modify
+ properly the ConnectPoint associated to the HostId.
+
+ In particular, it leverages the Host Location which is a collection of ONOS
+ ConnectPoint where the Host is attached. This information is maintained
+ updated by ONOS thanks to several discovery mechanisms implemented in the
+ Host subsystem. Following image try to summarize the work flow of the
+ Multicast Routing subsystem after the aforementioned changes:
.. image:: ../images/config-mcast-internal.png
:width: 600px
:align: center
.. caution::
- We should avoid using reserved Multicast groups. Please check here for the reserved values: http://www.iana.org/assignments/multicast-addresses/multicast-addresses.xhtml
+ We should avoid using reserved Multicast groups. Please check here for the
+ reserved values:
+ http://www.iana.org/assignments/multicast-addresses/multicast-addresses.xhtml
Activate Multicast
@@ -39,7 +56,8 @@
onos> app activate mcast
-You can also add ``mcast`` into ``$ONOS_APPS`` environment variable such that it gets started automatically every time.
+You can also add ``mcast`` into ``$ONOS_APPS`` environment variable such that
+it gets started automatically every time.
CLI commands
@@ -47,25 +65,35 @@
Creating a route and adding sinks
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-In order to add a new Multicast route, we have to use ``mcast-host-join`` command and we need to provide following information:
+In order to add a new Multicast route, we have to use ``mcast-host-join``
+command and we need to provide following information:
+
1) **source address**; 2) **group address**; 3) **sources** ; 4) **sinks**.
+
Here's an example:
.. code-block:: console
onos> mcast-host-join -sAddr * -gAddr 224.0.0.1 -srcs 00:AA:00:00:00:01/None -srcs 00:AA:00:00:00:05/None -sinks 00:AA:00:00:00:03/None -sinks 00:CC:00:00:00:01/None
-- ``-sAddr``: we can provide **\* for ASM** or an **IP address for SSM**.
- At the time being we do not enforce the matching of the -sAddr on the data-plane;
-- ``-gAddr`` specifies Multicast group address
-- ``-srcs`` identifies the source we changed from ConnectPoint to HostId in this release.
-- ``-sinks`` specifies the sinks of the group, we changed from ConnectPoint to HostId in this release.
+- ``-sAddr``: we can provide **\* for ASM** or an **IP address for SSM**
+ At this time being we do not enforce the matching of the ``-sAddr`` on the
+ data plane.
+- ``-gAddr`` specifies Multicast group address
+
+- ``-srcs`` identifies the source we changed from ConnectPoint to HostId in
+ this release.
+
+- ``-sinks`` specifies the sinks of the group, we changed from ConnectPoint to
+ HostId in this release.
Removing a sink
^^^^^^^^^^^^^^^
-A sink can be removed through ``mcast-sink-delete`` command.
-The major difference is that the sinks to be removed is provided through ``-h`` option.
+
+A sink can be removed through ``mcast-sink-delete`` command. The major
+difference is that the sinks to be removed is provided through ``-h`` option.
+
If ``-h`` is not provided the command will issue a route removal.
.. code-block:: console
@@ -75,9 +103,11 @@
Modifying a source
^^^^^^^^^^^^^^^^^^
-We also provide a command to modify the sources of a Multicast group.
-In particular ``mcast-source-delete`` allows to modify the source connect points of a source.
-If ``-src`` is not provided, the command will issue a route removal
+
+We also provide a command to modify the sources of a Multicast group. In
+particular ``mcast-source-delete`` allows to modify the source connect points
+of a source. If ``-src`` is not provided, the command will issue a route
+removal
.. code-block:: console
@@ -109,14 +139,20 @@
REST APIs
---------
+
You can find the REST API documentation at ``http://<ONOS-IP>:8181/onos/v1/docs``.
+
Please click the drop down list and select Multicast API.
.. image:: ../images/config-mcast-rest.png
We configure and implement multicast routes through the dedicated REST API.
-Some of the APIs are outlined here, you can find complete list in the ONOS docs page.
-The referenced files in these examples can be found in ``$ONOS_ROOT/apps/mcast/web/src/main/resources/jsonExamples``
+
+Some of the APIs are outlined here, you can find complete list in the ONOS docs
+page.
+
+The referenced files in these examples can be found in
+``$ONOS_ROOT/apps/mcast/web/src/main/resources/jsonExamples``
GET operations
^^^^^^^^^^^^^^
@@ -246,8 +282,14 @@
}
- ``group``: IP address of the Multicast Group
-- ``source``: IP address of the Multicast source. If specified (e.g. 10.0.1.1) we will treat this route as SSM (Single Source Multicast), if instead * is used the route will be ASM (Any Source Multicast)
-- ``sources``: Array containing the connect points to which the source is connected to.
-- ``sinks``: Array containing a set of Hosts that we want as sinks of the multicast stream. ONOS will automatically handle the connect points of these hosts if they are single or dual homed.
+- ``source``: IP address of the Multicast source. If specified (e.g. 10.0.1.1)
+ we will treat this route as SSM (Single Source Multicast), if instead * is
+ used the route will be ASM (Any Source Multicast)
+- ``sources``: Array containing the connect points to which the source is
+ connected to.
+
+- ``sinks``: Array containing a set of Hosts that we want as sinks of the
+ multicast stream. ONOS will automatically handle the connect points of these
+ hosts if they are single or dual homed.
diff --git a/configuration/pseudowire.rst b/configuration/pseudowire.rst
index 60d1ca0..09b3917 100644
--- a/configuration/pseudowire.rst
+++ b/configuration/pseudowire.rst
@@ -1,19 +1,28 @@
Pseudowire
==========
-In Trellis, we assign different subnets to hosts that attach to different edge (leaf) switches.
-These hosts communicate over IP when traversing the spine switches, meaning that the layer-2 header is not preserved across transportation.
-However, in some use cases it is beneficial for hosts that connect to different leaves to be able to talk directly over L2 and also belong to the same IP subnet.
+In Trellis, we assign different subnets to hosts that attach to different edge
+(leaf) switches. These hosts communicate over IP when traversing the spine
+switches, meaning that the layer-2 header is not preserved across
+transportation. However, in some use cases it is beneficial for hosts that
+connect to different leaves to be able to talk directly over L2 and also belong
+to the same IP subnet.
+
We implemented Pseudowire (PW) in Trellis to fulfill this requirement.
-Pseudowires create tunnels between different connection points that transport L2 traffic between them.
-Pseudowires can handle single-tagged/double-tagged traffic and are able to modify the VLAN tags in some cases.
+Pseudowires create tunnels between different connection points that transport
+L2 traffic between them.
+
+Pseudowires can handle single-tagged/double-tagged traffic and are able to
+modify the VLAN tags in some cases.
.. note::
Pseudowires are only supported on OF-DPA hardware switches.
Managing pseudowires via REST
-----------------------------
+
You can find the REST API documentation at ``http://${ONOS-IP}:8181/onos/v1/docs``.
+
Please click the drop down list and select Segment Routing Server.
.. image:: ../images/config-pw-rest.png
@@ -27,9 +36,9 @@
$ curl --user onos:rocks -X GET -H 'Content-Type:application/json' http://${ONOS-IP}:8181/onos/segmentrouting/pseudowire/ -d@pseudowire.json
$ curl --user onos:rocks -X DELETE -H 'Content-Type:application/json' http://${ONOS-IP}:8181/onos/segmentrouting/pseudowire/ -d@pseudowire.json
-Below is an example configuration for a single pseudowire,
-which transfers L2 double tagged traffic from cp1 to cp2 and also changes the outer VLAN tag of the packets.
-
+Below is an example configuration for a single pseudowire, which transfers L2
+double tagged traffic from cp1 to cp2 and also changes the outer VLAN tag of
+the packets.
.. code-block:: json
@@ -47,13 +56,21 @@
}
- ``cP1``: First connection point of PW
+
- ``cP2``: Second connection point of PW
-- ``cP1InnerTag / cp1OuterTag``: The expected inner and outer VLAN tags for ``cp1``. If you only want to use a single incoming tag,
- use the innerTag, and use ``None`` for the outerTag.
+
+- ``cP1InnerTag / cp1OuterTag``: The expected inner and outer VLAN tags for
+ ``cp1``. If you only want to use a single incoming tag, use the innerTag, and
+ use ``None`` for the outerTag.
+
- ``cP2InnerTag / cp2OuterTag``: The expected inner and outer VLAN tags for ``cp2``.
+
- ``mode``: Pseudowire mode - RAW and TAGGED. Only RAW mode is supported in Trellis.
+
- ``sdTag``: Service delimiting tag, only meaningful when in TAGGED mode.
+
- ``pwLabel``: Unique pseudowire label.
+
- ``pwId``: ID of the pseudowire.
The REST API also supports bulk addition and removal of pseudowire.
@@ -131,50 +148,76 @@
Type of pseudowires
-------------------
+
- **Leaf - Leaf** : As explained above we support leaf - leaf pseudowires.
-- **Spine - Leaf** : We further support leaf to spine pseudowires,
- where one termination point resides in a spine switch rather than a leaf switch.
- Please note that the encapsulated traffic for these types of pseudowires is carried **tagged** between leaf and spines,
- the reason for this is due to some hardware limitation. The **reserved transport VLAN is 4093**.
+- **Spine - Leaf** : We further support leaf to spine pseudowires, where one
+ termination point resides in a spine switch rather than a leaf switch.
+ Please note that the encapsulated traffic for these types of pseudowires is
+ carried **tagged** between leaf and spines, the reason for this is due to
+ some hardware limitation. The **reserved transport VLAN is 4093**.
- Multi-stage
- - **Leaf - Spine-1 - Spine-2**: Where Spine-1 and Spine-2 are interconnected between them and are part of distinct topologies.
- - **Leaf_1 - Spine_1 - Spine_2 - Leaf_2**: Where leaf-1 and leaf-2 are part of different fabric topologies which are interconnected between them between two spines.
+ - **Leaf - Spine-1 - Spine-2**: Where Spine-1 and Spine-2 are interconnected
+ between them and are part of distinct topologies.
+
+ - **Leaf_1 - Spine_1 - Spine_2 - Leaf_2**: Where leaf-1 and leaf-2 are part
+ of different fabric topologies which are interconnected between them
+ between two spines.
Common errors
-------------
-If your pseudowires do not work make sure you have done the following in the host side :
-- Disabled VLAN offloading in the VM interfaces and also in the compute node interfaces.
+If your pseudowires do not work make sure you have done the following in the
+host side:
+
+- Disabled VLAN offloading in the VM interfaces and also in the compute node
+ interfaces.
+
- Set the compute node interfaces and the VM interfaces at promiscuous mode.
-- In general, do the above for all the interfaces and bridges participated in the topology (either in a VM or physical server).
-- Before programming pseudowires make sure that the topology is completely up and running and also the configuration is added to the controller and the fabric is
- in an operational state.
+
+- In general, do the above for all the interfaces and bridges participated in
+ the topology (either in a VM or physical server).
+
+- Before programming pseudowires make sure that the topology is completely up
+ and running and also the configuration is added to the controller and the
+ fabric is in an operational state.
Restrictions
------------
-We list now some restrictions of pseudowire support, some of them stem from the implementation while others from switch hardware constraints.
+
+We list now some restrictions of pseudowire support, some of them stem from the
+implementation while others from switch hardware constraints.
- We only support pseudowires with the same type of tagged traffic at each end.
- For example, if cP1 handles single tagged traffic then cP2 must handle single tagged traffic (same applies for single tagged and double tagged traffic).
- However, for pseudowires with single tagged traffic we support changing the tag,
- and for pseudowires with double tagged traffic we support changing the outer tag.
- We would like to support more use cases, however we are limited by hardware pipeline restrictions.
- In the future, and if needed, we might address this by using the egress tables of OF-DPA.
-- As of now, we only support RAW mode. We will support also TAGGED mode in the future if needed by any use case.
- Since, we only support RAW mode the sdTag must always be empty or None.
+ For example, if cP1 handles single tagged traffic then cP2 must handle single
+ tagged traffic (same applies for single tagged and double tagged traffic).
+ However, for pseudowires with single tagged traffic we support changing the
+ tag, and for pseudowires with double tagged traffic we support changing the
+ outer tag. We would like to support more use cases, however we are limited
+ by hardware pipeline restrictions. In the future, and if needed, we might
+ address this by using the egress tables of OF-DPA.
+- As of now, we only support RAW mode. We will support also TAGGED mode in the
+ future if needed by any use case. Since, we only support RAW mode the
+ ``sdTag`` must always be empty or None.
Implementation limits
---------------------
+
- **Link failures**: We partially support link failures.
- Specifically, our PW implementation does not properly handle link failures when the link involved is the one used for pseudowire traffic at the spine switches towards a leaf switch.
- This is due to an OF-DPA limitation which does not allow the use of MPLS ECMP groups in the spine.
+ Specifically, our PW implementation does not properly handle link failures
+ when the link involved is the one used for pseudowire traffic at the spine
+ switches towards a leaf switch.
+
+ This is due to an OF-DPA limitation which does not allow the use of MPLS ECMP
+ groups in the spine.
+
We plan to address this in the next release.
-- **Device failures**: We do not handle device failures currently, on a device failure please remove and re-install the pseudowire.
- We will address this soon.
+- **Device failures**: We do not handle device failures currently, on a device
+ failure please remove and re-install the pseudowire. We will address this
+ soon.
diff --git a/configuration/xconnect.rst b/configuration/xconnect.rst
index 6486d96..a33d2b2 100644
--- a/configuration/xconnect.rst
+++ b/configuration/xconnect.rst
@@ -1,11 +1,16 @@
VLAN Cross Connect
==================
-VLAN cross connect creates a L2 bridge between two given ports on the same device.
-Once configured, every packets arriving at one of the port with specific VLAN tag will be sent to another port directly.
-Current implementation only matches on the outermost VLAN tag. If the packet is double tagged, the S-tag will be matched and both the S-tag and C-tag will be persisted.
-VLAN cross connect only works on the same device.
-If you are looking for transporting L2 traffic across devices, please refer to :doc:`pseoduwire <pseudowire>`
+VLAN cross connect creates a L2 bridge between two given ports on the same
+device. Once configured, every packets arriving at one of the port with
+specific VLAN tag will be sent to another port directly. Current
+implementation only matches on the outermost VLAN tag. If the packet is double
+tagged, the S-tag will be matched and both the S-tag and C-tag will be
+persisted.
+
+VLAN cross connect only works on the same device. If you are looking for
+transporting L2 traffic across devices, please refer to :doc:`pseoduwire
+<pseudowire>`
View cross connect via CLI
--------------------------
@@ -70,4 +75,4 @@
}' 'http://${ONOS-IP}:8181/onos/segmentrouting/xconnect'
- ``deviceId``: device ID, e.g. **of:0000000000000201**
-- ``vlanId``: VLAN ID, e.g. **94**
\ No newline at end of file
+- ``vlanId``: VLAN ID, e.g. **94**