Pseudowire
==========
In Trellis, we assign different subnets to hosts that attach to different edge
(leaf) switches.  These hosts communicate over IP when traversing the spine
switches, meaning that the layer-2 header is not preserved across
transportation.  However, in some use cases it is beneficial for hosts that
connect to different leaves to be able to talk directly over L2 and also belong
to the same IP subnet.

We implemented Pseudowire (PW) in Trellis to fulfill this requirement.

Pseudowires create tunnels between different connection points that transport
L2 traffic between them.

Pseudowires can handle single-tagged/double-tagged traffic and are able to
modify the VLAN tags in some cases.

.. note::
  Pseudowires are only supported on OF-DPA hardware switches.

Managing pseudowires via REST
-----------------------------

You can find the REST API documentation at ``http://${ONOS-IP}:8181/onos/v1/docs``.

Please click the drop down list and select Segment Routing Server.

.. image:: ../images/config-pw-rest.png
    :width: 700px

We configure and implement PWs through the dedicated REST API.

.. code-block:: console

    $ curl --user onos:rocks -X POST -H 'Content-Type:application/json' http://${ONOS-IP}:8181/onos/segmentrouting/pseudowire/ -d@pseudowire.json
    $ curl --user onos:rocks -X GET -H 'Content-Type:application/json' http://${ONOS-IP}:8181/onos/segmentrouting/pseudowire/ -d@pseudowire.json
    $ curl --user onos:rocks -X DELETE -H 'Content-Type:application/json' http://${ONOS-IP}:8181/onos/segmentrouting/pseudowire/ -d@pseudowire.json

Below is an example configuration for a single pseudowire, which transfers L2
double tagged traffic from cp1 to cp2 and also changes the outer VLAN tag of
the packets.

.. code-block:: json

    {
        "pwId" : 42,
        "cP1": "of:0000000000000205/25",
        "cP2": "of:0000000000000206/33",
        "cP1InnerTag": "101",
        "cP1OuterTag": "110",
        "cP2InnerTag": "101",
        "cP2OuterTag": "111",
        "mode": "RAW",
        "sdTag": "",
        "pwLabel": "1572"
    }

- ``cP1``: First connection point of PW

- ``cP2``: Second connection point of PW

- ``cP1InnerTag / cp1OuterTag``: The expected inner and outer VLAN tags for
  ``cp1``. If you only want to use a single incoming tag, use the innerTag, and
  use ``None`` for the outerTag.

- ``cP2InnerTag / cp2OuterTag``: The expected inner and outer VLAN tags for ``cp2``.

- ``mode``: Pseudowire mode - RAW and TAGGED. Only RAW mode is supported in Trellis.

- ``sdTag``: Service delimiting tag, only meaningful when in TAGGED mode.

- ``pwLabel``: Unique pseudowire label.

- ``pwId``: ID of the pseudowire.

The REST API also supports bulk addition and removal of pseudowire.

.. code-block:: console

    $ curl --user onos:rocks -X POST -H 'Content-Type:application/json'http://<controller-ip>:8181/onos/segmentrouting/pseudowire/bulk -d@pseudowires.json
    $ curl --user onos:rocks -X DELETE -H 'Content-Type:application/json'http://<controller-ip>:8181/onos/segmentrouting/pseudowire/bulk -d@pseudowires.json

with the following example configuration:

.. code-block:: json

    {
      "pseudowires": [{
          "pwId": 43,
          "cP1": "of:0000000000000227/15",
          "cP2": "of:0000000000000205/33",
          "cP1InnerTag": "101",
          "cP1OuterTag": "",
          "cP2InnerTag": "101",
          "cP2OuterTag": "",
          "mode": "RAW",
          "sdTag": "",
          "pwLabel": "2572"
        },
        {
          "pwId": 42,
          "cP1": "of:0000000000000201/12",
          "cP2": "of:0000000000000204/12",
          "cP1InnerTag": "101",
          "cP1OuterTag": "",
          "cP2InnerTag": "101",
          "cP2OuterTag": "",
          "mode": "RAW",
          "sdTag": "",
          "pwLabel": "1572"
        }
      ]
    }


Managing pseudowires via CLI
----------------------------

To add a pseudowire:

.. code-block:: text

    onos> sr-pw-add --help
    DESCRIPTION
            onos:sr-pw-add

        Add a pseudowire to the network configuration, if it already exists update it.

    SYNTAX
            onos:sr-pw-add [options] pwId pwLabel mode sDTag cP1 cP1InnerVlan cP1OuterVlan cP2 cP2InnerVlan cP2OuterVlan

    <omitted>

    onos> sr-pw-add 42 1000 RAW None of:0000000000000204/12 100 10 of:0000000000000205/12 100 40

To list existing pseudowires:

.. code-block:: text

    onos> sr-pw-list

To remove a pseudowire:

.. code-block:: text

    onos> sr-pw-remove 42


Type of pseudowires
-------------------

- **Leaf - Leaf** : As explained above we support leaf - leaf pseudowires.

- **Spine - Leaf** : We further support leaf to spine pseudowires, where one
  termination point resides in a spine switch rather than a leaf switch.
  Please note that the encapsulated traffic for these types of pseudowires is
  carried **tagged** between leaf and spines, the reason for this is due to
  some hardware limitation. The **reserved transport VLAN is 4093**.

- Multi-stage

  - **Leaf - Spine-1 - Spine-2**: Where Spine-1 and Spine-2 are interconnected
    between them and are part of distinct topologies.

  - **Leaf_1 - Spine_1 - Spine_2 - Leaf_2**: Where leaf-1 and leaf-2 are part
    of different fabric topologies which are interconnected between them
    between two spines.


Common errors
-------------

If your pseudowires do not work make sure you have done the following in the
host side:

- Disabled VLAN offloading in the VM interfaces and also in the compute node
  interfaces.

- Set the compute node interfaces and the VM interfaces at promiscuous mode.

- In general, do the above for all the interfaces and bridges participated in
  the topology (either in a VM or physical server).

- Before programming pseudowires make sure that the topology is completely up
  and running and also the configuration is added to the controller and the
  fabric is in an operational state.


Restrictions
------------

We list now some restrictions of pseudowire support, some of them stem from the
implementation while others from switch hardware constraints.

- We only support pseudowires with the same type of tagged traffic at each end.
  For example, if cP1 handles single tagged traffic then cP2 must handle single
  tagged traffic (same applies for single tagged and double tagged traffic).
  However, for pseudowires with single tagged traffic we support changing the
  tag, and for pseudowires with double tagged traffic we support changing the
  outer tag.  We would like to support more use cases, however we are limited
  by hardware pipeline restrictions.  In the future, and if needed, we might
  address this by using the egress tables of OF-DPA.

- As of now, we only support RAW mode. We will support also TAGGED mode in the
  future if needed by any use case.  Since, we only support RAW mode the
  ``sdTag`` must always be empty or None.

Implementation limits
---------------------

- **Link failures**: We partially support link failures.
  Specifically, our PW implementation does not properly handle link failures
  when the link involved is the one used for pseudowire traffic at the spine
  switches towards a leaf switch.

  This is due to an OF-DPA limitation which does not allow the use of MPLS ECMP
  groups in the spine.

  We plan to address this in the next release.

- **Device failures**: We do not handle device failures currently, on a device
  failure please remove and re-install the pseudowire.  We will address this
  soon.