Pseudowire
==========
In Trellis, we assign different subnets to hosts that attach to different edge (leaf) switches.
These hosts communicate over IP when traversing the spine switches, meaning that the layer-2 header is not preserved across transportation.
However, in some use cases it is beneficial for hosts that connect to different leaves to be able to talk directly over L2 and also belong to the same IP subnet.
We implemented Pseudowire (PW) in Trellis to fulfill this requirement.

Pseudowires create tunnels between different connection points that transport L2 traffic between them.
Pseudowires can handle single-tagged/double-tagged traffic and are able to modify the VLAN tags in some cases.

.. note::
  Pseudowires are only supported on OF-DPA hardware switches.

Managing pseudowires via REST
-----------------------------
You can find the REST API documentation at ``http://${ONOS-IP}:8181/onos/v1/docs``.
Please click the drop down list and select Segment Routing Server.

.. image:: ../images/config-pw-rest.png
    :width: 700px

We configure and implement PWs through the dedicated REST API.

.. code-block:: console

    $ curl --user onos:rocks -X POST -H 'Content-Type:application/json' http://${ONOS-IP}:8181/onos/segmentrouting/pseudowire/ -d@pseudowire.json
    $ curl --user onos:rocks -X GET -H 'Content-Type:application/json' http://${ONOS-IP}:8181/onos/segmentrouting/pseudowire/ -d@pseudowire.json
    $ curl --user onos:rocks -X DELETE -H 'Content-Type:application/json' http://${ONOS-IP}:8181/onos/segmentrouting/pseudowire/ -d@pseudowire.json

Below is an example configuration for a single pseudowire,
which transfers L2 double tagged traffic from cp1 to cp2 and also changes the outer VLAN tag of the packets.


.. code-block:: json

    {
        "pwId" : 42,
        "cP1": "of:0000000000000205/25",
        "cP2": "of:0000000000000206/33",
        "cP1InnerTag": "101",
        "cP1OuterTag": "110",
        "cP2InnerTag": "101",
        "cP2OuterTag": "111",
        "mode": "RAW",
        "sdTag": "",
        "pwLabel": "1572"
    }

- ``cP1``: First connection point of PW
- ``cP2``: Second connection point of PW
- ``cP1InnerTag / cp1OuterTag``: The expected inner and outer VLAN tags for ``cp1``. If you only want to use a single incoming tag,
  use the innerTag, and use ``None`` for the outerTag.
- ``cP2InnerTag / cp2OuterTag``: The expected inner and outer VLAN tags for ``cp2``.
- ``mode``: Pseudowire mode - RAW and TAGGED. Only RAW mode is supported in Trellis.
- ``sdTag``: Service delimiting tag, only meaningful when in TAGGED mode.
- ``pwLabel``: Unique pseudowire label.
- ``pwId``: ID of the pseudowire.

The REST API also supports bulk addition and removal of pseudowire.

.. code-block:: console

    $ curl --user onos:rocks -X POST -H 'Content-Type:application/json'http://<controller-ip>:8181/onos/segmentrouting/pseudowire/bulk -d@pseudowires.json
    $ curl --user onos:rocks -X DELETE -H 'Content-Type:application/json'http://<controller-ip>:8181/onos/segmentrouting/pseudowire/bulk -d@pseudowires.json

with the following example configuration:

.. code-block:: json

    {
      "pseudowires": [{
          "pwId": 43,
          "cP1": "of:0000000000000227/15",
          "cP2": "of:0000000000000205/33",
          "cP1InnerTag": "101",
          "cP1OuterTag": "",
          "cP2InnerTag": "101",
          "cP2OuterTag": "",
          "mode": "RAW",
          "sdTag": "",
          "pwLabel": "2572"
        },
        {
          "pwId": 42,
          "cP1": "of:0000000000000201/12",
          "cP2": "of:0000000000000204/12",
          "cP1InnerTag": "101",
          "cP1OuterTag": "",
          "cP2InnerTag": "101",
          "cP2OuterTag": "",
          "mode": "RAW",
          "sdTag": "",
          "pwLabel": "1572"
        }
      ]
    }


Managing pseudowires via CLI
----------------------------

To add a pseudowire:

.. code-block:: text

    onos> sr-pw-add --help
    DESCRIPTION
            onos:sr-pw-add

        Add a pseudowire to the network configuration, if it already exists update it.

    SYNTAX
            onos:sr-pw-add [options] pwId pwLabel mode sDTag cP1 cP1InnerVlan cP1OuterVlan cP2 cP2InnerVlan cP2OuterVlan

    <omitted>

    onos> sr-pw-add 42 1000 RAW None of:0000000000000204/12 100 10 of:0000000000000205/12 100 40

To list existing pseudowires:

.. code-block:: text

    onos> sr-pw-list

To remove a pseudowire:

.. code-block:: text

    onos> sr-pw-remove 42


Type of pseudowires
-------------------
- **Leaf - Leaf** : As explained above we support leaf - leaf pseudowires.

- **Spine - Leaf** : We further support leaf to spine pseudowires,
  where one termination point resides in a spine switch rather than a leaf switch.
  Please note that the encapsulated traffic for these types of pseudowires is carried **tagged** between leaf and spines,
  the reason for this is due to some hardware limitation. The **reserved transport VLAN is 4093**.

- Multi-stage

  - **Leaf - Spine-1 - Spine-2**: Where Spine-1 and Spine-2 are interconnected between them and are part of distinct topologies.
  - **Leaf_1 - Spine_1 - Spine_2 - Leaf_2**: Where leaf-1 and leaf-2 are part of different fabric topologies which are interconnected between them between two spines.


Common errors
-------------
If your pseudowires do not work make sure you have done the following in the host side :

- Disabled VLAN offloading in the VM interfaces and also in the compute node interfaces.
- Set the compute node interfaces and the VM interfaces at promiscuous mode.
- In general, do the above for all the interfaces and bridges participated in the topology (either in a VM or physical server).
- Before programming pseudowires make sure that the topology is completely up and running and also the configuration is added to the controller and the fabric is
  in an operational state.


Restrictions
------------
We list now some restrictions of pseudowire support, some of them stem from the implementation while others from switch hardware constraints.

- We only support pseudowires with the same type of tagged traffic at each end.
  For example, if cP1 handles single tagged traffic then cP2 must handle single tagged traffic (same applies for single tagged and double tagged traffic).
  However, for pseudowires with single tagged traffic we support changing the tag,
  and for pseudowires with double tagged traffic we support changing the outer tag.
  We would like to support more use cases, however we are limited by hardware pipeline restrictions.
  In the future, and if needed, we might address this by using the egress tables of OF-DPA.
- As of now, we only support RAW mode. We will support also TAGGED mode in the future if needed by any use case.
  Since, we only support RAW mode the sdTag must always be empty or None.


Implementation limits
---------------------
- **Link failures**: We partially support link failures.
  Specifically, our PW implementation does not properly handle link failures when the link involved is the one used for pseudowire traffic at the spine switches towards a leaf switch.
  This is due to an OF-DPA limitation which does not allow the use of MPLS ECMP groups in the spine.
  We plan to address this in the next release.

- **Device failures**: We do not handle device failures currently, on a device failure please remove and re-install the pseudowire.
  We will address this soon.