Add details about SRIOV enablement to the networking guide

Neutron and Nova methods are included.

Addressed comments

Change-Id: Ic3d30bc76a298185fe978a470c64e805dd19a162

doc/source/reference/networking-guide.rst

@@ -271,3 +271,212 @@ prior to upgrading.
 
 On ubuntu network manager is required for tunnel networking.
 This requirement will be removed in the future.
+
+
+Neutron SRIOV
+=============
+
+Preparation and deployment
+--------------------------
+
+SRIOV requires specific NIC and BIOS configuration and is not supported on all
+platforms.  Consult NIC and platform specific documentation for instructions
+on enablement.
+
+Modify the configuration file ``/etc/kolla/globals.yml``:
+
+::
+
+    enable_neutron_sriov: "yes"
+
+Modify the file ``/etc/kolla/config/neutron/ml2_conf.ini``.  Add ``sriovnicswitch``
+to the mechanism drivers and add the provider networks for use by SRIOV.  Both
+flat and VLAN are configured with the same physical network name in this example:
+
+::
+
+    [ml2]
+    mechanism_drivers = openvswitch,l2population,sriovnicswitch
+
+    [ml2_type_vlan]
+    network_vlan_ranges = sriovtenant1:1000:1009
+
+    [ml2_type_flat]
+    flat_networks = sriovtenant1
+
+
+Modify the file ``/etc/kolla/config/nova.conf``.  The Nova Scheduler service
+on the control node requires the ``PciPassthroughFilter`` to be added to the
+list of filters and the Nova Compute service(s) on the compute node(s) need
+PCI device whitelisting:
+
+::
+
+    [DEFAULT]
+    scheduler_default_filters = <existing filters>, PciPassthroughFilter
+    scheduler_available_filters = nova.scheduler.filters.all_filters
+
+    [pci]
+    passthrough_whitelist = [{"devname": "ens785f0", "physical_network": "sriovtenant1"}]
+
+
+Modify the file ``/etc/kolla/config/neutron/sriov_agent.ini``.  Add physical
+network to interface mapping.  Specific VFs can also be excluded here.  Leave
+blank to enable all VFs for the interface:
+
+::
+
+    [sriov_nic]
+    physical_device_mappings = sriovtenant1:ens785f0
+    exclude_devices =
+
+Run deployment.
+
+Verification
+------------
+
+Check that VFs were created on the compute node(s).  VFs will appear in the
+output of both ``lspci`` and ``ip link show``.  For example:
+
+::
+
+    lspci | grep net
+    05:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
+
+::
+
+    ip -d link show ens785f0
+    4: ens785f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP mode DEFAULT qlen 1000
+    link/ether 90:e2:ba:ba:fb:20 brd ff:ff:ff:ff:ff:ff promiscuity 1
+    openvswitch_slave addrgenmode eui64
+    vf 0 MAC 52:54:00:36:57:e0, spoof checking on, link-state auto, trust off
+    vf 1 MAC 52:54:00:00:62:db, spoof checking on, link-state auto, trust off
+    vf 2 MAC fa:16:3e:92:cf:12, spoof checking on, link-state auto, trust off
+    vf 3 MAC fa:16:3e:00:a3:01, vlan 1000, spoof checking on, link-state auto, trust off
+
+Verify the SRIOV Agent container is running on the compute node(s):
+
+::
+
+    docker ps --filter name=neutron_sriov_agent
+    CONTAINER ID   IMAGE                                                                COMMAND        CREATED         STATUS         PORTS  NAMES
+    b03a8f4c0b80   10.10.10.10:4000/registry/centos-source-neutron-sriov-agent:17.04.0  "kolla_start"  18 minutes ago  Up 18 minutes         neutron_sriov_agent
+
+
+Verify the SRIOV Agent service is present and UP:
+
+::
+
+    openstack network agent list
+    +--------------------------------------+--------------------+-------------+-------------------+-------+-------+---------------------------+
+    | ID                                   | Agent Type         | Host        | Availability Zone | Alive | State | Binary                    |
+    +--------------------------------------+--------------------+-------------+-------------------+-------+-------+---------------------------+
+    | 7c06bda9-7b87-487e-a645-cc6c289d9082 | NIC Switch agent   | av09-18-wcp | None              | :-)   | UP    | neutron-sriov-nic-agent   |
+    +--------------------------------------+--------------------+-------------+-------------------+-------+-------+---------------------------+
+
+Create a new provider network.  Set ``provider-physical-network`` to the
+physical network name that was configured in ``/etc/kolla/config/nova.conf``.
+Set ``provider-network-type`` to the desired type.  If using VLAN, ensure
+``provider-segment`` is set to the correct VLAN ID.  Type VLAN is used in this example:
+
+
+::
+
+    openstack network create --project=admin \
+                             --provider-network-type=vlan \
+                             --provider-physical-network=sriovtenant1 \
+                             --provider-segment=1000 \
+                             sriovnet1
+
+Create a subnet with a DHCP range for the provider network:
+
+::
+
+    openstack subnet create --network=sriovnet1 \
+                            --subnet-range=11.0.0.0/24 \
+                            --allocation-pool start=11.0.0.5,end=11.0.0.100 \
+                            sriovnet1_sub1
+
+Create a port on the provider network with vnic_type set to direct:
+
+::
+
+    openstack port create --network sriovnet1 --vnic-type=direct sriovnet1-port1
+
+Start a new instance with the SRIOV port assigned:
+
+::
+
+    openstack server create --flavor flavor1 \
+                            --image fc-26 \
+                            --nic port-id=`openstack port list | grep sriovnet1-port1 | awk '{print $2}'` \
+                            vm1
+
+Verify the instance boots with the SRIOV port.  Verify VF assignment by running
+dmesg on the compute node where the instance was placed.
+
+::
+
+    dmesg
+    [ 2896.849970] ixgbe 0000:05:00.0: setting MAC fa:16:3e:00:a3:01 on VF 3
+    [ 2896.850028] ixgbe 0000:05:00.0: Setting VLAN 1000, QOS 0x0 on VF 3
+    [ 2897.403367] vfio-pci 0000:05:10.4: enabling device (0000 -> 0002)
+
+For more information see `OpenStack SRIOV documentation <https://docs.openstack.org/neutron/pike/admin/config-sriov.html>`_.
+
+
+Nova SRIOV
+==========
+
+Preparation and deployment
+--------------------------
+
+Nova provides a separate mechanism to attach PCI devices to instances that
+is independent from Neutron.  Using the PCI alias configuration option in
+nova.conf, any PCI device (PF or VF) that supports passthrough can be attached
+to an instance.  One major drawback to be aware of when using this method is
+that the PCI alias option uses a device's product id and vendor id only,
+so in environments that have NICs with multiple ports configured for SRIOV,
+it is impossible to specify a specific NIC port to pull VFs from.
+
+Modify the file ``/etc/kolla/config/nova.conf``.  The Nova Scheduler service
+on the control node requires the ``PciPassthroughFilter`` to be added to the list
+of filters and the Nova Compute service(s) on the compute node(s) need PCI
+device whitelisting.  The Nova API service on the control node and the Nova
+Compute service on the compute node also require the ``alias`` option under the
+``[pci]`` section.  The alias can be configured as 'type-VF' to pass VFs or 'type-PF'
+to pass the PF. Type-VF is shown in this example:
+
+::
+
+    [DEFAULT]
+    scheduler_default_filters = <existing filters>, PciPassthroughFilter
+    scheduler_available_filters = nova.scheduler.filters.all_filters
+
+    [pci]
+    passthrough_whitelist = [{"vendor_id": "8086", "product_id": "10fb"}]
+    alias = [{"vendor_id":"8086", "product_id":"10ed", "device_type":"type-VF", "name":"vf1"}]
+
+Run deployment.
+
+Verification
+------------
+
+Create (or use an existing) flavor, and then configure it to request one PCI device
+from the PCI alias:
+
+::
+
+    openstack flavor set sriov-flavor --property "pci_passthrough:alias"="vf1:1"
+
+Start a new instance using the flavor:
+
+::
+
+    openstack server create --flavor sriov-flavor --image fc-26 vm2
+
+
+Verify VF devices were created and the instance starts successfully as in
+the Neutron SRIOV case.
+
+For more information see `OpenStack PCI passthrough documentation <https://docs.openstack.org/nova/pike/admin/pci-passthrough.html>`_.
\ No newline at end of file