Identify and Configure USB Devices for PCI Passthrough

Updated

Introduction

PCI passthrough allows a virtual machine (VM) to have direct access to a physical PCI device on the host machine. PCI passthrough to a VM for entities like an ethernet controller or USB_Controller is a common use case for ZEDEDA. To do this, you must identify the device by its IOMMU group and PCI address in the hardware model JSON file.

This article explains how to identify and configure individual USB devices for passthrough in your hardware model.

This is a series of articles. You will likely follow them in this order.

  1. Create a Hardware Model 
  2. Use the ZEDEDA CLI to Upload a Hardware Model
  3. Use ZCLI to Update a Hardware Model Logo

If you configure a USB_Controller for passthrough, then you need to be aware that all associated individual USB_Devices (ports) will be passed through together. In a scenario where your EVE-OS device has multiple external USB_Devices (ports) associated with a single USB_Controller (internal), you need to follow this article to use the “USB_Device Splitting” function to assign individual USB devices (ports) independently.

Considerations

Individual external USB_Device (ports) might have underlying USB2.0 and/or USB3.0 mappings that you must identify.

For example a “blue” USB_Device (port) might have a standalone USB3.0 “bus:port” map, or it might have a single USB3.0 “bus:port” mapping. A “black” USB_Device (port) might have a single USB2.0 “bus:port” mapping. *Color-coding and internal bus:port architecture might vary by manufacturer and/or device model.

Note: the USB_Device (port) or USB_Controller entity in the hardware model manifest can also be explicitly mapped to a specific VendorID. If mapped to a specific VendorID, EVE-OS will only passthrough an external device connected to the respective USB_Device (port) if the VendorID is a match.

*If present, a WWAN modem may also be associated with a parent USB_Controller. If the parent USB_Controller is configured as a passthrough, all associated USB_Devices (ports) and the WWAN controller will be passedthrough to the same endpoint. See Identify and Configure a WWAN Modem for PCI Passthrough.

If the WWAN modem is split from the USB_Controller, all individual USB_Device (port) entities must also be individually identified. 

Attempts to simultaneously assign a USB_Controller if an associated WWAN modem or USB_Device (port) is in use will result in a PCI passthrough conflict error.

Warning: Updates to an existing hardware model manifest in the Marketplace will be automatically reflected to ALL onboarded Edge Nodes in the Enterprise that reference the respective Model name. You should complete your testing in a lab environment first.

Prerequisites

  • You have an EVE-OS image installed on your edge device.
  • You have access to the device (for example: SSH, ZCLI with ssh/console/vga/usb flags set via ZCLI if locked, or direct console & keyboard).
  • Your device has physical USB ports for USB devices (for example: 2.0, 3.0, keyboard/etc)
  • You have the Reference Hardware Model .json files (which you can find in the post-installation /Inventory directory on the installer USB)
    • controller-model.json
    • controller-model-usb.json
  • You are comfortable modifying and uploading a Hardware Model .json file.

Example device: Advantech 2271G V2

This procedure uses an Advantech 2271G V2 (with two "blue" USB 3.2 ports) as an example. CLI outputs are truncated for brevity. 

Identify the USB_Controller

  1. Access the EVE-OS CLI on your edge device. The example device is configured for SSH access.

  2. Identify all USB controllers via respective PCI address and IOMMU group using the command lspci -kv
    Example Output:

    00:14.0 USB controller: Intel Corporation Device 4b7d (rev 11) (prog-if 30 [XHCI])
    DeviceName: Onboard - Other
    Flags: bus master, medium devsel, latency 0, IRQ 126, IOMMU group 2
    Memory at 6001100000 (64-bit, non-prefetchable) [size=64K]
    Capabilities: [70] Power Management version 2
    Capabilities: [80] MSI: Enable+ Count=1/8 Maskable- 64bit+
    Capabilities: [90] Vendor Specific Information: Len=14 <?>
    Capabilities: [b0] Vendor Specific Information: Len=00 <?>
    Kernel driver in use: xhci_hcd
  3. From the output, you can see this device has a single USB_Controller with the PCI address 0000:00:14.0, and resides in IOMMU Group 2.

Identify the USB Top 2.0 “bus:port” mapping

  1. Insert a USB2.0 device (for example, a Logitech keyboard) to the “top” USB port.

  2. Execute the command lsusb to verify that the Logitech keyboard is identified, then note the reported bus.

    Example:

    67d61819-7c87-4174-baa1-7c8a9bef5b65:~# lsusb

    Example Output:

    Bus 001 Device 005: ID 046d:c31c Logitech USB Keyboard
Bus 002 Device 001: ID 1d6b:0003 Linux 6.1.112-linuxkit-272f44dbfe09 xhci-hcd xHCI Host
  3. Run the lsusb -tv command to identify the respective port. 

  4. Cross-reference the previously identified bus (lsusb output) and VendorID to determine the respective port. 

    Example:

    67d61819-7c87-4174-baa1-7c8a9bef5b65:~# lsusb -tv

    Example Output:

    /:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 10000M
    ID 1d6b:0003  
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/10p, 480M
    ID 1d6b:0002  
    |__ Port 6: Dev 7, If 0, Class=, Driver=usbhid, 1.5M
        ID 046d:c31c  
    |__ Port 6: Dev 7, If 1, Class=, Driver=usbhid, 1.5M
        ID 046d:c31c 
  5. From the output, you can see the Logitech keyboard resides on Bus 01:Port6.
  6. Remove the USB2.0 device (Logitech keyboard).

Identify the USB Top 3.0 “bus:port” mapping

  1. Insert a USB3.0 device (for example, a Patriot Memory USB3.0 stick) to the “top” USB port. 

  2. Repeat the comparison between lsusb and lsusb -tv output to cross-reference the previously identified bus (lsusb output) and VendorID to determine the respective port using a  USB3.0 device.

    Example:

    67d61819-7c87-4174-baa1-7c8a9bef5b65:~# lsusb

    Example Output:

    Bus 001 Device 001: ID 1d6b:0002 Linux 6.1.112-linuxkit-272f44dbfe09 xhci-hcd xHCI Host
Bus 002 Device 003: ID 13fe:6300          Patriot Memory

    Example:

    67d61819-7c87-4174-baa1-7c8a9bef5b65:~# lsusb -tv

    Example Output:

    /:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 10000M
    ID 1d6b:0003  
    |__ Port 1: Dev 3, If 0, Class=, Driver=usb-storage, 5000M
        ID 13fe:6300  
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/10p, 480M
    ID 1d6b:0002 
  3. From the output, you can see the identified bus is Bus 02, and the identified port is Port 1 (Bus2:Port1).
  4. Remove the USB3.0 device (Patriot Memory USB3.0 stick)

Identify all remaining USB_Device “bus:port” mappings

  1. Repeat steps as needed to identify all additional physical USB_Devices (ports) and respective “bus:port” mapping. The example device has 1 additional additionall USB port (bottom) with two associated USB_devices (USB 2.0 and USB3.0). *USB-C/Thunderbolt ports are supported.

  2. Identify USB Bottom 2.0 “bus:port” mapping.
    Example:

    67d61819-7c87-4174-baa1-7c8a9bef5b65:~# lsusb

    Example Output:

    Bus 002 Device 001: ID 1d6b:0003 Linux 6.1.112-linuxkit-272f44dbfe09 xhci-hcd xHCI Host
Bus 001 Device 006: ID 046d:c31c Logitech USB Keyboard

    Example:

    67d61819-7c87-4174-baa1-7c8a9bef5b65:~# lsusb -tv

    Example Output:

    /:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 10000M
    ID 1d6b:0003  
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/10p, 480M
    ID 1d6b:0002  
    |__ Port 8: Dev 6, If 1, Class=, Driver=usbhid, 1.5M
        ID 046d:c31c  
    |__ Port 8: Dev 6, If 0, Class=, Driver=usbhid, 1.5M
        ID 046d:c31c  
  3. From the outputs, you can see Bus1:Port8. 
  4. Remove the USB2.0 device (Logitech keyboard)

  5. Identify USB Bottom 3.0 “bus:port” mapping.
    Example:

    67d61819-7c87-4174-baa1-7c8a9bef5b65:~# lsusb

    Example Output:

    Bus 002 Device 004: ID 13fe:6300          Patriot Memory
Bus 002 Device 001: ID 1d6b:0003 Linux 6.1.112-linuxkit-272f44dbfe09 xhci-hcd xHCI Host 
67d61819-7c87-4174-baa1-7c8a9bef5b65:~# lsusb -tv
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 10000M
    ID 1d6b:0003  
    |__ Port 2: Dev 4, If 0, Class=, Driver=usb-storage, 5000M
        ID 13fe:6300  
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/10p, 480M
    ID 1d6b:0002 
  6. From the outputs you can see, Bus2:Port2. 
  7. Remove the USB3.0 device (Patriot Memory USB3.0 stick)

Update the Hardware Model .json

If you used a USB installer to image the device, “controller-model-usb.json” can be referenced if the respective device’s Inventory directory is still intact. 

Alternatively, you can run the command spec.sh -u in the EVE-OS cli to retrieve a sample USB_Device entity template.

When you update your hardware model, it should contain the following 5 USB-related entities identified in the preceding steps:

  • USB_Controller
  • USB_Device_Top_2.0
  • USB_Device_Top_3.0
  • USB_Device_Bottom_2.0
  • USB_Device_Bottom_3.0

Define the USB_Controller

By default, EVE-OS will automatically build a Hardware Model .json file that identifies USB_Controller entities by PCI address and respective IOMMU group and written to the installer USB under the filename  “controller-model.json”. Update the json file with the identified entities.

    {
      "ztype": "IO_TYPE_USB_CONTROLLER",
      "phylabel": "USB_Controller",
      "assigngrp": "group2",
      "phyaddrs": {
        "PciLong": "0000:00:14.0"
      },
      "logicallabel": "USB_Controller",
      "usagePolicy": {}
    }

Example of ZEDEDA GUI Marketplace > Model: Standalone USB_Controller defined and visible

Define the USB_Device_Top_2.0

Execute the command spec.sh -u in EVE-OS cli to retrieve a sample USB_Device entity template. Update the json file with the identified entities.

    {
      "ztype": "IO_TYPE_USB_DEVICE",
      "phylabel": "USB2.0-top",
      "assigngrp": "USB2.0-top",
      "parentassigngrp":",
      "cost": 0,
      "phyaddrs": {
        "usbaddr": "1:6",
        "usbproduct": ""
      },
      "logicallabel": "USB2.0-top",
      "usagePolicy": {}
    }

Define the USB_Device_Top_3.0

Update the json file with the identified entities.

    {
      "ztype": "IO_TYPE_USB_DEVICE",
      "phylabel": "USB3.0-top",
      "assigngrp": "USB3.0-top",
      "parentassigngrp": "",
      "cost": 0,
      "phyaddrs": {
        "usbaddr": "2:1",
        "usbproduct": ""
      },
      "logicallabel": "USB3.0-top",
      "usagePolicy": {}
    }

Upload the hardware model

Upload the json file via ZCLI and confirm updates are reflected in the Adapters section of the Marketplace > Device Model section in the ZEDEDA GUI.

Full device manifest sample

This example is specific to the ADV-2271G-V2 device and is for reference use only.

{
  "arch": 2,
  "productURL": "",
  "productStatus": "production",
  "attr": {
    "memory": "7666M",
    "storage": "535G",
    "Cpus": "4",
    "watchdog": "true",
    "hsm": "1",
    "leds": "0"
  },
  "logo": {
    "logo_back":"/workspace/spec/logo_back_.jpg",
    "logo_front":"/workspace/spec/logo_front_.jpg"
  },
  "ioMemberList": [
    {
      "ztype": "IO_TYPE_USB_CONTROLLER",
      "phylabel": "USB_Controller",
      "assigngrp": "group2",
      "phyaddrs": {
        "PciLong": "0000:00:14.0"
      },
      "logicallabel": "USB_Controller",
      "usagePolicy": {}
    },
    {
      "ztype": "IO_TYPE_WWAN",
      "phylabel": "wwan0",
      "assigngrp": "wwan0",
      "parentassigngrp": "",
      "cost": 10,
      "phyaddrs": {
        "usbaddr": "1:5",
        "usbproduct": ""
      },
      "logicallabel": "wwan0",
      "usagePolicy": {}
    },
    {
      "ztype": 1,
      "usage": 1,
      "phylabel": "eth0",
      "logicallabel": "eth0",
      "usagePolicy": {},
      "cost": 0,
      "assigngrp": "group10",
      "phyaddrs": {
        "Ifname": "eth0",
        "PciLong": "0000:01:00.0"
      }
    },
    {
      "ztype": 5,
      "usage": 1,
      "phylabel": "wlan0",
      "logicallabel": "wlan0",
      "usagePolicy": {},
      "cost": 0,
      "assigngrp": "group12",
      "phyaddrs": {
        "Ifname": "wlan0",
        "PciLong": "0000:03:00.0"
      }
    },
        {
      "ztype": "IO_TYPE_USB_DEVICE",
      "phylabel": "USB2.0-top",
      "assigngrp": "USB2.0-top",
      "parentassigngrp": "",
      "cost": 0,
      "phyaddrs": {
        "usbaddr": "1:6",
        "usbproduct": ""
      },
      "logicallabel": "USB2.0-top",
      "usagePolicy": {}
    },
    {
      "ztype": "IO_TYPE_USB_DEVICE",
      "phylabel": "USB3.0-top",
      "assigngrp": "USB3.0-top",
      "parentassigngrp": "",
      "cost": 0,
      "phyaddrs": {
        "usbaddr": "2:1",
        "usbproduct": ""
      },
      "logicallabel": "USB3.0-top",
      "usagePolicy": {}
    },
    {
      "ztype": "IO_TYPE_USB_DEVICE",
      "phylabel": "USB2.0-bottom",
      "assigngrp": "USB2.0-bottom",
      "parentassigngrp": "",
      "cost": 0,
      "phyaddrs": {
        "usbaddr": "1:8",
        "usbproduct": ""
      },
      "logicallabel": "USB2.0-bottom",
      "usagePolicy": {}
    },
    {
      "ztype": "IO_TYPE_USB_DEVICE",
      "phylabel": "USB3.0-bottom",
      "assigngrp": "USB3.0-bottom",
      "parentassigngrp": "",
      "cost": 0,
      "phyaddrs": {
        "usbaddr": "2:2",
        "usbproduct": ""
      },
      "logicallabel": "USB3.0-bottom",
      "usagePolicy": {}
    }
  ]
}

Next Steps

This is a series of articles. You will likely follow them in this order.

  1. Create a Hardware Model 
  2. Use the ZEDEDA CLI to Upload a Hardware Model
  3. Use ZCLI to Update a Hardware Model Logo
Was this article helpful?
0 out of 0 found this helpful