linux-kernelusbcdc

Linux Gadget (Universal Device Controller): Dual CDC ECM or Dual RNDIS Ethernet


Working on the Nvidia Orin Nano device.
I would like to create either 2xRNDIS or 2xCDC-ECM interfaces as part of a UDC composite device.

OS: 20.04.6 LTS (Focal Fossa)
Linux kernel: v5.10.120-tegra

The default UDC configuration files in /opt/nvidia/l4t-usb-device-mode provide the following.
USB Composite Device:

On a Windows 11 machine, all these interfaces enumerate as expected.
Two independent network interfaces are created from the composite device.
One RNDIS, One CDC NCM.
Other Windows OS (Windows IoT) will not enumerate CDC NCM because they lack CDC NCM drivers.

I have changed the configuration file to create the following:
USB Composite Device:

Windows 11 does not enumerate both RNDIS devices (only one, the second device gives an error). Windows 11 does enumerate both CDC NCM devices but gives them both the same physical mac address.
On a Linux machine both CDC NCM devices also enumerate but with the same physical MAC addresses. Having the same MAC address for two network interfaces is problematic.

The configuration files explicitly provide unique mac addresses to each interface.
Is dual ethernet functions of the same type not allowed in Linux Gadget Configurations?
Am I missing some key configuration? I change the vendor and product id when making changes to prevent Windows configuration caching.

Has anyone successfully created a Linux USB gadget that supports dual network interfaces of the same type? (2 RNDIS or 2 CDC NCM)

Ideally, any number of RNDIS / CDC NCM devices could be created in a composite device. It can be difficult to support both RNDIS / CDC NCM drivers on a single machine. More versions of Windows support RNDIS drivers (Windows IoT).

Below is the modified Linux Gadgets configuration file. Using the default nvidia systemd service to trigger execution.

#!/bin/bash

# SPDX-FileCopyrightText: Copyright (c) 2017-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: BSD-3-Clause
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice, this
# list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# 3. Neither the name of the copyright holder nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

set -e

script_dir="$(cd "$(dirname "$0")" && pwd)"
. "${script_dir}/nv-l4t-usb-device-mode-config.sh"

modprobe libcomposite

for attempt in $(seq 60); do
    udc_dev_t210=700d0000.xudc
    if [ -e "/sys/class/udc/${udc_dev_t210}" ]; then
        udc_dev="${udc_dev_t210}"
        break
    fi
    udc_dev_t186=3550000.xudc
    if [ -e "/sys/class/udc/${udc_dev_t186}" ]; then
        udc_dev="${udc_dev_t186}"
        break
    fi
    udc_dev_t186=3550000.usb
    if [ -e "/sys/class/udc/${udc_dev_t186}" ]; then
        udc_dev="${udc_dev_t186}"
        break
    fi
    sleep 1
done
if [ "${udc_dev}" == "" ]; then
    echo No known UDC device found
    exit 1
fi

macs_file="${script_dir}/mac-addresses"
if [ -f "${macs_file}" ]; then
    . "${macs_file}"
    if ! [[ "${mac_rndis_h1}" =~ ^([a-fA-F0-9]{2}:){5}[a-fA-F0-9]{2}$ &&
         "${mac_rndis_d1}" =~ ^([a-fA-F0-9]{2}:){5}[a-fA-F0-9]{2}$ &&
         "${mac_rndis_h2}" =~ ^([a-fA-F0-9]{2}:){5}[a-fA-F0-9]{2}$ &&
         "${mac_rndis_d2}" =~ ^([a-fA-F0-9]{2}:){5}[a-fA-F0-9]{2}$ &&
         "${mac_ecm_h1}" =~ ^([a-fA-F0-9]{2}:){5}[a-fA-F0-9]{2}$ &&
         "${mac_ecm_d1}" =~ ^([a-fA-F0-9]{2}:){5}[a-fA-F0-9]{2}$ &&
         "${mac_ecm_h2}" =~ ^([a-fA-F0-9]{2}:){5}[a-fA-F0-9]{2}$ &&
         "${mac_ecm_d2}" =~ ^([a-fA-F0-9]{2}:){5}[a-fA-F0-9]{2}$ ]]; then
        rm "${macs_file}"
    fi
fi

if ! [ -f "${macs_file}" ]; then
    # Generate unique data
    if [ -f /proc/device-tree/serial-number ]; then
        random="$(md5sum /proc/device-tree/serial-number|cut -c1-12)"
    else
        random="$(echo "no-serial"|md5sum|cut -c1-12)"
    fi

    # Extract 6 bytes
    b1="$(echo "${random}"|cut -c1-2)"
    b2="$(echo "${random}"|cut -c3-4)"
    b3="$(echo "${random}"|cut -c5-6)"
    b4="$(echo "${random}"|cut -c7-8)"
    b5="$(echo "${random}"|cut -c9-10)"
    b6="$(echo "${random}"|cut -c11-12)"

    # Clear broadcast/multicast, set locally administered bits
    b1="$(printf "%02x" "$(("0x${b1}" & 0xfe | 0x02))")"

    # Set 4 LSBs to unique value per interface
    b6_rndis_h1="$(printf "%02x" "$(("0x${b6}" & 0xf8 | 0x00))")"
    b6_rndis_d1="$(printf "%02x" "$(("0x${b6}" & 0xf8 | 0x01))")"
    b6_rndis_h2="$(printf "%02x" "$(("0x${b6}" & 0xf8 | 0x02))")"
    b6_rndis_d2="$(printf "%02x" "$(("0x${b6}" & 0xf8 | 0x03))")"
    b6_ecm_h1="$(printf "%02x" "$(("0x${b6}" & 0xf8 | 0x04))")"
    b6_ecm_d1="$(printf "%02x" "$(("0x${b6}" & 0xf8 | 0x05))")"
    b6_ecm_h2="$(printf "%02x" "$(("0x${b6}" & 0xf8 | 0x06))")"
    b6_ecm_d2="$(printf "%02x" "$(("0x${b6}" & 0xf8 | 0x07))")"

    # Construct complete MAC per interface
    mac_rndis_h1="${b1}:${b2}:${b3}:${b4}:${b5}:${b6_rndis_h1}"
    mac_rndis_d1="${b1}:${b2}:${b3}:${b4}:${b5}:${b6_rndis_d1}"
    mac_rndis_h2="${b1}:${b2}:${b3}:${b4}:${b5}:${b6_rndis_h2}"
    mac_rndis_d2="${b1}:${b2}:${b3}:${b4}:${b5}:${b6_rndis_d2}"
    mac_ecm_h1="${b1}:${b2}:${b3}:${b4}:${b5}:${b6_ecm_h1}"
    mac_ecm_d1="${b1}:${b2}:${b3}:${b4}:${b5}:${b6_ecm_d1}"
    mac_ecm_h2="${b1}:${b2}:${b3}:${b4}:${b5}:${b6_ecm_h2}"
    mac_ecm_d2="${b1}:${b2}:${b3}:${b4}:${b5}:${b6_ecm_d2}"

    # Save values for next boot
    echo "mac_rndis_h1=${mac_rndis_h1}" > "${macs_file}"
    echo "mac_rndis_d1=${mac_rndis_d1}" >> "${macs_file}"
    echo "mac_rndis_h2=${mac_rndis_h2}" >> "${macs_file}"
    echo "mac_rndis_d2=${mac_rndis_d2}" >> "${macs_file}"
    echo "mac_ecm_h1=${mac_ecm_h1}" >> "${macs_file}"
    echo "mac_ecm_d1=${mac_ecm_d1}" >> "${macs_file}"
    echo "mac_ecm_h2=${mac_ecm_h2}" >> "${macs_file}"
    echo "mac_ecm_d2=${mac_ecm_d2}" >> "${macs_file}"
fi

mkdir -p /sys/kernel/config/usb_gadget/l4t
cd /sys/kernel/config/usb_gadget/l4t
echo 0x4422 > idVendor
echo 0x2244 > idProduct
echo 0x0001 > bcdDevice
echo 0xEF > bDeviceClass
echo 0x02 > bDeviceSubClass
echo 0x01 > bDeviceProtocol

mkdir -p strings/0x409
if [ -f /proc/device-tree/serial-number ]; then
    serialnumber="$(cat /proc/device-tree/serial-number|tr -d '\000')"
else
    serialnumber=no-serial
fi
echo "${serialnumber}" > strings/0x409/serialnumber
echo "MANUFACTURER" > strings/0x409/manufacturer
echo "PRODUCT" > strings/0x409/product

cfg=configs/c.1
mkdir -p "${cfg}"
cfg_str=""
is_remote_wakeup=0

# rndis0
if [ ${enable_rndis1} -eq 1 ]; then
    cfg_str="${cfg_str}+RNDIS"
    func=functions/rndis.usb0
    mkdir -p "${func}"
    echo "${mac_rndis_h1}" > "${func}/host_addr"
    echo "${mac_rndis_d1}" > "${func}/dev_addr"
    ln -sf "${func}" "${cfg}"

    echo 1 > os_desc/use
    echo 0xcd > os_desc/b_vendor_code
    echo MSFT100 > os_desc/qw_sign

    echo RNDIS > "${func}/os_desc/interface.rndis/compatible_id"
    echo 5162001 > "${func}/os_desc/interface.rndis/sub_compatible_id"
    ln -sf "${cfg}" os_desc

    is_remote_wakeup=1
fi

# rndis1
if [ ${enable_rndis2} -eq 1 ]; then
    cfg_str="${cfg_str}+RNDIS"
    func=functions/rndis.usb1
    mkdir -p "${func}"
    echo "${mac_rndis_h2}" > "${func}/host_addr"
    echo "${mac_rndis_d2}" > "${func}/dev_addr"
    ln -sf "${func}" "${cfg}"

    echo RNDIS > "${func}/os_desc/interface.rndis/compatible_id"
    echo 5162001 > "${func}/os_desc/interface.rndis/sub_compatible_id"

    is_remote_wakeup=1
fi

# serial
if [ ${enable_acm} -eq 1 ]; then
    cfg_str="${cfg_str}+ACM"
    func=functions/acm.GS0
    mkdir -p "${func}"
    ln -sf "${func}" "${cfg}"
fi

# usb0
if [ ${enable_ecm1} -eq 1 ]; then
    cfg_str="${cfg_str}+${ecm_ncm_name}"
    func=functions/${ecm_ncm}.usb0
    mkdir -p "${func}"
    echo "${mac_ecm_h1}" > "${func}/host_addr"
    echo "${mac_ecm_d1}" > "${func}/dev_addr"
    ln -sf "${func}" "${cfg}"
    is_remote_wakeup=1
fi

# usb1
if [ ${enable_ecm2} -eq 1 ]; then
    cfg_str="${cfg_str}+${ecm_ncm_name}"
    func=functions/${ecm_ncm}.usb1
    mkdir -p "${func}"
    echo "${mac_ecm_h2}" > "${func}/host_addr"
    echo "${mac_ecm_d2}" > "${func}/dev_addr"
    ln -sf "${func}" "${cfg}"
    is_remote_wakeup=1
fi

if [ ${is_remote_wakeup} -eq 1 ]; then
    echo "0xe0" > "${cfg}/bmAttributes"
else
    echo "0xc0" > "${cfg}/bmAttributes"
fi

mkdir -p "${cfg}/strings/0x409"
echo "${cfg_str:1}" > "${cfg}/strings/0x409/configuration"
echo "${udc_dev}" > UDC

cd - # Out of /sys/kernel/config/usb_gadget

# trigger if device connected before script completed
udevadm trigger -v --action=change --property-match=SUBSYSTEM=android_usb
udevadm trigger -v --action=change --property-match=SUBSYSTEM=usb_role

exit 0

Solution

  • There is a bug in the Linux Kernel that causes the MAC addresses to be the same.

    The kernel fix is already present in the ecm driver.
    f_ecm.c driver file: https://elixir.bootlin.com/linux/v6.16/source/drivers/usb/gadget/function/f_ecm.c
    On line 700, the MAC address is updated inside the bind function.

        ecm_string_defs[1].s = ecm->ethaddr;
    
        us = usb_gstrings_attach(cdev, ecm_strings,
                     ARRAY_SIZE(ecm_string_defs));
    

    The equivalent line needs to be added to the f_ncm.c driver.

        ncm_string_defs[1].s = ecm->ethaddr;
        
        us = usb_gstrings_attach(cdev, ncm_strings,
                     ARRAY_SIZE(ncm_string_defs));
    

    I have sent a patch request to USB Linux Kernel maintainer with the patch applied to f_ncm driver.

    The quickfix, use the ecm driver when configuring Linux Gadgets.
    It is an older standard but the fix is already present.
    Since: Kernel v4.18
    Commit: d3ac41bb330bea3a2929a70b8f4df20a0fa55d18