Nexus VMware VM-FEX  Video Configuration Guide

Document created by timoma on Nov 7, 2013Last modified by timoma on Nov 8, 2013
Version 9Show Document
  • View in full screen mode



A virtualized server implementation consists of one or more VMs running as 'guests' on a single physical server. The guest VMs are hosted and managed by a software layer called the hypervisor or virtual machine manager (VMM). The hypervisor typically presents a virtual network interface to each VM and performs Layer 2 switching of traffic from a VM to other local VMs or to a physical interface to the external network. Working with a Cisco virtual interface card (VIC) adapter, Cisco Virtual Machine Fabric Extender (VM-FEX)bypasses software-based switching of VM traffic by the hypervisor in favor of external hardware-based switching in the fabric interconnect. This method results in a reduced load on the server CPU, faster switching,and the ability to apply a rich set of network management features to local and remote traffic. VM-FEX extends the (prestandard) IEEE 802.1Qbh port extender architecture to the VMs, providing each VM interface with a virtual Peripheral Component Interconnect Express (PCIe) device and a virtual port ona switch. This solution allows precise rate limiting and quality of service (QoS) guarantees on the VM interface.


Infrastructure Requirements

  • Nexus VMware VM-FEX is available with Nexus 5500 Series and UCS standalone C series with VIC (P81E,VIC1225, VIC1225T)
  • Each VIC card supports up to 96 Virtual Machine Interface
  • Enterprise Plus License required (as is for any DVS) on Host
  • Standard License and above is required for vCenter
  • Hypervisor features are supported for both emulated and hypervisor bypass mode

     – vMotion, HA, DRS, Snapshot and Hot add/remove virtual device, Suspend/Resume

  • VMDirectPath (Hypervisor Bypass) with VM-FEX is supportted with ESXi5.0+
  • VM-FEX upgrade is supportted from ESXi4.X to ESXi5.X with Customized ISO and VMware Update Manager


VM-FEX Mode of Operations





Hypervisor Bypass

Hypervisor Version

ESX 4.0 U1 +

ESXi 5.0 +

Minmum NX-OS Version

Minimum VIC Firmware Version

P81E:         1.6(2a)

VIC1225:     HUU 1.5.1(b)

VIC1225T:   HUU 1.5.1(b)

P81E:             1.6(2a)

VIC1225:         HUU 1.5.1(b)

VIC1225T:       HUU 1.5.1(b)

Minimum CIMC Version

P81E:         1.4(2.3)

VIC1225:     HUU 1.5.1(b)

VIC1225T:   HUU 1.5.1(b)

P81E:             1.4(2.3)

VIC1225:         HUU 1.5.1(b)

VIC1225T:       HUU 1.5.1(b)

VMotion / Live Migration

Support Guest OS Any OS support by ESX/ ESXi

RHEL 6.0+ , SLES 11+

Window 2008 SP2, Windows 2008 R2, Windows 2012



Guest OS/vSphere Version
vSphere 5.0
vSphere 5.0 U1
vSphere 5.1
Windows 2012N/A1.2.24.0 (vmwaretools) (vmwaretools)
Windows 2008 R21.2.22.0 (vmwaretools) (vmwaretools) (vmwaretools)
Windows 2008 SP21.2.22.0 (vmwaretools) (vmwaretools) (vmwaretools)
RHEL (inbox) (inbox) (inbox)
RHEL (inbox) (inbox) (inbox)
RHEL 6.2N/A (inbox) (vmwaretools) (inbox) (vmwaretools)

SLES111.0.36.0 (vmwaretools) (vmwaretools) (vmwaretools)
SLES11 SP11.0.36.0 (vmwaretools) (vmwaretools) (vmwaretools)



VM-FEX Scalability


UCSM Configuration Limit 2.1 Release -



Cisco UCS 6100 / 6200 Series



Half-Width Blade with Single VIC


Full-Width Blade with Dual VIC

ESX 4.0 – 4.1 (DirectPath I/O)

56 (54 vNIC + 2 vHBA)

56 (54 vNIC + 2 vHBA)

ESXi 5.0 – 5.1 (DirectPath I/O)

116 (114 vNIC + 2 vHBA)

116 (114 vNIC + 2 vHBA)*

Windows 2012 (SR-IOV)

116 (114 vNIC + 2 vHBA)

232 (228 vNIC + 4 vHBA)

KVM 6.1 – 6.3 (SR-IOV)

116 (114 vNIC + 2 vHBA)

232 (228 vNIC + 4 vHBA)


* Additional VIC will NOT increase the total VIF count due to OS limitation

* Multiple VICs are Supported for full width blade and B200M3


Nexus 5500  Series
HypervisorAdapter FEXVM-FEX

ESX 4.1 – ESXi 5.1

96 vNIC

96 vNIC


* Only one VIC (P81E/VIC1225) is Supported for each C series rack server


Nexus Verified Scalability 6.0(2)N1(1) release


Verified Topology
Verified Maximum
Number of VFCs over Virtual Ethernet Interfaces40 40
Number of Port Profiles1,0001,000
Number of Virtual Machines (VMs) Concurrently not VMotioned5 VMs with 10 vNICs each5 VMs with 10 vNICs each
Number of Virtual Ethernet Interfaces Enabled with vNIC Shaping2,0002,000
Number of Virtual Ethernet Interfaces Enabled with Untagged CoS2,0002,000
Server - Number of Adapters per Server11
Server - Number of vNICs per Server5096



VM-FEX Feature Comparision





Adapter FEX






Port Security YesYesYes
IGMP Snooping YesYes
Qos Shaping YesYesYes
QoS Classification Yes YesYes
PVLANYes YesIsloate
L2 Multipath Edge Port Edge Port
TrustSec Yes Yes
DHCP Snooping YesYes
IP Source Guard YesYes




Reference Documentation


Nexus VM-FEX Configuration Guide


Cisco-Live-BRKCOM-2005-Virtual Machine Fabric Extension (VM-FEX)-2013

VM-FEX Configuration & Best Practice for Multiple Hypervisor


Please refer to the attachmenta


Cisco VM-FEX Performance Testing verus VMware vSwitch


External (Light Reading) VM-FEX Peroformance Testing verus VMware vSwitch


VM-FEX Performance Testing with SAP


VM-FEX Performance Testing with Microsoft SharePoint 2010


VM-FEX Performance Testing with Microsoft SQL Server 2012


VM-FEX Performance Testing with Citrix Xenapp 6.5


Microsoft Private Cloud Fast Track 3.0 with VM-FEX


Video Configuration Demo Topology


  • Nexus 5548/C22 M3/ VIC1225/ESXi
  • Nexus 5548-A is pre-configured, focus on 5548-B with the same configuration in the demo
  • Uplink Redundancy – Each static vNIC configure as Active/Standby

     -No need for OS teaming software

     -Required for hypervisor uplink

  • Each dynamic vNIC attach to uplink in Round Robin fashion
  • vPC Doman and Peer Link is configured to synchronize veth numbering for the same VM

          -Not used for the forwarding plane


nexus 5k VM-FEX.png



Nexus 5500 VM-FEX Configuration Workflow