Compute

Platform choice: 3-tier vs HCI

What the platform choice changes for the server: form factor, hardware freedom, firmware ownership, and licensing. The HCI vs 3-tier comparison, HCI models, cluster sizing and HCI networking are on HCI. External arrays and storage networking are on Storage. Licensing terms change often; recheck this section about every 6 months. No pricing. Sources checked 5–6 Oct 2026.

From the server’s point of view

The HCI, dHCI and 3-tier comparison and decision table are on HCI: HCI vs 3-tier. This section keeps only what the choice changes for the server.

Form factor and hardware freedom. 3-tier hosts need boot media only, so 1U is viable: on the same Dell 16G sheet, the 1U R660 takes 10 × 2.5 in drives and the 2U R760 takes 24 (ladder below). Any server and NIC on the hypervisor’s compatibility list will do. HCI nodes carry capacity drives and must come from the HCI vendor’s certified node or component list, which narrows CPU, NIC, controller and drive choice.

Firmware ownership. In 3-tier you own the server firmware baseline and must keep it inside both the hypervisor HCL and the array’s support matrix. In HCI the vendor’s lifecycle manager usually applies firmware, drivers and hypervisor as one validated bundle, so the server baseline is no longer yours to set.

Hypervisor licence metrics

Compare hypervisors by licence metric and lifecycle here; features and HCI-bundled hypervisors are on HCI. The metric decides whether core count is a performance choice or a cost choice.

Core count is a licence decision. Under per-core metrics, a socket with many cores costs proportionally more to license than one at the per-CPU minimum, while per-socket subscriptions do not change with cores. Model licence cost per host against core count and VM density before choosing the CPU SKU. Fewer, larger hosts also raise the share of capacity held back for HA. (Derived from the metrics above.)

Platform choice design / build / manage

Design

• Make the HCI vs 3-tier decision on HCI, then apply its server-side consequences here
• Model licence cost against core count per host for each candidate hypervisor
• For HCI, pick server models from the vendor’s certified list before fixing CPU, NIC and drives

Build

• Record which interoperability matrix governs each host: hypervisor HCL, array matrix, or HCI bundle

Manage

• Recheck licence terms at each renewal and before each refresh; they change faster than hardware

Server networking

The host side of the server-to-leaf link. Leaf port speeds, oversubscription and ECMP are on Network: Fabric design. Dual-homing on the switch side (MLAG/vPC vs EVPN ESI multihoming) is on Network: multihoming, and the tenant and underlay MTU on Network: MTU. Transceivers and cables are on Network: Optics. Storage networking is on Storage; HCI cluster traffic and RDMA on HCI; AI back-end NICs on AI Factory. NIC and DPU generations change every 12 to 24 months; recheck about every 6 months. Sources checked 5–6 Oct 2026.

What compute decides and what the fabric decides

NIC speed

Follow the leaf. The NIC speed is the leaf downlink speed chosen on Fabric design; a faster NIC than the leaf port buys nothing, and a slower one strands leaf capacity. Current 1U/2U OEM servers offer 1, 10, 25, 100 and 400GbE options on OCP NIC 3.0 slots (Dell R670 spec sheet). The OCP NIC 3.0 specification is at version 1.6.0 (31 Mar 2025) and defines SFF, TSFF, LFF and DSFF card sizes (OCP NIC 3.0).

Check the slot, not just the port. PCIe 5.0 signals at 32.0 GT/s per lane and PCIe 6.0 at 64.0 GT/s (PCI-SIG, PCIe 6.0). A Gen5 x16 slot therefore carries roughly 500 Gb/s per direction before protocol overhead: enough for one 400G port or two 200G ports at line rate, not two 400G ports. A dual-port 400G card needs a Gen6 x16 slot (or two Gen5 x16 links) to run both ports at line rate. Dual-port NICs used for redundancy, not throughput, are not limited by this.

NIC count and traffic separation

Two adapters, two leaves. Use two physical adapters with one port from each to each leaf, so neither a card nor a leaf is a single point of failure. Two ports on one card protect against a leaf or cable failure, not a card failure.

Converged or dedicated. Converged designs run every traffic class on one teamed pair, separated by VLAN and QoS or DCB. Microsoft documents a converged Switch Embedded Teaming design with DCB reserving 50% for SMB, and states this is “an example and not a support requirement” (Azure Local). Dedicated designs add ports for storage or migration, at the cost of more leaf ports and cables. The storage side of that choice is on Storage.

Storage multipath is not a LAG. IP storage multipathing (iSCSI MPIO, NVMe multipath) expects one path per adapter, usually on its own subnet or VLAN. Microsoft’s example keeps each storage adapter on its own subnet so traffic stays on its own leaf and off the MC-LAG peer link (Azure Local). Broadcom states that LACP is not compatible with software iSCSI multipathing, and that iSCSI multipathing should not run over static EtherChannel or LACP bonds (Broadcom KB 315347).

Host bonding and LACP

The host can run LACP (802.3ad/802.1AX), a static LAG, or switch-independent teaming. Which modes exist depends on the hypervisor and its virtual switch, not on the NIC.

LACP needs one partner. A host LAG across two leaves works only if the host sees a single LACP partner on both links and bonds them as one aggregate. Making two leaves look like one partner is the switch side’s job (Network: multihoming and LACP). Broadcom’s wording is that ESXi supports teaming to “a single physical switch or stacked switches” (KB 324555).

Choosing. LACP gives per-flow hashing across both links and fast detection of a mis-cabled or unidirectional member, but ties host and leaf configuration together and needs leaf-side LAGs for every host. Switch-independent teaming needs nothing on the leaf: each leaf port is an ordinary access or trunk port, and each vNIC is pinned to one uplink. Under SET it is the only choice; under vSphere it avoids the distributed-switch and LACP limits above. In either model a single flow never exceeds one link’s speed.

What switch-independent teaming does to the fabric. On failover, the vNIC’s MAC moves to the other leaf.

• On an MLAG/vPC pair the move stays inside the pair, behind its shared VTEP IP, and is not an EVPN MAC move. But the host ports are orphan (single-homed) ports on the pair, so traffic can cross the peer link (Network: multihoming).
• On ESI or single-homed leaves the move is an EVPN MAC Mobility event. Repeated flaps can trip EVPN duplicate-MAC detection, after which the fabric stops advertising that MAC (Network: service insertion).
• The hypervisor must announce the move (vSphere “Notify switches”, or RARP/gratuitous ARP from the virtual switch). Without it, traffic to that MAC blackholes until the old entry ages out.

Provisioning before the bond exists. A host that PXE-boots, or a freshly installed hypervisor, does not yet speak LACP. If the leaf requires LACP, the port stays down. Use the switch’s LACP fallback or bypass for provisioning ports, or provision on a switch-independent setup and convert later. During bypass the host must bring up only one member, or keep provisioning traffic on one link, to avoid loops or duplicate frames. Cumulus documents LACP bypass on EVPN-MH bonds and stops advertising the segment’s Type 1 and Type 4 routes while in bypass (Cumulus EVPN-MH).

DPU / SmartNIC

Vendors use “SmartNIC”, “DPU” and “IPU” loosely. This page uses three categories by what runs on the card.

Value is highest where the DPU takes work off licensed host cores (overlay, firewall, storage initiator), and lowest on plain VLAN-backed VM hosts. Before buying, confirm the hypervisor release supports that exact DPU in that exact server. AI SuperNICs are on AI Factory; RDMA for HCI on HCI.

BMC port

Most rack servers let the BMC use either a dedicated management port or a shared host NIC port with a VLAN. Dell’s iDRAC 10 documentation describes the dedicated port as “not shared with the host operating system” and separated from application traffic; shared LOM saves a cable and can fail over to a second LOM port (Dell KB 000374138, modified 26 Mar 2026). A shared port puts the BMC behind the production NIC and leaf, so it is not reachable when that path is down, which defeats the purpose of the OOB network. Cable the dedicated BMC port to the OOB network (Network: OOB), and harden it as on Security: BMCs. Automate through Redfish, DMTF’s current management API (release 2026.2, 14 Sep 2026) (DMTF Redfish).

Server networking design / build / manage

Design

• Take the NIC speed from the leaf downlink speed (fabric design) and check the PCIe slot can carry it
• Specify two adapters per host and decide converged or dedicated storage ports with Storage
• Pick a teaming mode the hypervisor supports and agree it with the fabric’s dual-homing model (multihoming)
• Keep storage multipath on separate paths, not in a LAG
• Buy a DPU only for a named offload on a supported hypervisor, DPU and server combination

Build

• Set host MTU to the agreed tenant MTU and test end to end with DF set
• Configure LACP fallback or a switch-independent build profile for provisioning ports
• Cable the dedicated BMC port to the OOB network
• Fail each leaf and each adapter in turn and confirm VM, storage and management traffic survive

Manage

• Keep NIC and DPU firmware and drivers in the host baseline and update them as a set
• Watch per-member link errors and LACP state; a silently down member halves capacity
• Recheck teaming and LACP limits after every hypervisor major release