Internal NICs on UCS E-Series Servers

General rules:

  1. The router’s ‘ucse1/0/0 + ucse1/0/1’ ports will map to GE0 and GE1 on the blade
  2. Bridging must be used to join the physical interfaces to the same broadcast domain
  3. Separate bridges must be used for each VLAN
  4. To use a native vlan, use “encapsulation untagged”

In our case we wanted to use tagging in the connection to the switch, but put the blade’s GE0 and GE1 interfaces on vlan 123 without tagging:

bridge-domain 123
!
interface Port-channel1
 no ip address
 no negotiation auto
 service instance 123 ethernet
 encapsulation dot1q 123
 rewrite ingress tag pop 1 symmetric
 snmp ifindex persist
 bridge-domain 123
 !
interface GigabitEthernet0/0/1
 description Switch:1
 no ip address
 negotiation auto
 channel-group 1 mode active
interface GigabitEthernet0/0/2
 description Switch:2
 no ip address
 negotiation auto
 channel-group 1 mode active
!
interface ucse1/0/0
 description UCS E-Series Blade:GE0
 no ip address
 no negotiation auto
 switchport mode trunk
 service instance 123 ethernet
 encapsulation untagged
 bridge-domain 123
 !
interface ucse1/0/1
 description UCS E-Series Blade:GE1
 no ip address
 no negotiation auto
 switchport mode trunk
 service instance 123 ethernet
 encapsulation untagged
 bridge-domain 123
 !

 

Advertisement

Fixing stuck UCS-E Server

I purchased a UCS-E140DP-M1 blade off eBay for $900 so we could throw it in one of our ISR G2s and play with it.  It powered on fine, and the first step was to get the firmware upgraded.  The CIMC went from 1.0 to 2.4.1 no problem, but upon taking it to 3.1.3, the blade would not boot.  Seemed to be stuck at the BIOS prompt..

ucse_bios_stuck

My theory right away was an incompatibility between the CIMC and the BIOS.  The original BIOS version shipped was 4.6.4.8, but the newer versions were lower, indicating something major had changed.  I attempted multiple times to to upgrade the BIOS to the versions matching CIMC 1.0.2, 2.4.1, and 2.5.0.3, but would always get some error message.  Common output looked like this:

Router#ucse subslot 2/0 session imc 
Trying 10.10.10.10, 2131 ... Open
Username: admin
Password: 
E140DP-FOC16490HEF# scope bios
E140DP-FOC16490HEF /bios # show detail
BIOS:
    BIOS Version: 4.6.4.8
    Boot Order: CDROM:Virtual-CD
    FW Update/Recovery Status: Error, Update invalid
    Active BIOS: backup

After downgrading the CIMC back to 2.4.1, I was copy and map huu-2.4.1.iso, follow the wizards, and then upgrade the BIOS.  This looked much better.

E140DP-FOC16490HEF /bios # show detail
BIOS:
    BIOS Version: "UCSED.1.5.0.2 (Build Date: 05/15/2013)"
    Boot Order: CDROM:Virtual-CD
    FW Update/Recovery Status: Done, OK
    Active BIOS on next reboot: main

So long story short – use the HUU to do all firmware updates, so the CIMC and BIOS firmware versions are in sync.