Bridging GIG0/0 to ESXi

Version 1
    This document was generated from CDN thread

    Created by: SCOTT PITTS on 11-10-2011 04:23:49 PM
    I am trying to get an untagged interface GIG0/0 to bridge to VLAN10 which is only connected to VMWare on the SRE. I have successfully done this when I use EHWIC-D8ESGP and sometime it works without that card. My problem is that you can not tell if it is going to work and wether VLAN 10 will actualy come up and if vlan 10 will be trunked to SM1/1. Sometimes when I activate the Bridge IRB and the BVI10 interface the router becomes sluggish when entering config t or term mon, but the processor is only at 10%. The behavior is very strange and causes tracebacks. This happens on both c2900-universalk9-mz.SPA.151-4.M.bin and  c2900-universalk9-mz.SPA.151-3.T.bin. I am attaching a tracback and the config that works on some routers.
     

    Oct 11 21:50:18.847 CDT: %BIT-4-OUTOFRANGE: bit 10 is not in the expected range of 0 to -1
    -Traceback= 262ABB8Cz 216C5CC8z 222B4954z 222B64D0z 222B7D6Cz 22494B18z 2249B018z 224AA3E0z 23CFED44z 23CF5080z 237CC950z 237CC934z
    Oct 11 21:50:41.179 CDT: %ALIGN-3-SPURIOUS: Spurious memory access made at 0x262ABB78z  reading 0x8
    Oct 11 21:50:41.183 CDT: %ALIGN-3-TRACE: -Traceback= 0x262ABB78z 0x216C5CC8z 0x222B4954z 0x222B64D0z 0x222B7D6Cz 0x22494B18z 0x2249B018z 0x224AA3E0z 
    Oct 11 21:50:41.183 CDT: %ALIGN-3-TRACE: -Traceback= 0x262ABAB0z 0x262ABB8Cz 0x216C5CC8z 0x222B4954z 0x222B64D0z 0x222B7D6Cz 0x22494B18z 0x2249B018z 
    Oct 11 21:50:41.183 CDT: %ALIGN-3-TRACE: -Traceback= 0x262ABAC4z 0x262ABB8Cz 0x216C5CC8z 0x222B4954z 0x222B64D0z 0x222B7D6Cz 0x22494B18z 0x2249B018z 
    Oct 11 21:50:41.183 CDT: %ALIGN-3-TRACE: -Traceback= 0x262ABF48z 0x216C5D3Cz 0x222B4954z 0x222B64D0z 0x222B7D6Cz 0x22494B18z 0x2249B018z 0x224AA3E0z 
    Oct 11 21:50:41.183 CDT: %ALIGN-3-TRACE: -Traceback= 0x262ABF54z 0x216C5D3Cz 0x222B4954z 0x222B64D0z 0x222B7D6Cz 0x22494B18z 0x2249B018z 0x224AA3E0z 

     
    bridge irb

    !
    interface GigabitEthernet0/0
     description <<Primary LAN>>
     encapsulation dot1Q 10
     no ip route-cache
     bridge-group 10

     duplex auto
     speed auto

    !


    interface SM1/0
     ip address 10.163.31.50 255.255.255.192
     service-module ip address 10.163.31.51 255.255.255.192
     !Application: SRE-V Running on SMV
     service-module ip default-gateway 10.163.31.50
     service-module mgf ip address 10.163.31.150 255.255.255.192
     service-module mgf ip default-gateway 10.163.31.145
    !         
    interface SM1/1
     description Internal switch interface connected to Service Module
     switchport mode trunk
    !         

    interface Vlan1
     ip address 10.163.31.145 255.255.255.192
    !         
    interface Vlan10
     no ip address
     bridge-group 10
    !         
    interface BVI10
     ip address 10.162.16.1 255.255.240.0
     ip helper-address 10.10.230.99
    !      

    bridge 10 protocol ieee
    bridge 10 route ip
     

    Subject: RE: Bridging GIG0/0 to ESXi
    Replied by: SCOTT PITTS on 11-10-2011 04:28:58 PM
    Some additional information is that VLAN10 will not come active unless I use no autostate. The SM1/1 is in TRUNK but not getting VLAN10 even when it is forced up with the no autostate command. This is true on SRE versions on 1.0.1, 1.0.2, and 1.5.1.0.


    Port      Mode         Encapsulation  Status        Native vlan
    SM1/1     on           802.1q         trunking      1

    Port      Vlans allowed on trunk
    SM1/1     1-4094

    Port      Vlans allowed and active in management domain
    SM1/1     1

    Port      Vlans in spanning tree forwarding state and not pruned
    SM1/1     1

    Subject: RE: Bridging GIG0/0 to ESXi
    Replied by: Brett Tiller on 11-10-2011 05:26:00 PM
    Hi Scott,

    I just wanted to get some clarification regarding your topology and design first, and have a few questions below. 

    1. Is there a reason why you are not just sending packets over the vlan rather than bridging the Gigabit interface to the vlan?  In our recommended configurations BVI is not required or utilized.


    2.  Also I think you've upgraded to SRE-V 1.5.1 correct?  In that case, is there a reason why you are still using the mgf cli?


    3. Do you intend to use the EHWIC card? The recommended configuration utilizes the EHWIC card as it offloads packets from the router cpu.


    Thanks,

    Brett

    Subject: RE: Bridging GIG0/0 to ESXi
    Replied by: SCOTT PITTS on 12-10-2011 04:45:20 PM
    1. GIG0/0 is a switchport interface not a trunk interface so I need some way to bridge VLAN10 to GIG0/0. VLAN10 will not come active when the only trunk is SM1/1. We tried making VLAN10 ip-unnumbered gig 0/0 but that failed.

    2. Most of the deployed SRE's are version 1.0.2 and migration is not the easiest from there to 1.5.1.0 raid1.

    3. We do use the EHWIC card at some facilities and they work just fine and VLAN10 behaves as expected and we have VM's working at those location on the BVI10 subnet. Without the EHWIC, there are kernel panics, routing perfomance becomes impacted and the router pauses on some commands like term mon and config t.
     

    Subject: RE: Bridging GIG0/0 to ESXi
    Replied by: Brett Tiller on 13-10-2011 07:03:00 PM
    Hi Scott,

    I've been discussing your configuration with other team members and we have the following questions below.  Essentially we want to get a very clear understanding as to why your using BVI.

    1. You mentioned that the router Gig0/0 is a switchport interface.  Typically the switch trunks tagged vlan data to the router which means that the Gig0/0 interface is essentially a trunked interface as well.  Maybe you could describe your topology which then might explain why the Gig0/0 interface is not a trunked interface.


    2.  Is there any special reason why you are not comfortable with router terminating the VLAN and MAC layer?


    3.  Are you using BVI due to issues with the Vlan10 interface not coming up and/or the point-to-point configuration failing?


    4.  When using EHWIC cards it sounds like your configuration works as desired, and that you're having issues only when not using this card?


    Thanks,

    Brett

    Subject: RE: Bridging GIG0/0 to ESXi
    Replied by: SCOTT PITTS on 14-10-2011 09:37:35 AM
    1. Our routers are co-managed so they did not want to manage a trunked interface so GIG0/0 is the Interface they manage. This typically goes to a Cisco switch which is on VLAN1. VLAN1 causes a conflict with the router and the SRE in the versions we mainly run so we created VLAN10 for the local-lan (aka what the switch thinks is VLAN1). This created the need to bridge the VLAN10 interface to GIG0/0. We simply took the example that Cisco provided for the EHWIC card since it looked like the same desired outcome and applied it and expected it to work. Without the EHWIC, things just get random. I tried plain bridging but the VLAN10 does not want to come up with just the SM1/0 being a trunk and does not show VLAN10 being trunked. So if I tag as VLAN10 to the switch then everything is wrong on that side and VLAN1 is required on the SRE build 1.0.2, so it is catch22.

    2. I do not understand this question.

    3. I used the BVI because it is the only way I knew to make it work. If you have another way I welcome that.

    4. EHWIC implementation is working great and behaves as desired. Router not happy with BVI configuration without the EHWIC. GIG0/0/BVI10/VLAN10 is configured exactly the same with or without the EHWIC card.

    Subject: RE: Bridging GIG0/0 to ESXi
    Replied by: Brett Tiller on 14-10-2011 06:52:40 PM
    Scott,

    So your use case then is that you want to span the vlan across routers which is why you're using BVI correct?  That would answer question 2 which was asking why not terminate the vlan at the router.

    I set up and tested a working configuration which I've provided below.  Vlan10 iniitially did not come up which I resolved by directly adding it the vlan database via the commands below. 

    #>vlan database
    2951Router(vlan)#vlan 10


    Since in the router interface gig0/0.10 you are using the statement 'encapsulation dot1q 10', vlan 10 needs to be set up on the switch which I did.   A snippet of the switch configuration is below:

    interface fastethernet 0/24
    switchport access vlan 10

    interface fastethernet 0/18
    switchport mode trunk
    switchport trunk encapsulation dot1Q


    The router interface snippet is below. As I mentioned this configuration worked for me, so please take a look and let me know if it doesn't match your configuration/design and why.

    bridge irb 

    interface GigabitEthernet0/0.10                                                
    encapsulation dot1Q 10                                                        
    no ip route-cache                                                             
    bridge-group 10

    interface SM2/0                                                                
    ip address 10.163.31.50 255.255.255.192                                                                                                        
    service-module ip address 10.163.31.51 255.255.255.192                        
      service-module ip default-gateway 10.163.31.50                                
    service-module mgf ip address 10.163.31.150 255.255.255.192                   
    service-module mgf ip default-gateway 10.163.31.145                           
                                                                                   
    interface SM2/1                                                                
    description Internal switch interface connected to Service Module             
    switchport mode trunk                                                         
    no ip address                                                                 
    !
    interface Vlan10                                                               
    no ip address                                                                 
    bridge-group 10                   

    interface BVI10                                                                
    ip address 10.162.16.1 255.255.240.0                                          
    ip helper-address 10.10.230.99

    bridge 10 protocol ieee                                                        
    bridge 10 route ip