Category: Routing

AWS or GCP IPSec Tunnels with BGP routing on a FortiGate software version 6.x

To use BGP routing on an AWS or GCP VPN connection, the tunnel interface needs to have its IP address assigned as a /32 and then the remote IP specified:

config system interface
    edit "GCP"
        set vdom "root"
        set ip 169.254.0.2 255.255.255.255
        set type tunnel
        set remote-ip 169.254.0.1 255.255.255.255
        set interface "wan1"
    next
end

BGP can be configured under the GUI in Network -> BGP in most cases, but the CLI has additional options. Here’s an example config for a peer 169.254.0.1 with ASN 64512, announcing the 192.168.1.0/24 prefix. 

config router bgp
    set as 65000
    set router-id 192.168.1.254
    set keepalive-timer 10
    set holdtime-timer 30
    set scan-time 15
    config neighbor
       edit "169.254.0.1"
           set remote-as 64512
       next
    end
    config network
        edit 1
            set prefix 192.168.1.0 255.255.255.0
        next
    end
Advertisement

CheckPoint Dedicated Management Route

New feature (finally!) in R80.30 is the ability to enabled Management data plane Separation, in order to have a separate route table for the management interface and all management related functions (Policy installation, SSH, SNMP, syslog, GAIA portal, etc).

Let’s assume the interface “Mgmt” has already been set as the management interface with IP address 192.168.1.100 and wants default gateway 192.168.1.1, and “eth5” has been setup as the dedicated sync interface:

set mdps mgmt plane on
set mdps mgmt resource on
set mdps interface Mgmt management on
set mdps interface eth5 sync on
add mdps route 0.0.0.0/0 nexthop 192.168.1.1
save config
reboot

After the box comes up you can verify the management route has been set by going in to expert mode and the the “mplane” command to enter management space:

> expert
[Expert@MyCheckPoint:0]# mplane
Context set to Management Plane
[Expert@MyCheckPoint:1]# netstat -rn
Kernel IP routing table
Destination  Gateway       Genmask         Flags MSS Window irtt Iface
169.254.0.0  0.0.0.0       255.255.255.252 U     0   0      0    eth5
192.168.1.0  0.0.0.0       255.255.255.0   U     0   0      0    Mgmt
0.0.0.0      192.168.1.1   0.0.0.0         UGD   0   0      0    Mgmt

Routes from the main route table relating to management can then be deleted, which makes the data plane route table much cleaner:

[Expert@MyCheckpoint:1]# dplane
Context set to Data Plane

[Expert@MyCheckPoint:0]# netstat -rn
Kernel IP routing table
Destination   Gateway       Genmask         Flags MSS Window irtt Iface
203.0.113.32  0.0.0.0       255.255.255.224 U     0   0      0    bond1.11
192.168.222.0 0.0.0.0       255.255.255.0   U     0   0      0    bond1.22
0.0.0.0       203.0.113.33  0.0.0.0         UGD   0   0      0    bond1.11
192.168.0.0   192.168.222.1 255.255.0.0     UGD   0   0      0    bond1.22

Route Filtering and Aggregation in Hybrid Cloud scenarios (EIGRP -> BGP)

One of the challenges of cloud is route table limits .  For example, AWS has a limit of 100 per table.  This can pose a real challenge in hybrid cloud scenarios where the on-prem infrastructure can easily support hundreds or thousands of internal routes no problem, leaving you (aka “cloud guy”) responsible for performing filtering and aggregation.

Consider this scenario:

EIGRPtoBGPredistribution

The CSR1000v learns about 150 routes via EIGRP, mostly in RFC 1918 space:

D EX 10.4.0.0/16 [170/51307520] via 10.1.4.73, 00:05:02, Tunnel100
D EX 10.5.0.0/16 [170/51307520] via 10.1.4.61, 00:05:02, Tunnel100
D EX 10.6.8.0/22 [170/51307520] via 10.1.4.12, 00:05:02, Tunnel100
D EX 192.168.11.0/24 [170/52234240] via 10.1.4.88, 00:05:02, Tunnel100
D EX 192.168.22.0/23 [170/51829760] via 10.1.4.99, 00:05:02, Tunnel100
D EX 192.168.33.0/24 [170/51824640] via 10.1.4.123, 00:05:02, Tunnel100

So obviously we need need to do some filtering or summarization before passing the routes to the AWS route tables via BGP.

The quick and simple fix: summarize the 10.0.0.0/8 & 192.168.0.0/16 prefixes on the CSR1000v:

router bgp 65000
 bgp log-neighbor-changes
 !
  address-family ipv4 
  aggregate-address 10.0.0.0 255.0.0.0 summary-only  
  aggregate-address 192.168.0.0 255.255.0.0 summary-only
  redistribute eigrp 100
  neighbor 169.254.1.2 remote-as 65100
  neighbor 169.254.1.2 activate

Upon initial examination, this seems to work great.  Only the aggregate routes are passed to the BGP neighbors:

CSR1000v#sh ip bgp nei 169.254.1.2 advertised-routes | inc (10\.|192\.168)
*>  10.0.0.0         0.0.0.0       32768 i
*>  192.168.0.0/16   0.0.0.0       32768 i

But there’s a nasty surprise when the EIGRP neighbors are reset.  The “summary-only” option briefly stops working for about 20 seconds:

CSR1000v#sh ip bgp nei 169.254.1.2 advertised-routes | inc 10\.
*> 10.0.0.0      0.0.0.0              32768 i
*> 10.4.0.0/16   10.1.4.73  51307520  32768 ?
*> 10.5.0.0/16   10.1.4.61  51307520  32768 ?
*> 10.6.8.0/22   10.1.4.12  51307520  32768 ?
*> 10.7.12.0/22  10.1.4.52  51307520  32768 ?
*> 10.8.8.0/24   10.1.4.7   51307520  32768 ?
*> 10.9.0.0/24   10.1.4.41  51307520  32768 ?
*> 10.77.0.0/16  10.1.4.8   51312640  32768 ?

This exceeds the 100 route limit, and AWS will disable the BGP peering session for 5 minutes.

One fix is use filters instead of the “summary-only” option:

router bgp 65000
 bgp log-neighbor-changes
 !
 address-family ipv4
  aggregate-address 10.0.0.0 255.0.0.0
  aggregate-address 172.16.0.0 255.240.0.0
  aggregate-address 192.168.0.0 255.255.0.0
  redistribute eigrp 100
  distribute-list prefix SUMM_RFC_1918 out
!
ip prefix-list SUMM_RFC_1918 seq 10 deny 10.0.0.0/8 ge 9
ip prefix-list SUMM_RFC_1918 seq 20 deny 172.16.0.0/12 ge 13
ip prefix-list SUMM_RFC_1918 seq 30 deny 192.168.0.0/16 ge 17
ip prefix-list SUMM_RFC_1918 seq 99 permit 0.0.0.0/0 le 32

Another solution is simply don’t do EIGRP to BGP redistribution, and instead just advertise the RFC 1918 blocks with the network statement:

router bgp 65000
 bgp log-neighbor-changes
 !
 address-family ipv4 
  network 10.0.0.0
  network 172.16.0.0 mask 255.240.0.0
  network 192.168.0.0 mask 255.255.0.0
!
ip route 10.0.0.0 255.0.0.0 Null0 254
ip route 172.16.0.0 255.240.0.0 Null0 254
ip route 192.168.0.0 255.255.0.0 Null0 254

Monitoring CPU & Memory in IOS-XE

ios-xe_cpu

One important thing to understanding in IOS-XE is the different numbers that can be returned when checking CPU and memory statistics.  There’s some very down in the weeds docs on this, but the simplest way to break it down is process vs. platform.  Processes is essentially control plane, while platform is data plane.

CPU

Processor CPU

CLI command: show processes cpu

SNMP OIDs:

1.3.6.1.4.1.9.2.1.56.0 = 5 second
1.3.6.1.4.1.9.2.1.57.0 = 1 minute
1.3.6.1.4.1.9.2.1.58.0 = 5 minute

Platform CPU

CLI command: show processes cpu platform

SNMP OIDs:

1.3.6.1.4.1.9.9.109.1.1.1.1.3.7 = 5 second
1.3.6.1.4.1.9.9.109.1.1.1.1.4.7 = 1 minute
1.3.6.1.4.1.9.9.109.1.1.1.1.5.7 = 5 minute

Note – Most platforms will be multi-core.

Memory

Processor Memory

CLI command: show processes memory

SNMP OIDs:

1.3.6.1.4.1.9.9.48.1.1.1.5.1 = Memory Used
1.3.6.1.4.1.9.9.48.1.1.1.6.1 = Memory Free

Platform Memory

CLI command: show platform resources

SNMP OIDs:

1.3.6.1.4.1.9.9.109.1.1.1.1.12.7 = Memory Used
1.3.6.1.4.1.9.9.109.1.1.1.1.13.7 = Memory Free
1.3.6.1.4.1.9.9.109.1.1.1.1.27.7 = Memory Committed

Cacti Templates

These were written for Cacti 0.8.8f

https://spaces.hightail.com/space/FoUD1PvlXA

 

Cacti Template: Temperature on Cisco ISR 4000 / ASR 1000

4351_temp

Our new Cisco ISR 4351s are deployed in a small closet with basic AC and not the best circulation,so  I wanted to make sure temperature was being trended in Cacti.

https://www.hightail.com/download/dDZHZEU2U1BoeWJMYnRVag

Finding the newer SNMP OIDs was a challenge but here they are:

$ snmpwalk -v 2c -c public isr.mydomain.com 1.3.6.1.2.1.47.1.1.1.1.2
iso.3.6.1.2.1.47.1.1.1.1.2.4 = STRING: "Temp: Temp 1"
iso.3.6.1.2.1.47.1.1.1.1.2.5 = STRING: "Temp: Temp 2"
iso.3.6.1.2.1.47.1.1.1.1.2.6 = STRING: "Temp: Temp 3"
iso.3.6.1.2.1.47.1.1.1.1.2.7001 = STRING: "Temp: Inlet 1"
iso.3.6.1.2.1.47.1.1.1.1.2.7002 = STRING: "Temp: Inlet 2"
iso.3.6.1.2.1.47.1.1.1.1.2.7003 = STRING: "Temp: Outlet 1"
iso.3.6.1.2.1.47.1.1.1.1.2.7004 = STRING: "Temp: Outlet 2"
iso.3.6.1.2.1.47.1.1.1.1.2.7005 = STRING: "Temp: CPU"
$ snmpwalk -v 2c -c public isr.mydomain.com 1.3.6.1.4.1.9.9.91.1.1.1.1.4
SNMPv2-SMI::enterprises.9.9.91.1.1.1.1.4.4 = INTEGER: 45
SNMPv2-SMI::enterprises.9.9.91.1.1.1.1.4.5 = INTEGER: 35
SNMPv2-SMI::enterprises.9.9.91.1.1.1.1.4.6 = INTEGER: 38
SNMPv2-SMI::enterprises.9.9.91.1.1.1.1.4.7001 = INTEGER: 33
SNMPv2-SMI::enterprises.9.9.91.1.1.1.1.4.7002 = INTEGER: 30
SNMPv2-SMI::enterprises.9.9.91.1.1.1.1.4.7003 = INTEGER: 37
SNMPv2-SMI::enterprises.9.9.91.1.1.1.1.4.7004 = INTEGER: 32
SNMPv2-SMI::enterprises.9.9.91.1.1.1.1.4.7005 = INTEGER: 36

The OIDs should be the same for the ASR 1000 series.  Since we’ll be replacing our 6504s likely with an ASR 1001-x pair this year, I’ll circle back and verify.

Activating Throughput License on 4351 (FL-44-PERF-K9)

Time to replace the office 2951s with 4351s.  Since the Internet pipes are 300 Mbps, we purchased the FL-44-PERF-K9 upgrades, which bump the throughput from a 200 Mbps to 400 Mbps cap.  I entered the PAK on the Cisco License Portal, installed the license, but noticed upon testing there was still a 200 Mbps limit.  The license commands indicated that the license had been installed, but not enabled/activated.

Router#show license feature 
Feature name Enforcement Evaluation Subscription Enabled RightToUse 
appxk9 yes yes no no yes 
uck9 yes yes no no yes 
securityk9 yes yes no no yes 
ipbasek9 no no no yes no 
FoundationSuiteK9 yes yes no no yes 
AdvUCSuiteK9 yes yes no no yes 
cme-srst yes yes no no yes 
hseck9 yes no no no no 
throughput yes yes no no yes 
internal_service yes no no no no

The licensing magic trick? Configure the platform to jump from 200 Mbps to 400 Mbps:

Router(config)#platform hardware throughput level 400000 
% The config will take effect on next reboot

Upon rebooting, NOW the throughput license is enabled.

Router#sh license feature
Feature name Enforcement Evaluation Subscription Enabled RightToUse 
throughput yes yes no yes yes

Router#show platform hardware throughput level 
The current throughput level is 400000 kb/s