Saturday, November 29, 2008

MPLS VPN

MPLS (Multiprotocol Label Switching) offers Layer 3 VPN services by means of MPLS core provider network. MPLS VPNs make use of another MPLS application namely MPLS unicast IP routing in the MPLS backbone. For enabling an MPLS core backbone, all routers including PE must support LDP (Label Distribution Protocol) and CEF (Cisco Express forwarding).

The essential parts of MPLS VPN are
  • Provider Router (P) : A LSR (Label Switch Router) in the core of provider network, which does not have direct link to CE. It only forwards labelled packets and ignores customer VPN routes.
  • Provider Edge Router (PE) : A LSR, that has atleast one link to a CE router and associates each CE routes to appropriate routing tables. It will also be running a control plane protocol such MP-BGP to exchange CE routes with other PEs.
  • Customer Edge Router (CE): Edge router on customer site that has no knowledge of MPLS and directly connects to a MPLS PE.

MPLS VPN working can be summed up as follows
  • CEs exchange routes with PEs.
  • PEs in turn store CE routes into its own routing table (VRF)
  • PEs advertise CE routes to only other PEs in the MPLS cloud using MP-BGPs. MP-BGP allows the addition of an address family with a traditional BGP NLRI, which will discussed more in the detail later.

Some of the other key concepts, which needs to be covered are
  • VRFs (Virtual Routing and Forwarding tables)
  • RD (Route Distinguishers)
  • RT (Route Targets)

Virtual Routing and Forwarding tables (VRF)
Each customer must see only routes belonging to the customer. This is made possible by a VRF table on an MPLS aware router such as the PE. So, each customer is associated with a VRF on the PE to which it connects. Three components make the VRF table
  • An IP routing table
  • A CEF FIB, derived from the VRF RIB
  • An interface or group of interfaces associated with the VRF
  • A separate instance of the routing protocol (per VRF) used to exchange routes with CEs

Route Distinguishers (RD)
As mentioned before PEs make use of MP-BGP to advertise customer routes to other PEs in the MPLS cloud. A normal BGP NLRI (prefix) just consists of prefix. To differniate between overlapping prefixes (when multiple customers advertise the same prefix) , MPLS deals with this by adding another number before the traditional BGP NLRI, namely an address family. BGP/MPLS IP Virtual Private networks specifies an address family for supporting IPv4 MPLS VPNs called Route Distinguishers. The new NLRI format called a VPN-V4, has a 64-bit RD and a 32-bit IPv4 prefix.

Route Targets (RT)
When say PE1 advertises certain routes to PE2, it also advertises Route Targets as BGP extended community attributes. By associating a Route-Target with a particular a route advertisement, PE2 could dertermine into which VRF the associate route must go. As we'll see in the configuration examples later, each VRF has an export and import Route-Target statement. A simple pseudocode below would make this clearer

PE1
VRF definition for Customer A
export 10
import 10


PE2
VRF definition for Customer A
export 10
import 10

When PE1 advertises a route 10.3.3.0/24 from customer A to PE 2, it will associate a route target 10. PE2 seeing a route-target 10 with the prefix 10.3.3.0/24 and comparing the route-target value with its import statement in its definition of Customer As VRF places that particular prefix in customer As VRF.

Configuration Example

Keeping these concepts in mind, lets move onto the configuration details. I'd highly recommend reading more on MPLS VPN control plane and data plane, BGP and learning how to configure basic MPLS IP unicast forwarding.
Here comes the fun part. Take a look at the above topology and lets make list of what needs to be implemented. The dynagen file for the above topology is given below.

1) Configure Routing Protocols on the Customer Edge Routers (CE-A1 , CE-A2 , CE-B2 ,CE-B2)
2) Define VRFs on the PE Routers for the customers and associating VRF interfaces on the PE Routers (PE1 and PE2)
3) Configuring Routing protocols between PE routers and CE routers.
4) Configure the Provider Network
5) Verification


1) Configure Routing Protocols on the Customer Edge Routers
If you're reading how to configure MPLS VPNs, I can safely assume you already know this part. However, below is configuration example for one of the customer edge routers (CE-A2). RIP is the routing protocol run on the CEs

CE-A2#sh run | be rip
router rip
version 2
network 10.0.0.0
network 12.0.0.0
network 172.31.0.0
no auto-summary

2) Define VRFs on the PE Routers for the customers and associating VRF interfaces on the PE Routers (PE1 and PE2)

As mentioned before, VRFs need to be defined only on PEs. P routers have no idea of VRFs or customer routes, it simply forwards based on labels. CE-A1, CE-A2 are customer A sites and CE-B1, CE-B2 are customer B sites.


PE1

ip vrf customerA
rd 1.1.1.1:1
route-target export 1:100
route-target import 1:100
!
ip vrf customerB
rd 1.1.1.1:2
route-target export 2:200
route-target import 2:200

#Associate the interface that connects to customer A and customer B to the appropriate VRF

interface Ethernet0/0
ip vrf forwarding customerA
ip address 172.31.241.1 255.255.255.0
!
interface Ethernet0/1
ip vrf forwarding customerB
ip address 172.31.242.1 255.255.255.0
PE2

ip vrf customerA2
rd 3:200
route-target export 1:100
route-target import 1:100
!
ip vrf customerB2
rd 4:200
route-target export 2:200
route-target import 2:200

#Associate the interface that connects to customer A and customer B to the appropriate VRF

interface Ethernet0/0
ip vrf forwarding customerA2
ip address 172.31.243.1 255.255.255.0
!
interface Ethernet0/1
ip vrf forwarding customerB2
ip address 172.31.244.1 255.255.255.0



As seen from the above configuration, the VRF names for customer A is customerA on PE1 and customerA2 on PE2. This is done intentionally to show that customer VRF names are only locally significant. The routes exported from one PE and imported into another PE into a particular VRF definition solely relies on route-target values.

3) Configuring Routing protocols between PE routers and CE routers.


Now that VRFs have been defined for each customer, we can configure dynamic routing protocols between PE and CE routers. We go into the same RIP configuration (ignore the sections in green for now). Using the address-family ipv4 vrf command, we run dynamic routing protocols between PE and CE. Please note that RIP has already been configured on CEs in the first step.

PE1#sh run | be rip
router rip
version 2
network 192.168.1.0
no auto-summary
!
address-family ipv4 vrf customerB
network 172.31.0.0
no auto-summary
version 2
exit-address-family
!
address-family ipv4 vrf customerA
network 172.31.0.0
no auto-summary
version 2
exit-address-family
!

PE2#sh run | be rip
router rip
version 2
network 192.168.2.0
no auto-summary
!
address-family ipv4 vrf customerB2
network 172.31.0.0
no auto-summary
version 2
exit-address-family
!
address-family ipv4 vrf customerA2
network 172.31.0.0
no auto-summary
version 2
exit-address-family
!



Lets do some verification, we can indeed see the routes from CE-A1 on PE1 and CE-A2 on PE2.

PE1#sh ip route vrf customerA

Routing Table: customerA
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

172.31.0.0/24 is subnetted, 1 subnets
C 172.31.241.0 is directly connected, Ethernet0/0
182.2.0.0/24 is subnetted, 1 subnets
R 182.2.2.0 [120/1] via 172.31.241.2, 00:00:13, Ethernet0/0


PE2#sh ip route vrf customerA2

Routing Table: customerA2
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

172.31.0.0/24 is subnetted, 1 subnets
C 172.31.243.0 is directly connected, Ethernet0/0
10.0.0.0/24 is subnetted, 1 subnets
R 10.3.3.0 [120/1] via 172.31.243.2, 00:00:09, Ethernet0/0
12.0.0.0/24 is subnetted, 1 subnets
R 12.3.3.0 [120/1] via 172.31.243.2, 00:00:09, Ethernet0/0



At this point CE-A1 will not be able to reach CE-A2, similarly CE-B1 to CE-B2, as the provider network has not yet been fully configured.


4) Configure the Provider Network


This can be further broken down into 3 steps

i) Configure routing protocols in the Provider Core
ii) Configure MPLS in Provider Core
iii) Configure MP-BGP on PEs to exchange VRFs


i) Configure routing protocols in the Provider Core
Again this is a simple step. We'll be running RIP in the provider network.

PE1#sh run | be rip
router rip
version 2
network 192.168.1.0
no auto-summary

P1#sh run | be rip
router rip
version 2
network 192.168.1.0
network 192.168.2.0
no auto-summary

PE2#sh run | be rip
router rip
version 2
network 192.168.2.0
no auto-summary


ii) Configure MPLS in Provider Core
For configuring MPLS in the core, CEF must be enabled on all provider core routers (PE1, P1,PE2). Also a Label Distribution protocol needs to be configured (LDP or TDP). In our example here, we'll be using LDP.

PE1(config)#ip cef
PE1(config)#mpls ip
PE1(config)#mpls label protocol ldp

ip route 2.2.2.2 255.255.255.255 192.168.1.2
ip route 3.3.3.3 255.255.255.255 192.168.1.2

#enable mpls ip for the interfaces in the provider core

PE1(config)#int e0/2
PE1(config-if)#mpls ip


P1(config)#ip cef
P1(config)#mpls ip
P1(config)#mpls label protocol ldp

ip route 1.1.1.1 255.255.255.255 192.168.1.1
ip route 3.3.3.3 255.255.255.255 192.168.2.1

#enable mpls ip for the interfaces in the provider core

P1(config)#int e0/0
P1(config-if)#mpls ip

P1(config)#int e0/1
P1(config-if)#mpls ip


Verification Commands
Verification Commands
PE1#sh mpls ldp neighbor
Peer LDP Ident: 2.2.2.2:0; Local LDP Ident 1.1.1.1:0
TCP connection: 2.2.2.2.55086 - 1.1.1.1.646
State: Oper; Msgs sent/rcvd: 48/47; Downstream
Up time: 00:35:33
LDP discovery sources:
Ethernet0/2, Src IP addr: 192.168.1.2
Addresses bound to peer LDP Ident:
192.168.1.2 2.2.2.2 192.168.2.2
P1#sh mpls ldp neighbor
Peer LDP Ident: 3.3.3.3:0; Local LDP Ident 2.2.2.2:0
TCP connection: 3.3.3.3.18852 - 2.2.2.2.646
State: Oper; Msgs sent/rcvd: 48/48; Downstream
Up time: 00:35:58
LDP discovery sources:
Ethernet0/1, Src IP addr: 192.168.2.1
Addresses bound to peer LDP Ident:
192.168.2.1 3.3.3.3
Peer LDP Ident: 1.1.1.1:0; Local LDP Ident 2.2.2.2:0
TCP connection: 1.1.1.1.646 - 2.2.2.2.55086
State: Oper; Msgs sent/rcvd: 48/48; Downstream
Up time: 00:35:52
LDP discovery sources:
Ethernet0/0, Src IP addr: 192.168.1.1
Addresses bound to peer LDP Ident:
192.168.1.1 1.1.1.1
PE2(config)#ip cef
PE2(config)#mpls ip
PE2(config)#mpls label protocol ldp

ip route 1.1.1.1 255.255.255.255 192.168.2.2
ip route 2.2.2.2 255.255.255.255 192.168.2.2

#enable mpls ip for the interfaces in the provider core

PE2(config)#int e0/2
PE2(config-if)#mpls ip


In the topology, all provider core routers are configured with a loopback interface. When enabling mpls, just like OSPF, MPLS routers are known by an ID known as MPLS LDP router-id. The following steps are considered when selecting MPLS LDP router-id on a router
1) All IP addresses on active interfaces
2) Of these interface addresses, loopback interface configured with an IP address gets higher priority.

Now since loopback interfaces are configured on Provider core routers (PE1, P1, PE2), enabling MPLS automatically sets these loopback addresses as LDP router-ids. And because of that the routers need to know how to reach those loopback addresses. Hence those static routes are added, alternative those loopback addresses could be advertised through RIP. Also when configuring MP-BGP in the next step, these loopback addresses will be used in to form neighbour relationships.

iii) Configure MP-BGP on PEs to exchange VRFs

Lets walkthrough a configuration example on PE2. When configuring BGP apart from neigbour statement, a new command namely address-family vpn4 needs to be configured to advertise VPNv4 addresses between PEs. Recall when we discussed Route Distinguishers, VPNv4 addresses also include a 64bit route-distinguisher in addition to the 32bit IPv4 prefix


PE2(config)#router bgp 64513
PE2(config-router)#bgp log-neighbor-changes
PE2(config-router)# neighbor 1.1.1.1 remote-as 64513
PE2(config-router)# neighbor 1.1.1.1 ebgp-multihop 2
PE2(config-router)# neighbor 1.1.1.1 update-source Loopback0
PE2(config-router)# no auto-summary

#Activates the advertisement of the IPv4 address family.
PE2(config-router)# neighbor 1.1.1.1 activate

#The previous line automatically added the following lines to BGP configuration (address family ipv4)

PE2#sh run | be bgp
router bgp 64513
bgp log-neighbor-changes
neighbor 1.1.1.1 remote-as 64513
neighbor 1.1.1.1 ebgp-multihop 2
neighbor 1.1.1.1 update-source Loopback0
!
address-family ipv4
neighbor 1.1.1.1 activate
no auto-summary
no synchronization
exit-address-family
!

#Defines IBGP parameters for VPNv4 NLRI exchange
PE2(config-router)#address-family vpnv4 unicast
PE2(config-router-af)#neighbor 1.1.1.1 activate

*Mar 1 01:30:58.911: %BGP-5-ADJCHANGE: neighbor 1.1.1.1 Down Address family activated
*Mar 1 01:31:01.015: %BGP-5-ADJCHANGE: neighbor 1.1.1.1 Up

# Notice VRFs have automatically been added BGP

PE2#sh run | be bgp
router bgp 64513
bgp log-neighbor-changes
neighbor 1.1.1.1 remote-as 64513
neighbor 1.1.1.1 ebgp-multihop 2
neighbor 1.1.1.1 update-source Loopback0
!
address-family ipv4
neighbor 1.1.1.1 activate
no auto-summary
no synchronization
exit-address-family
!
address-family vpnv4
neighbor 1.1.1.1 activate
neighbor 1.1.1.1 send-community extended
exit-address-family
!
address-family ipv4 vrf customerB2
no synchronization
exit-address-family
!
address-family ipv4 vrf customerA2
no synchronization
exit-address-family
!


The configuration for PE1 is similar to the one above, hence not shown.

With this we complete the configuration of the provider network. Of course we need to redistribute CE RIP routes into BGP so PEs can exchange customer routes.
PE2#sh run | be bgp

router bgp 64513
#some lines deleted
address-family ipv4 vrf customerB2
redistribute rip
no synchronization
exit-address-family
!
address-family ipv4 vrf customerA2
redistribute rip
no synchronization
exit-address-family
!

PE1#sh run | be bgp

router bgp 64513
#some lines deleted
address-family ipv4 vrf customerB
redistribute rip
no synchronization
exit-address-family
!
address-family ipv4 vrf customerA
redistribute rip
no synchronization
exit-address-family
!



6) Verification

By examining the routing table for CustomerA, on PE1 we see that it is learning routes from CE-A2 through PE2. Similarly for CustomerB on PE2. Please note that since we're not redistributing BGP into RIP on PEs, CE will not see routes from other CEs. I'd rather add a default route on each CE pointing to the PE. Also, I thought it would be better to leave somethings for you to play with.

PE1#sh ip route vrf customerA

Routing Table: customerA
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

172.31.0.0/24 is subnetted, 2 subnets
B 172.31.243.0 [200/0] via 3.3.3.3, 00:04:52
C 172.31.241.0 is directly connected, Ethernet0/0
10.0.0.0/24 is subnetted, 1 subnets
B 10.3.3.0 [200/1] via 3.3.3.3, 00:04:52
182.2.0.0/24 is subnetted, 1 subnets
R 182.2.2.0 [120/1] via 172.31.241.2, 00:00:10, Ethernet0/0

PE2#sh ip route vrf customerB2

Routing Table: customerB2
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

172.31.0.0/24 is subnetted, 2 subnets
B 172.31.242.0 [200/0] via 1.1.1.1, 00:06:05
C 172.31.244.0 is directly connected, Ethernet0/1
10.0.0.0/24 is subnetted, 1 subnets
R 10.3.3.0 [120/1] via 172.31.244.2, 00:00:03, Ethernet0/1
183.2.0.0/24 is subnetted, 1 subnets
B 183.2.2.0 [200/1] via 1.1.1.1, 00:06:05

I hope this has been informative for you.

References
CCIE Routing and Switching Exam Certification guide
Cisco IOS Multiprotocol Label Switching Configuration Guide,

Dynagen Configuration
autostart=false
[localhost:7200]

workingdir = /opt/dynamips/dynagen-0.11.0/testlabs/mplsvpnworking

[[3640]]
# Specify 3640 IOS image on Linux here:
image = /opt/dynamips/images/c3640-jk9o3s-mz.124-10c.image
#
ram = 128
disk0 = 0
disk1 = 0
# Choose an idlepc value from the below
# idlepc = 0x604fd214
idlepc = 0x603e16e0

mmap = true
# ghostios = true


###########################
#
# Define router instances
#
###########################

#CE-A1
[[Router R1]]
model = 3640
console = 2001
autostart = false
slot0 = NM-4E
E0/0 = R3 E0/0


#CE-B1
[[Router R2]]
model = 3640
console = 2002
autostart = false
slot0 = NM-4E
E0/0 = R3 E0/1

#PE1
[[Router R3]]
model = 3640
console = 2003
autostart = false
slot0 = NM-4E
slot1 = NM-4T
E0/2 = R4 E0/0


#PE2
[[Router R4]]
model = 3640
console = 2004
autostart = false
slot0 = NM-4E
E0/1 = R5 E0/2

#PE3
[[Router R5]]
model = 3640
console = 2005
autostart = false
slot0 = NM-4E
E0/0 = R6 E0/0
E0/1 = R7 E0/0

#CE-A2
[[Router R6]]
model = 3640
console = 2006
autostart = false
slot0 = NM-4E

#CE-B2
[[Router R7]]
model = 3640
console = 2007
autostart = false
slot0 = NM-4E


1 comment:

Anonymous said...

I really dig your article! It is very well written, and extremely helpful. Good Stuff!