Update:

First time ever I met Yakov Rekhter!!!

MTU made me cry :-)

MPLS

  MPLS:
  MPLS VPN Troubleshooting:
  MPLS-TE:


MPLS:

MPLS Advantage or Why to use MPLS: Explain advantage of MPLS Layer-2/3 VPN, MPLS-TE & VPLS as well as explain BGP free core.

LABEL: 4 byte in size:  20 bit label/3 bit EXP/ 1 bit of S bit/ 8 bit TTL.

POP function: It can be done using implicit-null label (PHP Process) (explicit-null label is used to preserver the QOS value)

Label Distribution Methods:

  1. LDP
  2. TDP
  3. MP-BGP (RFC 3107)
  4. RSVP

LDP/TDP Router id: loopback address or highest interface IP address

Ldp discovery transport-address: by default LDP/TDP router-id will be a LDP transport address. But in some cases if routers do not have reachability to each other LOOPBACK address then they are not going to build up LDP/TDP neighbor relationship. At that time you will have to use mpls ldp discovery transport-address command and you will have to specify the directly connected interface IP address to build LDP/TDP neighbor relationship. This is same as BGP update-source deployment.

Cell MODE MPLS: Exchanging label between IP ATM switch through VPI/VCI.

MPLS label written into ATM cell.

ATM use VPI/VCI 0/32 for control information exchange for routing protocol as well as label distribution protocol, We can configure different VPI/VCI control VC.

Enabling MPLS under ATM interface then we are running MPLS inside the PVC which is called Frame Mode. MPLS control message or data packet will be encapsulated into cell and then send it over the VC.

But when you create ATM subinterface with mpls (interface ATM0/0.1 mpls) then it will called Cell mode MPLS.

Cell Mode ATM Configuration:

Interface ATM0/3.1 mpls

Ip address 150.1.12.2 255.255.255.0

PVC 0/102

mpls atm control-vc 201

Frame Mode: label will be inserted between L-2 and L-3 header.

Cell Mode: VPI/VCI file are used as label as well as for label switching also. ATM LSR connot forward IP Packets.

 

ATM LSR ( Those are the routers between EDGE PE or Edge ATM LSR): They can only forward the router.

ATM edge LSR: Segment packets into cell and forward them into MPLS ATM domain as well as reassemble cells into packet.

 

Per-platform label allocation is default in Frame-mode MPLS.

Benefits of per-platform label allocation:

1. Smaller LFIB          

2. Faster label exchange. Disadvantage is less secure. LDP-ID= 1.1.1.1:0; 0 after: is indicating per-platform allocation mode.

 

Per-interface label allocation is default in Cell-mode MPLS. Therefore if single router has two parallel link to same ATM switch there will be two LDP sessions are established and they will have separate labels allocation.

 

LDP session between ATM LSRs:

  1. An IP adj between AMT LSR is established over control vc (0/32)
  2. Control VC also used for LDP.

 

CEF Default:

When standard CEF or distributed CEF operations are enabled globally, all interface that support CEF are enabled by default.

Frame Mode MPLS in ATM network:

When ATM infrastructure is not capable to support cell mode MPLS then you will have to build ATM PVC between the both end routers separated by ATM switch network ( which is not capable of running cell-mode MPLS.) Here both the end router will establish there LDP/TDP neighbor relationship with each other, not with any of the ATM switch in network. It is like you will have to crate Layer-2 tunnel of ATM PVC between two end points routers through ATM network and LDP/TDP messages will be travel through that tunnel transparently. You can have multiple LSP passing over one ATM PVC (in case of MPLS TE you can have multiple LSP).

In Cell mode MPLS: Cisco use VPI value of 1 by default (“mpls atm vpi” is used to change the value), control vc 0/32 by default (“mpls atm control-vc” is used to change the value, but both end should have same configuration of this parameter). Uses the MPLS type for the sub-interface (interface atm 0/0.2 mpls), “mpls ldp maxhop “ command is used for LDP loop-detection, “mpls ldp atm vc-merge” is used to enable VC merge by default VC merge is enabled on ATM switches that support VC merge functionality.

ATM Cell mode can be used where all atm devices in the path are capable of running layer-3 routing as well as supporting cell mode ATM.

Frame mode MPLS over ATM is used where two ATM MPLS routers are seperated by the cloude of ATM switches which are not supporting any layer-3 functionality and MPLS, these switches can only understand VPI/VCI value and switch the cell using that.

 

RT:

RT is 8 byte BGP Extended community.  Format of  RT ASN:NN or IP Address:NN.

BGP VPNv4 will accept only those VPNv4routes which has exact RT configured on one of the local VRF on the router. You can disable that feature by using “no bgp default route-target filter”

BGPVPNv4 will assign unique label to each and every prefix of VRF, to verify use “show ip bgp vpn4 all label”  (In later IOS you can have option of having only One VPN label for VRF, it means all prefixes in VRF will have same label)

In “show ip cef vrf x 1.1.1.1” left hand side label is transport label and right hand side label is VPN label

R1#show ip cef vrf GREEN 22.22.22.22  

  22.22.22.22/32, version 9, epoch 0, cached adjacency to Serial0/0/0.101  

  0 packets, 0 bytes  

  tag information set  

 local tag: VPN-route-head  fast tag rewrite with Se0/0/0.101, point2point, tags imposed: {18<-LDP Label 24<-VPN Label

show ip bgp vpn4 all <ip prefix>” will give you nexthop/RD(VPNv4)/RT information for given prefix. This command will also show you any other extended or standard community attached to the prefix(route).

show ip bgp vpnv4 all label” will give show you the VPN label associated with the appropriate route of particular VRF.

In MPLS VPN PE-CE protocol metrics will be converted to BGP MED during redistribution from IGP to SP MP-BGP and BGP MED will be converted to the IGP metrics during redistribution from SP MP-BGP to PE-CE IGP.

MPLS VPN Troubleshooting:

1. Routes are not showing up in PE VPNv4 table

            1. Check for RT under VRF configuration of LOCAL ROUTER and REMOTE ROUTER

            2. Check for the VRF IGP to BGP VRF address family redistribution on REMOTE ROUTER

            3. Check for the NEXT-HOP-SELF configuration in IBGP or EBGP configuration of REMOTE router

4. Check the REMOTE ROUTER IP ROUTE VRF table or CE-PE routing protocol configuration on REMOTE ROUTER.

            5. Check for VPNv4 session between the REMOTE and LOCAL Router

                        a. check for proper neighbor configuration

                        b. check for update-source if BGP is between LOOPBACK addresses

                        c. Check for ebgp-multihop configuration (MPLS VPN option C)

                        d. Check the activation of neighbor under BGP address-family configuration

                        e. Check if you need local-as configuration option

 

            6. Check for CE-PE configuration on both end if BGP is the CE-PE protocol

As BGP use different way to advertise the routes and each one need specific component in routing or bgp table

                        1. Network command

                                    a. Network command only

b. Network command with mask (Needs specific component in Routing table of same mask, use static route to achieve this)

                        2. Aggregate command (Need component in BGP table)

3. Redistribute static/connected/IGPs (Check for prefix-list or route-map associated with that)

4. Check neighbor configuration command syntax for Prefix-list or route-map configuration

                        5. Allowas-in on CE routers and AS-OVERRIDE on PE routers

                        *7. no BGP Route-target filter                        

                      

2. Routes are not showing up in CE Routing table

                        1. Check for the next-hop info in VPNv4 table of LOCAL ROUTER and in IP Routing table

                        2. Check for redistribution of BGP routes into the CE-PE table of local PE

                        3. Check for passive interface config on both PE and CE routes

                        4. Check for area connectivity in OSPF and configure Virtual link if needed

                        5. Check for CE-PE protocol version configuration in case of RIP

                        6. Check for BGP configuration

 

3. Ping ISSUE

                        1. Most of the issue is broken LSP

                                    a. Check for IP CEF is enabled or not

                                    b. Label distribution protocol mismatch LDP or TCP

                                    c. Make sure label protocol is enabled globally as well as on interface

                                    d. look at the LOOPBACK updates in IGP as well as label allocation for it

                                                Configure OSPF loopback with point to point network type

                                                                                    or

                                                Use loopback of /32

                                   

                                    e. check for end to end LSP using

                                                CEF table

                                                MPLS Forwarding Table

                                                LFIB

                                                BGP VPNv4 table with label

f. In MPLS VPN option B (Here BGP next hop will do the work, you can avoid the redistribution)   and C don’t forget to redistribute the BGP into IGP to generate the label for remote AS PE's LOOPBACK's in local AS ,  same way advertise local AS PE's loopback to the remote AS.

 

g. Check for MPLS configuration between both the AS if you are using the Option B with Multihope MP-EBGP between the  loopback interface of ASBR's of two different PE's of respective AS.

 

 

4. Desire Path for traffic:

            a. next-hope-unchange (MPLS VPN option C)

            b. Local Pref on local router

            c. MED on remote router

            d. redistribution with different metrics on differnet PEs facing to same Cutomer's CEs

            e. OSPF SHAM-LINK with metrics manipulations

 

5. Loop prevention in MPLS VPN:

            a. BGP uses SOO

                        Use with route-map

                        ip vrf sitemap

            b. EIGRP uses SOO

                        ip vrf sitemap <route-map>

            c. OSPF uses DOWN-BIT (LSA-3) & TAG-BIT (LSA-5 & 7)


MPLS-TE

Need to look at:
explicate path verbatim option.

TE required things:
1. ip cef
2. loopback with /32
3. mpls traffic-eng tunnel on global and interface configuration
4. ip rsvp bandwidth on interface configuration mode
5. Remember to use metric style wide with MPLS-TE (TLV 135 IP Extended TLV, provide the metric value up to 2^24.

Bandwidth allocation:

You can reserve bandwidth on interface by using "ip rsvp bandwidht" and you can verify that using show ip rsvp interface.
In show ip rsvp interface i/f max is the bandwidth that is configured and reserve by ip rsvp bandwidth command under interface configuration. whereas allocated bandwidth is the amount of bandwidth requested or reserve by MPLS-TE which is configured using tunnel mpls traffic-eng bandwidth command.
If you will not define ip rsvp bandwidth value, then router will reserve 75% of max-reservale bandwidth of an interface.
If "ip rsvp bandwidth" command is missing from the interface then there will not be any reservation of bandwidth so tunnel will not come up at all.

Priority of tunnel:

To assign priority to important tunnel mpls-te uses the two type of priority.
1. Setup priority
2. Hold priority
lower the priority higher the preference, priority value can be from 0 to 7.
How priority works: lets say if we have tunnel passing from router has priority of 5 and other tunnel has priority of 3 that need to be established and router doesn’t have any bandwidth to allocate to the tunnel with the setup priority 3 then router will tear down the tunnel with priority value of 5. Cisco IOS will not let you configure setup priority less than the hold priority.

Constrains you can configure in MPLS TE:
1. resolvable bandwidth
2. Administrative weight (same as IGP metric but use for MPLS-TE database)
3. Attributes/affinity

4.      Setup and hold priority.

Flooding:
IGP flooding occur at link change, configuration change and at periodically time to refresh the database.
What things need to be flood in order to keep MPLS-TE LSDB up-to-date: MPLS-TE setup is highly depends on available bandwidth through out the path, so routers in the networks need to have up to date information about the available bandwidth in network. So ISIS or OSPF should flood the link information when available bandwidth changes. This might be very expensive in terms of CPU as well as bandwidth resources. Flooding will be more if you have auto bandwidth configure for MPLS-TE. You can reduce the flooding by advertising the link available bandwidth when there is certain amount of % change in bandwidth. Flood the available interface bandwidth when certain % of bandwidth went down or up. You can configure this by using "mpls traffic-eng flooding threshold {up|down} percentage value" and you can verify this by using "show mpls traffic-eng link-management bandwidth-allocation interface-type"
Flooding is different in OSPF and ISIS: OSPF flood the only the link info which has bandwidth change just because of OSPF LSA Type 10 nature. whereas ISIS flood all the router link info even if there is a change in only one link available bandwidth because of TLV 22 nature.

How flooding work if there is no any change in available (resolvable) bandwidth or there is some change in bandwidth but it hasn’t cross the threshold configured as above?
 In this case Router by default check the reservable bandwidth at every three seconds so if there is change in bandwidth but didn’t cross the threshold then router will send that update at every 3 minutes.
 If there is no change in bandwidth then router will not flood the link information even at every 3 seconds. You can change the default interval of 3 min using “mpls traffic-eng link-management timers periodic-flooding 0-3600"

CSPF:

Keep in mind that here you might not be trying to find out the shortest path to the destination. MPLS-TE is all about to utilizing your unused resources.
IGP SPF will be run for all the destination where as MPLS-TE CSPF will be run for only one destination and that is tunnel end point.
What is C in CSPF? That is nothing but constrains defined for SPF calculation. What are constrains? Those are available bandwidth, administrative weight and affinity bits.

What is Administrative weight? The definition for administrative weight differs as per requirement and implementation for data types. If you are talking about only mask data transfer then Administrative-weight might be bandwidth unit for you.
if you are talking about voice data then that might be the delay parameter for you. Administrative weight is advertised with MPLS-TE database using IGP.

Affinity bits: if you are using in this constrains in network then configured tunnel affinity bits should be match with affinity bits configured on the link throughout the path in order to consider that link as a valid link to calculate CSPF.

MPLS-TE will select only one path as a best path: there might be multiple links having same available bandwidth or same administrative-weight or affinity bits set.
For Path selection:
1. Take the path with largest available bandwidth
2. if there is tie in bandwidth then take the path with the lowest hop count
3. if there is a tie in hop count also then router will select only one path randomly.

You can tell router to use IGP metric or MPLS-TE metric (admin weight) by configuring following command:
1. To select IGP Metric: on global config mode: "mpls traffic-eng path-selection metric igp" or on tunnel configuration "tunnel mpls traffic-eng path selection metric igp"
2. To select MPLS-TE metric: in some cases you have configure router to use IGP metric for TE-tunnel setup and want to use MPLS-TE metric only for one tunnel then use "traffic-eng path selection metric te" on tunnel configuration.


Explicit Path:
You can build up explicit path using "next-address" or "list" comand under "ip explicit path name" configuration.
You can exclude the link or router using "exclude-address" command in "ip explicite path" configuration.

Tunnel re-optimization:

Re-optimization is the process to have tunnel passing from the best available path.
There are four type of re-optimization:
1. Periodic re-optimization (default at every hr or 3600sec, you can configure it from 0 to 604800, 0 means never optimize) to configure use" mpls traffic-eng tunnels re-optimize timer’s frequency 0-604800"
2. Manual Re-optimization: if you know anything change in network which can help to reroute tunnel via much better path then you can do manual re-optimization using "mpls traffice0eng re-optimize tunnel-name"
3. Event-driven: If better link came up then also you can have your tunnel re-optimized automatically. Event-driven automatically re-optimization is not good if link is flapping frequently. You can configure it using "mpls traffic-eng re-optimize events link-up"
4. LOCKDOWN: Tunnel will not optimize even if there is better path available. Re-optimization will be triggered as path recalculation if primary link goes down on which tunnel is passing through.

RSVP:
RSVP protocol # 46
RSVP is soft state protocol it means, there should be some refresh messages that refresh the reservation periodically. Path and Resv messages are used to refresh the RSVP reservation refresh once tunnel is setup.
RSVP comes into the picture once Router has completed the CSPF. RSVP does the signaling to exchange the labels.
Keep in mind bandwidth reserved by the interface using "ip rsvp bandwidth" is exchanged by the IGP.
RSVP has different messages; messages use the different type of OBJECT-CLASS to carry the control information.
USE of RSVP is to exchange the labels as well as adding the capability of MAKE before Brake (like shared explicit and FRR). MPLS or IGP doesn’t have capability to support make-before brake kind of facility available.


RSVP Messages:
Path
Resv
Path Error
Resv Error
Path Tear
Resv Tear
Resv confirm
Resvtear confirm
Hello

*path and resv messages are sent at every 30 seconds. (to avoid the all router to send path and resv message at same time 50% jitter is added every time); path and resv messages are exchange between immediate neighbor independently. Upstream router (router near to head end) will send path message to downstream router (Router near to tail end) and resv messages sent in opposite direction. If upstream router will not receive resv message from downstream router upon sending 4 path message then upstream router will consider that reservation is removed.

RSVP MPLS-TE objects: Each RSVP messages carry the particular types of object to build LSP.

1. PATH Message Objects:

    A. Label Request Object: This object request the label for establishing end to end LSP. Egress LSR and Transit LSR will generate label upon arrival of PATH message with this object.

    B. Explicit Route Object (ERO): This object has the addresses of each nodes through which LSP must path (Explicit Path option in MPLS-TE)

    C. Record Route Object (RRO): This object will record the IP add or Router ID of the router it is passing through to reach egress router. This can be useful in loop prevention. RRO is     also carrying the "Label Record desire" flag in session attribute which will be used in MPLS FRR capability for Link and NODE protections.

    D. Sender TSPec: This object will request a bandwidth reservation for LSP.

2. RESV Message Objects:

    A. Label Object: On coming back to ingress LSR this OBJECT will carry the label to establish the LSP path. This label were requested by Label Request Object in PATH Message.

    B. Record Route Object (RRO): Same function as RRO in PATH messages.

LDP vs RSVP vs BGP:

               1. LDP (RFC5036)   :   independent & liberal retention mode, easy to configure, easy to add node in network, depends on IGP, need Targeted session or LDP Session protection to avoid                                                  any traffic black holing or looping when there is a change in IGP best path, not suitable for inter AS LSP setup. use UDP to discover neighbor and TCP to build                                                      session, use hellos as a keepalive to maintain the session, hellos are sent on 224.0.0.2 (all multicast Router address), Targeted hellos, session protection does                                                      need IP rechability between end routers, Targeted hellos are unicast, incremental updates for label binding, updates are reliable because of TCP session. Use                                                      TCP/UDP port 646 (TDP use TCP/UDP 711). TLV based protocol easy to expand.

               2. RSVP (RFC3209)  : Good to provide guaranteed bandwidth & fast fail over or path repair (MPLS FRR), create large number of session and need to maintain large number of sessions on                                                 router, provisioning need more configuration, addition of new node needs configuration on head end as well as local router. constrain based LSP computation, Head                                              end only will decide the LSP Path.

               3. BGP                   : Useful in inter-AS LSP establishment, most of the MPLS service inner label exchanged by MPBGP (exceptional Draft Martini based L2 VPN), Easy to extend or add                                                 new capability.

RFC Lists:

MPSL VPN        RFC 2547 & 4364 (new)                                                             IETF Working Group
BGP                 RFC 1771 & 4271                                                                     IETF Working Group
L2VPN               RFC 4761(BGP) & 4762(LDP) & Frame Work (4664)                         IETF Working Group
MPLS TE
          RFC 2702                                                                                IETF Working Group


RSVP FRR:

In case of backup (Protected) LSP exists then PLR will report the local protection available to headend.

In case of failure PLR will signal the PATH ERROR with ERROR_SPEC to head end that tunnel is locally repaired so that head end will not tear down the TE tunnel.

Locally protected link or node LSP will be up since you provision the FRR or Local link or node protection, it will not be signaled upon link failure, its already up. Once local protection is in use, headend will try to recalculate or reroute the LSP via alternate or another backup path. Locally protected LSP will remain in use till headend reroute the LSP to backup or alternate path, if headend fails to reroute or establish backup LSP, link protection will be used continuously and traffic will flow end to end and at the same time headend will keep trying to signal the LSP via alternate or backup path at periodic interval.


JUNOS MPLS timers:

retry-timer:
    Time between attempts to bring up failed primary path
    Default is 30 seconds
retry-limit:
    Number of failed attempts to bring up primary path
    Default is 0 (unlimited retries)
    If limit reached, human intervention required
revert-timer:
    Minimum time the primary must be up and stable before traffic is reverted to it
    Default is 60 seconds
    If set to 0 the LSP does not revert
Optimize-timer:
    It is used with CSPF, if LSP is running on primary path and you add much better route then primary path or any other path that comes up which is better then the existing path that LSP is using then LSP will not be able to use that link.  By default timer is 0 means optimization is disable, so in this case primary path will switch to other path only when it has failure occurred on existing primary path. To initiate manual optimization use: clear mpls lsp optimize

Adaptive:
Adaptive keyword will allow to use SE reservation to avoid the duplication of resource reservation.

lab@r1# show label-switched-path to-r2
to 192.168.1.2;
bandwidth 75m;
no-cspf;
primary test1 {
    adaptive;
    }
secondary test2 {
    standby;
    adaptive;
    }


Forwarding adjacency:
It advertises the LSP as point to point interface in IGP so IGP can use it in its SFP calculation. You can define the metrics on LSP to be used. Use only CSPF LSP with forwarding adjacency so that IGP will not put that LSP in TED.
In ISIS case, it needs LSP in both direction to use forwarding adjacency where as OSPF can work with one direction LSP and have IP reachability to local router from remote end.
LSPs are only used for data traffic, no control traffic or routing updates exchanged via LSPs.
In following case ISIS is advertising LSP to-r2 in ISIS in level 2 with metric of 5.
isis {
    interface all {
    level 1 disable;
    }
    label-switched-path to-r2 {
    level 2 metric 5;
    }
}
No primary path only secondary path:
    In ISP network if you have primary path and secondary path, if failure happens on primary path LSP will switch over to the backup path. Again if primary path comes backup then primary path will resignaled and traffic will revert back from secondary path to the primary path. To avoid this situation we can use multiple secondiary path and do not define any primary path at all. If one secondary path fails (1st path), traffic will switched to the next available secondary path(2nd path)
and if previously failed secondary path comes backup up router will not switch data back to the first secondary path.