Update:

First time ever I met Yakov Rekhter!!!

MTU made me cry :-)
JUNOS Technology‎ > ‎

JUNOS VPLS

JUNOS VPLS & Multihoming:


In our setup Site 1 has CEA1 customer end router connected to multiple PE (PE1 & PE2) sites for the redundancy. Each VPLS PE must be configured with the unique site ID but in case of VPLS multihoming we will have to configure multihomed PE1 & PE2 with the same site id "1" and other end PE (PE3) will be configured with the site ID of "2". Now the question is, why we need unique site ID on different PEs? and the answer for this is, VPLS VPN label (Inner Label) will be calculated on the based on the local PE site ID, if we have the same site ID for the multiple site then we will be having trouble with the inner label or VPLS VPN label calculation. But in case of multihoming we need the redundancy then we will have to use same site ID for PE1 & PE2.  We can use OSPF or ISIS as an IGP. As far as BGP is concerned we will have to enable the BGP l2signaling address family (between PEs) to support the VPLS.

To calculate the VPN or inner label we will be using the following formula:
local-site-id + remote-label-base - remote-label-offest

During the verification I will show you how to obtain various values used in given formula.

Most important part of the VPLS is mac learning, VPLS is based on the regular mac learning process, for example if CEA1 will try to send data to CEA2, broadcast frame will be sent to the PE1, PE1 will learn the MAC address of CEA1 on its CE facing interface, now lets assume that PE1 is the primary site for the VPLS customer CE-A then PE1 will flood that frame to all CE facing interface and LSI/VT interface to the remote end PE. Now PE3 will learn the CEA1's mac address on LSI/VT interface and same MAC learning concept will apply to date from CEA2 to CEA1.

How to signal which site is primary or secondary site at multihoming side or what is an election procedure for the primary & secondary site?
        1. You can configure PE as primary or Secondary (most controlled method)
        2. Advertise primary PE with higher local preference with compare to secondary PE.




ARP Note (ARP refresh):
ARP is most important part of the VPLS so can't neglect it. ARP also has the refresh interval which trigger right before the aging timer (again it might be the vendor specific to). ARP refresh uses the ARP REQUEST packet, this packet should be unicast to the remote host MAC address as per RFC.

From  RFC826:
    Another alternative is to have a daemon perform the timeouts.
    After a suitable time, the daemon considers removing an entry.
    It first sends (with a small number of retransmissions if needed)
    an address resolution packet with opcode REQUEST directly to the
    Ethernet address in the table. If a REPLY is not seen in a short
    amount of time, the entry is deleted. The request is sent
    directly so as not to bother every station on the Ethernet. Just
    forgetting entries will likely cause useful information to be
    forgotten, which must be regained.

Packet Capture:
Source MAC: 00:1d:b5:42:4c:89
Destination MAC: 2c:6b:f5:4e:be:73

At the time of first PING on Remote:
16:25:01.481518  In PFE proto 2 (ipv4): 192.168.33.1 > 192.168.33.2: ICMP echo request, id 1983, seq 0, length 64
16:25:01.481604 Out 2c:6b:f5:4e:be:73 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 46: vlan 575, p 0, ethertype ARP, arp who-has 192.168.33.1 tell 192.168.33.2
16:25:01.481875  In 2c:6b:f5:4e:be:73 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 575, p 0, ethertype ARP, arp who-has 192.168.33.1 tell 192.168.33.2
16:25:01.482262  In 0:1d:b5:42:4c:89 > 2c:6b:f5:4e:be:73, ethertype 802.1Q (0x8100), length 60: vlan 575, p 0, ethertype ARP, arp reply 192.168.33.1 is-at 0:1d:b5:42:4c:89
16:25:01.482270 Out 2c:6b:f5:4e:be:73 > 0:1d:b5:42:4c:89, ethertype 802.1Q (0x8100), length 102: vlan 575, p 0, ethertype IPv4, 192.168.33.2 > 192.168.33.1: ICMP echo reply, id 1983, seq 0, length 64
       

I stop the ping to see the ARP refresh and right before the remote end age out the ARP entry it sends ARP request packet (Broadcast), I noticed it when local router has ~120 seconds left before the aging timer expires:
16:43:10.925303  In 0:1d:b5:42:4c:89 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 575, p 0, ethertype ARP, arp who-has 192.168.33.2 tell 192.168.33.1
16:43:10.925307  In 0:1d:b5:42:4c:89 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 575, p 0, ethertype ARP, arp who-has 192.168.33.2 tell 192.168.33.1
16:43:10.925322 Out 2c:6b:f5:4e:be:73 > 0:1d:b5:42:4c:89, ethertype 802.1Q (0x8100), length 46: vlan 575, p 0, ethertype ARP, arp reply 192.168.33.2 is-at 2c:6b:f5:4e:be:73
16:43:10.925329 Out 2c:6b:f5:4e:be:73 > 0:1d:b5:42:4c:89, ethertype 802.1Q (0x8100), length 46: vlan 575, p 0, ethertype ARP, arp reply 192.168.33.2 is-at 2c:6b:f5:4e:be:73


LDP based VPLS in Junos:

Example of LDP based VPLS configuration in JUNOS. Here I am using the RSVP TE tunnel as a transport and LDP will be used to exchange the VPN labe. LDP also use label mapping and label withdrawal messages. MAC TLV is used to flush out all the MAC entries on particular VPLS domain using  LDP Address Withdraw Message. (RFC4762). How about MAC learning, its same as switch learn the mac addresses.

r1 {
    interfaces {
        ge-5/0/6 {
            unit 12 {
                vlan-id 12;
                family inet {
                    address 192.168.12.1/24;
                }
            }
        }
    }
}
r2 {
    interfaces {
        ge-5/0/5 {
            unit 23 {
                vlan-id 23;
                family inet {
                    address 192.168.23.2/24;
                }
                family iso;
                family mpls;
            }
            unit 24 {
                vlan-id 24;
                family inet {
                    address 192.168.24.2/24;
                }
                family iso;
                family mpls;
            }
        }
        ge-5/1/6 {
            unit 12 {
                encapsulation vlan-vpls;
                vlan-id 12;
            }
        }
        lo0 {
            unit 102 {
                family inet {
                    address 172.168.2.2/32;
                }
                family iso {
                    address 49.0001.0000.0000.0002.00;
                }
            }
        }
    }
    protocols {
        rsvp {
            interface ge-5/0/5.23;
        }
        mpls {
            label-switched-path r2-to-r4 {
                to 172.168.4.4;
                no-cspf;
            }
            interface ge-5/0/5.23;
            interface lo0.102;
        }
        isis {
            interface ge-5/0/5.23;
            interface lo0.102;
        }
        ldp {
            interface lo0.102;
        }
    }
    routing-instances {
        ldp-vpls-test {
            instance-type vpls;
            interface ge-5/1/6.12;
            protocols {
                vpls {
                    vpls-id 13;
                    neighbor 172.168.4.4;
                }
            }
        }
    }
}
r3 {
    interfaces {
        ge-5/1/5 {
            unit 13 {
                vlan-id 13;
                family inet {
                    address 192.168.13.3/24;
                }
                family mpls;
            }
            unit 23 {
                vlan-id 23;
                family inet {
                    address 192.168.23.3/24;
                }
                family iso;
                family mpls;
            }
            unit 34 {
                vlan-id 34;
                family inet {
                    address 192.168.34.3/24;
                }
                family iso;
                family mpls;
            }
        }
        lo0 {
            unit 103 {
                family inet {
                    address 172.168.3.3/32;
                }
                family iso {
                    address 49.0001.0000.0000.0003.00;
                }
            }
        }
    }
    protocols {
        rsvp {
            interface ge-5/1/5.23;
            interface ge-5/1/5.34;
        }
        mpls {
            interface ge-5/1/5.23;
            interface ge-5/1/5.34;
            interface lo0.103;
        }
        isis {
            interface ge-5/1/5.23;
            interface ge-5/1/5.34;
            interface lo0.103;
        }
        ldp {
            interface lo0.103;
        }
    }
}
r4 {
    interfaces {
        ge-5/0/5 {
            unit 34 {
                vlan-id 34;
                family inet {
                    address 192.168.34.4/24;
                }
                family iso;
                family mpls;
            }
        }
        ge-5/0/7 {
            unit 12 {
                encapsulation vlan-vpls;
                vlan-id 12;
            }
        }
        lo0 {
            unit 104 {
                family inet {
                    address 172.168.4.4/32;
                }
                family iso {
                    address 49.0001.0000.0000.0004.00;
                }
            }
        }
    }
    protocols {
        rsvp {
            interface ge-5/0/5.34;
        }
        mpls {
            label-switched-path r4-to-r2 {
                to 172.168.2.2;
                no-cspf;
            }
            interface ge-5/0/5.34;
            interface lo0.104;
        }
        isis {
            interface ge-5/0/5.34;
            interface lo0.104;
        }
        ldp {
            interface lo0.104;
        }
    }
    routing-instances {
        ldp-vpls-test {
            instance-type vpls;
            interface ge-5/0/7.12;
            protocols {
                vpls {
                    vpls-id 13;
                    neighbor 172.168.2.2;
                }
            }
        }
    }
}
r5 {
    interfaces {
        ge-5/1/7 {
            unit 12 {
                vlan-id 12;
                family inet {
                    address 192.168.12.2/24;
                }
            }
        }
    }
}
 

[edit]
lab@LAB# run show vpls mac-table logical-system r2   

MAC flags (S -static MAC, D -dynamic MAC,
           SE -Statistics enabled, NM -Non configured MAC)

Logical system   : r2
Routing instance : ldp-vpls-test
 Bridging domain : __ldp-vpls-test__, VLAN : NA
   MAC                 MAC      Logical
   address             flags    interface
   02:00:08:00:01:af   D        ge-5/1/6.12         
   02:00:08:00:01:c5   D        vt-0/0/10.84934656  

[edit]
lab@LAB# run show vpls mac-table logical-system r4   

MAC flags (S -static MAC, D -dynamic MAC,
           SE -Statistics enabled, NM -Non configured MAC)

Logical system   : r4
Routing instance : ldp-vpls-test
 Bridging domain : __ldp-vpls-test__, VLAN : NA
   MAC                 MAC      Logical
   address             flags    interface
   02:00:08:00:01:af   D        vt-0/0/10.101711872 
   02:00:08:00:01:c5   D        ge-5/0/7.12         


[edit]
lab@LAB# run show vpls connections extensive logical-system r2
Layer-2 VPN connections:

Legend for connection status (St)  
EI -- encapsulation invalid      NC -- interface encapsulation not CCC/TCC/VPLS
EM -- encapsulation mismatch     WE -- interface and instance encaps not same
VC-Dn -- Virtual circuit down    NP -- interface hardware not present
CM -- control-word mismatch      -> -- only outbound connection is up
CN -- circuit not provisioned    <- -- only inbound connection is up
OR -- out of range               Up -- operational
OL -- no outgoing label          Dn -- down                     
LD -- local site signaled down   CF -- call admission control failure     
RD -- remote site signaled down  SC -- local and remote site ID collision
LN -- local site not designated  LM -- local site ID not minimum designated
RN -- remote site not designated RM -- remote site ID not minimum designated
XX -- unknown connection status  IL -- no incoming label
MM -- MTU mismatch               MI -- Mesh-Group ID not availble
BK -- Backup connection          ST -- Standby connection

Legend for interface status
Up -- operational          
Dn -- down

Instance: ldp-vpls-test
  VPLS-id: 13
    Number of local interfaces: 1
    Number of local interfaces up: 1
    ge-5/1/6.12       
    vt-0/0/10.84934656            Intf - vpls ldp-vpls-test neighbor 172.168.4.4 vpls-id 13
    Neighbor                  Type  St     Time last up          # Up trans
    172.168.4.4(vpls-id 13)   rmt   Up     Dec 28 14:16:53 2010           1
      Remote PE: 172.168.4.4, Negotiated control-word: No
      Incoming label: 800000, Outgoing label: 800000
      Local interface: vt-0/0/10.84934656, Status: Up, Encapsulation: ETHERNET
        Description: Intf - vpls ldp-vpls-test neighbor 172.168.4.4 vpls-id 13
    Connection History:
        Dec 28 14:16:53 2010  status update timer 
        Dec 28 14:16:53 2010  PE route changed    
        Dec 28 14:16:53 2010  In lbl Update                     800000
        Dec 28 14:16:53 2010  Out lbl Update                    800000
        Dec 28 14:16:53 2010  In lbl Update                     800000
        Dec 28 14:16:53 2010  loc intf up           vt-0/0/10.84934656

[edit]
lab@LAB# run show vpls connections extensive logical-system r3   

[edit]
lab@LAB# run show vpls connections extensive logical-system r5   

[edit]
lab@LAB# run show vpls connections extensive logical-system r4   
Layer-2 VPN connections:

Legend for connection status (St)  
EI -- encapsulation invalid      NC -- interface encapsulation not CCC/TCC/VPLS
EM -- encapsulation mismatch     WE -- interface and instance encaps not same
VC-Dn -- Virtual circuit down    NP -- interface hardware not present
CM -- control-word mismatch      -> -- only outbound connection is up
CN -- circuit not provisioned    <- -- only inbound connection is up
OR -- out of range               Up -- operational
OL -- no outgoing label          Dn -- down                     
LD -- local site signaled down   CF -- call admission control failure     
RD -- remote site signaled down  SC -- local and remote site ID collision
LN -- local site not designated  LM -- local site ID not minimum designated
RN -- remote site not designated RM -- remote site ID not minimum designated
XX -- unknown connection status  IL -- no incoming label
MM -- MTU mismatch               MI -- Mesh-Group ID not availble
BK -- Backup connection          ST -- Standby connection

Legend for interface status
Up -- operational          
Dn -- down

Instance: ldp-vpls-test
  VPLS-id: 13
    Number of local interfaces: 1
    Number of local interfaces up: 1
    ge-5/0/7.12       
    vt-0/0/10.101711872           Intf - vpls ldp-vpls-test neighbor 172.168.2.2 vpls-id 13
    Neighbor                  Type  St     Time last up          # Up trans
    172.168.2.2(vpls-id 13)   rmt   Up     Dec 28 14:16:53 2010           1
      Remote PE: 172.168.2.2, Negotiated control-word: No
      Incoming label: 800000, Outgoing label: 800000
      Local interface: vt-0/0/10.101711872, Status: Up, Encapsulation: ETHERNET
        Description: Intf - vpls ldp-vpls-test neighbor 172.168.2.2 vpls-id 13
    Connection History:
        Dec 28 14:16:53 2010  status update timer 
        Dec 28 14:16:53 2010  PE route changed    
        Dec 28 14:16:53 2010  In lbl Update                     800000
        Dec 28 14:16:53 2010  Out lbl Update                    800000
        Dec 28 14:16:53 2010  In lbl Update                     800000
        Dec 28 14:16:53 2010  loc intf up           vt-0/0/10.101711872

[edit]
lab@LAB# run show vpls connections lo
                                             ^
'lo' is ambiguous.
Possible completions:
  local-site           Name or ID of VPLS local site
  logical-system       Name of logical system, or 'all'
[edit]
lab@LAB# run show vpls connections logical-system r2
Layer-2 VPN connections:

Legend for connection status (St)  
EI -- encapsulation invalid      NC -- interface encapsulation not CCC/TCC/VPLS
EM -- encapsulation mismatch     WE -- interface and instance encaps not same
VC-Dn -- Virtual circuit down    NP -- interface hardware not present
CM -- control-word mismatch      -> -- only outbound connection is up
CN -- circuit not provisioned    <- -- only inbound connection is up
OR -- out of range               Up -- operational
OL -- no outgoing label          Dn -- down                     
LD -- local site signaled down   CF -- call admission control failure     
RD -- remote site signaled down  SC -- local and remote site ID collision
LN -- local site not designated  LM -- local site ID not minimum designated
RN -- remote site not designated RM -- remote site ID not minimum designated
XX -- unknown connection status  IL -- no incoming label
MM -- MTU mismatch               MI -- Mesh-Group ID not availble
BK -- Backup connection          ST -- Standby connection

Legend for interface status
Up -- operational          
Dn -- down

Instance: ldp-vpls-test
  VPLS-id: 13
    Neighbor                  Type  St     Time last up          # Up trans
    172.168.4.4(vpls-id 13)   rmt   Up     Dec 28 14:16:53 2010           1
      Remote PE: 172.168.4.4, Negotiated control-word: No
      Incoming label: 800000, Outgoing label: 800000
      Local interface: vt-0/0/10.84934656, Status: Up, Encapsulation: ETHERNET
        Description: Intf - vpls ldp-vpls-test neighbor 172.168.4.4 vpls-id 13

[edit]
lab@LAB# run show vpls connections logical-system r3   

[edit]
lab@LAB# run show vpls connections logical-system r4   
Layer-2 VPN connections:

Legend for connection status (St)  
EI -- encapsulation invalid      NC -- interface encapsulation not CCC/TCC/VPLS
EM -- encapsulation mismatch     WE -- interface and instance encaps not same
VC-Dn -- Virtual circuit down    NP -- interface hardware not present
CM -- control-word mismatch      -> -- only outbound connection is up
CN -- circuit not provisioned    <- -- only inbound connection is up
OR -- out of range               Up -- operational
OL -- no outgoing label          Dn -- down                     
LD -- local site signaled down   CF -- call admission control failure     
RD -- remote site signaled down  SC -- local and remote site ID collision
LN -- local site not designated  LM -- local site ID not minimum designated
RN -- remote site not designated RM -- remote site ID not minimum designated
XX -- unknown connection status  IL -- no incoming label
MM -- MTU mismatch               MI -- Mesh-Group ID not availble
BK -- Backup connection          ST -- Standby connection

Legend for interface status
Up -- operational          
Dn -- down

Instance: ldp-vpls-test
  VPLS-id: 13
    Neighbor                  Type  St     Time last up          # Up trans
    172.168.2.2(vpls-id 13)   rmt   Up     Dec 28 14:16:53 2010           1
      Remote PE: 172.168.2.2, Negotiated control-word: No
      Incoming label: 800000, Outgoing label: 800000
      Local interface: vt-0/0/10.101711872, Status: Up, Encapsulation: ETHERNET
        Description: Intf - vpls ldp-vpls-test neighbor 172.168.2.2 vpls-id 13

[edit]


Logical system: r4
Routing table: ldp-vpls-test.vpls
VPLS:
Destination        Type RtRef Next hop           Type Index NhRef Netif
default            perm     0                    dscd   687     1
vt-0/0/10.101711872
                   intf     0                    indr 1048578     5
                              192.168.34.3      Push 800000, Push 299792(top)  1379     2 ge-5/0/5.34
02:00:08:00:01:af/48
                   user     0                    indr 1048578     5
                              192.168.34.3      Push 800000, Push 299792(top)  1379     2 ge-5/0/5.34
02:00:08:00:01:c5/48
                   user     0                    ucst  1402     4 ge-5/0/7.12
0x30004/51         user     0                    comp  1427     2
0x30002/51         user     0                    comp  1396     2
0x30152/51         user     0                    comp  1336     2
ge-5/0/7.12        intf     0                    ucst  1402     4 ge-5/0/7.12

Logical system: r2
Routing table: ldp-vpls-test.vpls
VPLS:
Destination        Type RtRef Next hop           Type Index NhRef Netif
default            perm     0                    dscd   683     1
vt-0/0/10.84934656 intf     0                    indr 1048579     5
                              192.168.23.3      Push 800000, Push 299776(top)  1381     2 ge-5/0/5.23
02:00:08:00:01:af/48
                   user     0                    ucst  1388     4 ge-5/1/6.12
02:00:08:00:01:c5/48
                   user     0                    indr 1048579     5
                              192.168.23.3      Push 800000, Push 299776(top)  1381     2 ge-5/0/5.23
0x30005/51         user     0                    comp  1438     2
ge-5/1/6.12        intf     0                    ucst  1388     4 ge-5/1/6.12
0x30003/51         user     0                    comp  1418     2
0x30151/51         user     0                    comp  1407     2


[edit]
lab@LAB# run ping 192.168.12.2 logical-system r1       
PING 192.168.12.2 (192.168.12.2): 56 data bytes
64 bytes from 192.168.12.2: icmp_seq=0 ttl=64 time=0.696 ms
64 bytes from 192.168.12.2: icmp_seq=1 ttl=64 time=0.685 ms
64 bytes from 192.168.12.2: icmp_seq=2 ttl=64 time=0.566 ms
^C
--- 192.168.12.2 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.566/0.649/0.696/0.059 ms

[edit]
lab@LAB# run ping 192.168.12.1 logical-system r5   
PING 192.168.12.1 (192.168.12.1): 56 data bytes
64 bytes from 192.168.12.1: icmp_seq=0 ttl=64 time=0.707 ms
64 bytes from 192.168.12.1: icmp_seq=1 ttl=64 time=0.579 ms
64 bytes from 192.168.12.1: icmp_seq=2 ttl=64 time=0.590 ms
^C
--- 192.168.12.1 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.579/0.625/0.707/0.058 ms

[edit]
lab@LAB#




RFC to read: RFC4761
Juniper VPLS Label block document: click here

............................Adding more information as time permits........................