So, I've got a new Internet uplink at home. And so it happens, inside interface of the newly arrived CPE has the same IPv4 subnet as the subnet to which majority of my home devices are attached. So what, you ask? Let me clarify the topology "before":
[Internet]---[Numericable]---[CableCPE]-----[cisco890]-----[home network]
^ ^
| |
| 192.168.1.0/24
192.168.0.0/24
After the addition of the second CPE this becomes:
+-------[Proximus]--[DSLCPE]--------------\ <-- 192.168.1.0/24
/ |
[Internet]---[Numericable]---[CableCPE]-----[cisco890]-----[home network]
^ ^
| |
| 192.168.1.0/24
192.168.0.0/24
See the problem ? In every other circumstance I would have of course changed the subnet in the DSL CPE configuration, and forgot about it, but I wanted to experiment with approaches of making it work regardless.
After all, I do not really use the subnets between my router and the CPEs for much at all - they are effectively point-to-point links.
So, how to deal with that ? First, I tried a light-weight approach which is as effective as it is ugly and brittle: I just need to find a way to route my inner network to the Proximus CPE. The CPE will think it has a bunch of directly connected hosts (that happen to have the same MAC, but who cares), and will NAT all the traffic for me.
In order to do this, I need to take care of two paths: upstream and downstream. Upstream is a very easy one: assign a bogus IP address to the CPE-facing interface, create a static ARP entry for another address within this bogus subnet with the MAC address of the CPE, done.
(As a reminder, if you were sleeping during your networking studies: when a router forwards the packets, the only reason it needs the actual IP address of the next hop is to resolve that address into the MAC address which it can then stick as the destination address of the packet it sends).
The downstream is trickier. Even if we were to pretend our devices are directly connected to the DSLCPE, because we assigned the bogus address on the interface facing it, the ARP requests sourced with 192.168.1.1 (the CPE's address) will be filtered and not answered. So we need something different.
Enter VRF-lite. Let's create a VRF for that interface:
ip vrf BELGACOM rd 51:51 interface Vlan51 ip vrf forwarding BELGACOM ip address 203.0.113.2 255.255.255.0 secondary ip address 192.168.1.2 255.255.255.0 ip access-group BELGACOM-ONLY out ip nat outside ip virtual-reassembly in end
Now we can happily assign the address 192.168.1.2 which is in the same subnet as my home network's interface. So, now the Vlan51 will not ignore the ARP requests anymore. Except now it does not know anything about the home network: it only has a single interface in that VRF. Moreover, our previous routing upstream is not working now, again because the next hop has disappeared into VRF. So we will need to make a route with an interface, to leak the default route into the global space and inject the packets into the VRF:
! remove the old route no ip route 0.0.0.0 0.0.0.0 203.0.113.1 ! add a new route ip route 0.0.0.0 0.0.0.0 vlan 51 203.0.113.1
But what do we do with the return traffic ? How do we make the vrf BELGACOM aware of the hosts on the home network ? We can not use the usual route leaking, because the IOS complains about wanting the next hop. And we can't set an "internal next hop".
However, it is an obvious observation that the next hop for each of the directly attached hosts is the host itself. Therefore we can add a series of these:
ip route vrf BELGACOM 192.168.1.3 255.255.255.255 192.168.1.3 global ip route vrf BELGACOM 192.168.1.4 255.255.255.255 192.168.1.4 global ip route vrf BELGACOM 192.168.1.5 255.255.255.255 192.168.1.5 global ip route vrf BELGACOM 192.168.1.6 255.255.255.255 192.168.1.6 global ip route vrf BELGACOM 192.168.1.7 255.255.255.255 192.168.1.7 global ip route vrf BELGACOM 192.168.1.8 255.255.255.255 192.168.1.8 global .... ip route vrf BELGACOM 192.168.1.250 255.255.255.255 192.168.1.250 global ip route vrf BELGACOM 192.168.1.251 255.255.255.255 192.168.1.251 global ip route vrf BELGACOM 192.168.1.252 255.255.255.255 192.168.1.252 global ip route vrf BELGACOM 192.168.1.253 255.255.255.255 192.168.1.253 global ip route vrf BELGACOM 192.168.1.254 255.255.255.255 192.168.1.254 global
This makes it work! But, oh boy, ugly and fragile it is! It was looking at this configuration that triggered me to tweet a yell of mixed joy and desperation.
That said, this did the trick of allowing me to cutover the IPv4 traffic from one ISP to the other ISP with absolutely zero downtime. But it bugged me to try to get rid of this batch of host routes and static ARPs. So I decided to pursue the VRF and route leaking idea to its logical conclusion:run a VRF per "domain" and leak the routes/translate between the VRFs as needed. As a parallel task, I've decided to also experiment with gentle cutover of the home network client segment to use 192.168.100.0/24, while still allowing the 192.168.1.x clients to coexist and work.
Without further ado, the annotated config is below:
! ip vrf BELGACOM rd 51:51 route-target export 51:51 route-target import 51:51 ! ip vrf HAPPYHOUSE rd 100:100 route-target export 100:100 route-target import 100:100 ! grab the routes from both ISPs route-target import 50:50 route-target import 51:51 ! ip vrf NUMERICABLE rd 50:50 route-target export 50:50 route-target import 50:50 ! ! ! Serve the clients some DHCP ! ip dhcp pool HAPPYHOUSE vrf HAPPYHOUSE network 192.168.100.0 255.255.255.0 default-router 192.168.100.1 domain-name stdio.be dns-server 192.168.100.53 ! interface Vlan50 description Outside - internet-facing ip vrf forwarding NUMERICABLE ip address 192.168.0.13 255.255.255.240 ip access-group NUMERICABLE-ONLY out no ip unreachables ip nat outside no ip virtual-reassembly in ! interface Vlan51 ip vrf forwarding BELGACOM ip address 203.0.113.2 255.255.255.0 secondary ip address 192.168.1.2 255.255.255.0 ip access-group BELGACOM-ONLY out no ip unreachables ip nat outside ip virtual-reassembly in ! interface Vlan100 ip vrf forwarding HAPPYHOUSE ip address 10.0.0.100 255.255.255.0 secondary ip address 192.168.1.1 255.255.255.0 secondary ip address 192.168.100.1 255.255.255.0 ip nat inside ip virtual-reassembly in nat64 enable ipv6 address FE80::1 link-local ipv6 address 2001:470:XXXXX::1/64 ipv6 mtu 1400 ipv6 nd reachable-time 3600000 ipv6 nd prefix default infinite 86400 no-onlink ipv6 nd router-preference High ipv6 nd ra solicited unicast ipv6 nd ra interval 300 ipv6 nd ra dns server 2001:470::XXXX 86400 no ipv6 redirects ipv6 tcp adjust-mss 1200 ipv6 dhcp server HappyHouse ntp broadcast ntp multicast ! ! The magic happens here. ! router bgp 1 ! address-family ipv4 vrf BELGACOM redistribute connected redistribute static default-information originate default-metric 200 exit-address-family ! address-family ipv4 vrf HAPPYHOUSE redistribute connected exit-address-family ! address-family ipv4 vrf NUMERICABLE redistribute connected redistribute static default-information originate default-metric 300 exit-address-family ! ip route vrf BELGACOM 0.0.0.0 0.0.0.0 192.168.1.1 ip route vrf NUMERICABLE 0.0.0.0 0.0.0.0 192.168.0.1 ! NAT to one or the other interface's address ! ip nat inside source route-map NAT-VLAN50 interface Vlan50 vrf HAPPYHOUSE overload ip nat inside source route-map NAT-VLAN51 interface Vlan51 vrf HAPPYHOUSE overload route-map NAT-VLAN50 permit 10 match ip address 101 match interface Vlan50 ! route-map NAT-VLAN51 permit 10 match ip address 151 match interface Vlan51 ! access-list 101 permit ip 192.168.1.0 0.0.0.255 any access-list 101 permit ip 192.168.0.0 0.0.255.255 any access-list 101 permit ip 172.16.0.0 0.15.255.255 any access-list 101 permit ip 10.0.0.0 0.255.255.255 any access-list 101 permit ip 198.51.100.0 0.0.0.255 any access-list 101 permit ip 192.0.2.0 0.0.0.255 any access-list 151 permit tcp 192.168.100.0 0.0.0.255 any access-list 151 permit udp 192.168.100.0 0.0.0.255 any access-list 151 permit icmp 192.168.100.0 0.0.0.255 any access-list 151 permit icmp 192.168.1.0 0.0.0.255 any access-list 151 permit tcp 192.168.1.0 0.0.0.255 any access-list 151 permit udp 192.168.1.0 0.0.0.255 any
You'll probably notice the NAT configuration is not consistent at this point, as well as other drawbacks - like, it might be nice, while having two ISP links, to actually use them as a backup of some sort. I might get to that later, and then will post the updated configurations. There's also another problem with this configuration - the NAT translations "stick" and the wrong source outside address is used even when the different ISP default gateway is selected, so the things like a VPN router take forever to recover from a switchover.
Also, a reminder is that I've done this mostly as an experiment/educational exercise - a much simpler solution would have been to simply change the configuration of the DSLCPE to have a different subnet inside, avoiding the problem in a simple way! Nonetheless, here are some more outputs:
ay-home#sh ip route vrf BELGACOM
Routing Table: BELGACOM
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
a - application route
+ - replicated route, % - next hop override
Gateway of last resort is 192.168.1.1 to network 0.0.0.0
S* 0.0.0.0/0 [1/0] via 192.168.1.1
192.168.1.0/24 is variably subnetted, 2 subnets, 2 masks
C 192.168.1.0/24 is directly connected, Vlan51
L 192.168.1.2/32 is directly connected, Vlan51
203.0.113.0/24 is variably subnetted, 2 subnets, 2 masks
C 203.0.113.0/24 is directly connected, Vlan51
L 203.0.113.2/32 is directly connected, Vlan51
ay-home#sh ip route vrf HAPPYHOUSE
Routing Table: HAPPYHOUSE
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
a - application route
+ - replicated route, % - next hop override
Gateway of last resort is 192.168.1.1 to network 0.0.0.0
B* 0.0.0.0/0 [20/200] via 192.168.1.1 (BELGACOM), 00:47:45
10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C 10.0.0.0/24 is directly connected, Vlan100
L 10.0.0.100/32 is directly connected, Vlan100
192.168.0.0/24 is variably subnetted, 2 subnets, 2 masks
B 192.168.0.0/28 is directly connected, 14:48:26, Vlan50
L 192.168.0.13/32 is directly connected, Vlan50
192.168.1.0/24 is variably subnetted, 3 subnets, 2 masks
C 192.168.1.0/24 is directly connected, Vlan100
L 192.168.1.1/32 is directly connected, Vlan100
S 192.168.1.64/32 [1/0] via 192.168.100.64
192.168.100.0/24 is variably subnetted, 2 subnets, 2 masks
C 192.168.100.0/24 is directly connected, Vlan100
L 192.168.100.1/32 is directly connected, Vlan100
203.0.113.0/24 is variably subnetted, 2 subnets, 2 masks
B 203.0.113.0/24 is directly connected, 14:45:01, Vlan51
L 203.0.113.2/32 is directly connected, Vlan51
ay-home#
Another question which probably is in your mind: what about IPv6 ? For now I've just made two tunnels, each with the appropriate "tunnel vrf .." configuration, and otherwise identical; which is far from optimal, but then same as with IPv4, this configuration indeed needs to evolve more, if I were to really to call it "functional". For now it is just what the subject reads: an experiment in overcomplication! :-)
| Name | Last Modified | Size | Type |
|---|---|---|---|
| Parent Directory/ | - | Directory |