We’ve recently done a few migrations of apps between different F5 LTM load balancers.  Thought I’d post some brief steps on on how we’ve done this in a couple different scenarios.

In our environment, the load balancers are deployed on a stick. The subnets that the VIP’s are on are routed to the LB’s, and SNAT is utilized to force return traffic back through the load balancers. LB on a Stick SNAT

The process we followed would likely be quite a bit different if the load balancers were acting as the default gateway for hosts. We also are able to dynamically advertise /32 routes to our VIP’s, using OSPF. This process also relies on that capability. The actual cutover will reset connections - but a new connection to the application should immediately succeed. For some environments like ours, this is OK if done outside of normal business hours. Obviously, if connection resets are not acceptable then you’ll have to handle this differently.

 Migrating between LTM’s on Different Routing Nets

This first process is how I’ve moved applications between LTM clusters that are on separate routing networks.

Assumptions:

  • This assumes that the VIP subnets are statically routed to the existing load balancers, and that those static routes are less specific than a /32. The new LTM will advertise /32 external OSPF routes (at least temporarily), and those /32’s will need to be more specific than the static routes so that they take precedence.

  • It’s fine if the existing LB’s are advertising the individual VIP’s via OSPF (though not required).

Steps

  1. Load same version of TMOS onto the new load balancers. Slightly different versions would be fine…but I would recommend making them the same version so that you are only changing one thing at a time. (Just changing the LB instance, not the LB instance and the OS version at the same time).
  2. Copy all configuration from existing load balancer to the new load balancer. On the new LB, use a SNAT pool that contains IP addresses on the new routing network.
  3. Configure OSPF on the new load balancers, and ensure that adjacencies are formed with the routers. (The route-advertisement setting is disabled by default on LB VIP’s. So configuring OSPF will not yet advertise the /32 routes to the VIP’s).
  4. Ensure that the static routes on the routers are pointing to the floating IP address of your existing load balancer cluster.
  5. If you’re using OSPF on the existing load balancers, disable it. The less specific static route on the routers will take over and continue to route traffic to the existing load balancer.
  6. This step actually cuts the apps over.  Enable route-advertisement on the new load balancer for the LB VIP’s that have been copied over. Depending on your comfort level / the criticality of the applications, this could be done one at a time or in bulk. Obviously, you’ll want to verify that the router are seeing the /32 routes for each of the VIP’s.  If needed - backing off during the cutover is pretty easy - just disable the route-advertisement on the new load balancers. Traffic to that VIP would then follow the more general static route.
  7. At a later point in time, the static routes on the routers can be updated to point to the floating IP on the new load balancers. This would be non-impactful. OSPF could also be disabled on the new load balancers after this is complete.
  8. If you’re using F5 GTM’s, you’ll also need to do some work there. Prior to retiring the original load balancers, configure the new pair of LB’s on the GTM. Once all of the virtual servers have been discovered by the GTM, update any pools that reference virtuals on the old pair of LB’s to point to the virtuals on the new LB’s.

Replacing a Pair of LTM’s on Same Routing Network

This is the process we used to move applications between pairs of LB’s on the same routing network.  The new pair of LB’s was replacing the original ones in our instance, so we wanted to end up using the same self-IP’s / SNAT IP’s.

Assumptions:

  • This assumes that you’re wanting to replace the load balancer pair, and want to end up using the same IP addresses that the original pair of load balancers used.
  • It also assumes that you’re able to use OSPF, that static routes for the VIP’s are configured on the upstream routers, and that those static routes are not /32’s.

Steps

  1. Bring up the new load balancers. Configure them on the network using temporary self IP’s that are different than the IP’s the current LTM’s are using.
  2. Load same version of TMOS onto the new load balancers. Slightly different versions would be fine…but I would recommend making them the same version so that you are only changing one thing at a time. (Just changing the LB instance, not the LB instance and the OS version at the same time).
  3. Copy all configuration from existing load balancer to the new load balancer. Use a SNAT pool that contains different temporary IP’s though (to prevent both pairs of LB’s from responding to ARP’s for those IP addresses)
  4. Ensure that the static routes on the routers are pointing to the floating IP address of your existing load balancer cluster.
  5. If OSPF is being used on the existing pair of LB’s, disable it. Traffic to the VIP’s will follow the more general static route now.
  6. Add new non-floating self IP’s on existing pair of LB’s.  This is to help with reconfiguring the GTM prior the self IP’s over to the new LB.
  7. If a GTM is being used - add the new non-floating self IP’s for the existing load balancers to the server object for the LTM’s on the GTM.
  8. We have iQuery debug logging enabled on our GTM’s, so we were able to verify that the GTM’s successfully could communicate to the LTM. We then removed the original self IP addresses from the server object on the GTM.
  9. Remove the original self IP’s from the existing set of LB’s. This will have no impact, because the static route on the router is pointing to the floating IP address - not the local IP addresses on each LTM.
  10. Re-assign the original non-floating self IP’s to the new set of load balancers, and configure the new load balancers on the GTM’s using those IP addresses. Verify that the GTM discovers all of the virtual servers on the new LB’s.
  11. Modify any pools on the GTM that reference the existing pair of load balancers to point to the equivalent virtual on the new pair of load balancers.
  12. This step actually cuts the applications over to the new load balancers.  Enable route-advertisement on the new load balancer for the LB VIP’s that have been copied over. The new active LTM will advertise /32 routes for each of the VIP’s. Depending on your comfort level / the criticality of the applications, this could be done one at a time or in bulk.
  13. Testing can be done now. If you need to back out, you can still easily do so for individual applications by disabling route-advertisement for that virtual IP.
  14. Once you’re satisfied that everything is working properly, remove the floating self IP from the existing pair of LB’s, and configure it on the new LB’s. OSPF could be disabled on the new LB’s at this point if you wanted.
  15. Remove the SNAT pool from the existing / original pair of LB’s. Add those IP’s into the SNAT pool on the new pair of LB’s. Finally, remove the temporary IP addresses that were initially configured in the SNAT pool on the new pair of LB’s. This may also reset open client connections. (If you don’t care if the SNAT pool on the new LB’s ends up containing the same addresses that were originally used, you can skip this.)

So there you have it.  We’ve used those general processes in our environment - obviously every environment has some unique aspects, so some tweaks might be needed.  Some of these steps could be done in a slightly different order.  But there’s definitely a number of different pieces to remember, particularly if you’re using GTM’s.