Cisco SDWAN Design Series-Part-4- The Magic of Cisco SDWAN Policies

Part-2 of this blog series discussed the architecture components of Cisco SDWAN, then Part-3 provided some insight about the routing/control plane logic and design considerations. This blog focuses on the SDWAN policies

As highlighted in previous blogs of this blog series, what’s interesting about Cisco SDWAN architecture, is the modularity aspect of its architecture components. Each architecture function or plane is deployed as a separate component, and all collectively construct the Cisco SDWAN solution architecture that operate as a “system”.

Similarly, the way policies are created and applied is very modular and structured, which makes it very flexible and scalable to handle global scale complex requirements. Later in this blog you will find out how and why.

As described in Part-2 of this blog series, vManage is the single pane of glass for the Cisco SDWAN management including creation of policies, in turn the vManage will either push the policy to the control plane controller to be pushed over OMP to the relevant SDWAN edge nodes, of the defined policy(s) can be pushed directly from the vManage to the relevant edge node(s).

So the question here, when and which type of policy the vManage needs to push it to the vSmart first vs. directly to the SDWAN edge node(s)?

The influencing factor here is the nature of the policy, whether it’s a centralized or localized type of policy. The following figure shows the logical structure of the policy models.

As described in Part-2 of this blog series, vManage is the single pane of glass for the Cisco SDWAN management including creation of policies, in turn the vManage will either push the policy to the control plane controller to be pushed over OMP to the relevant SDWAN edge nodes, of the defined policy(s) can be pushed directly from the vManage to the relevant edge node(s).

So the question here, when and which type of policy the vManage need to push it to the vSmart first vs. directly to the SDWAN edge node(s)?

The influencing factor here is nature of the policy, whether it’s a centralized or localized type of policy. The following charts shows logical structure of the policy models.

Cisco SDWAN Policies Building Blogs

The actual policy implementation of what is a a vSmart policy is performed by deploying the entire policy on the vSmart controller (typically from via the management plane > vManage), based on three foundational building blocks illustrated below

With this modular policy structure network engineers or Admins, can have more flexibility when dealing with creating and updating the policies.

In the above figure we have what is called ‘policy Lists’ which is the way we group related items, so that we can reference them in the policy later as one object, such as, prefixes, TLOCs, VPNs, and overlay network. Creating pre-defined list(s) as part of the vSmart policies, is the key of Cisco SDWAN policy flexibility. Because these lists can be referenced in two places: when you create a policy definition and when you apply a policy.

With this separation policy construct will be more modular > more flexible.

For example creating a list that  has the related items like you defined a policy with certain sites IDs and then you decide to add more sites as part of this policy, it will be as simple as adding the additional sites’ identifiers to the existing sites’ list. from the definition of policy means that when you can add or remove items from a lists. In other words, any changes to the impacted objects by the policy can be done without touching the actual policy, also,  any changes to the policy rule(s) can be done without the need to re-configure or manually modify the prefixes, VPNs, or other objects that the rules meant to take effect on.

Each policy type is meant to handle certain capabilities and functions.

Control policy: as discussed in the previous blogs of this blog series,  with SDWAN and specifically Cisco SDWAN, control plane intelligence ( in specific routing policies) is centralized and deployed and governed by the vSmart the central control plane element.

Technically, SDWAN edge nodes, periodically exchange OMP updates, which carry reachability information of the networks connected to the SDWAN fabric. As discussed before, in previous blog, these updates includes LAN-side routes/vRoute attributes and TLOC attributes.

Based on these attributes, vSmart controller can determine the topology and status of the overlay network, and installs routing information about the overlay network into its routing table. In turn, the vSmart will propagate the reachability information to the c/vEdge SDWAN nodes via OMP. With this learning and propagation of routing information, control policies can take effects to control what need to be advertised and to which SDWAN edge nodes.

Any changes that results from control policy are applied directionally, either inbound or outbound (the directionality is from the point of view of the vSmart controller).

Therefore, be aware that, by applying an inbound policy in the inbound direction will end up impacting the visibility of the vSmart controller (RIB) to determine the network topology and network reachability, and this may reduce its, flexibility to have full visibility and control over the flows of traffic across the overlay network.

In contrast, an outbound policy, typically is applied to OMP updates that the vSmart controller is propagating to the SDWAN edge nodes. Policy changes here take effect in the egress direction ( which is after the routing information installed in the vSmart controller’s route table and before the SDWAN edge node receives the update via OMP).

If you worked with routing policies and specifically BGP policies, you know that you can achieve many routing (traffic engineering). With the use of Cisco SDWAN and centralized control plane and policies, the number of the possible achievable routing scenarios are even more without adding the complexity of building and managing distributed policies(centralized), technically, these policies are configured from the vManage interface, and then pushed and enforced at the vSmart controller(s), therefore, they are not meant to be distrusted in which none-of these control policies are are forwarded to SDWAN edge nodes. router The following are some of the common use scenarios.

Scenario-1 make main data center (DC-1) preferred over second data center, for site-1. While site-2 should use data center-2 as the preferred DC for the same route prefixes. If any DC is down, traffic should reroute to the other data center.

Scenario-2 there are different virtual networks where SDWAN VPNs need to be used across the SDWAN overlay to maintain path separation end to end, at the same time each virtual network requires different connectivity model over the WAN, e.g. VPN-1 carries client-server type of application in which only ‘Hub & Spoke’ topology is required, while VPN-2 used for VoIP traffic in which full mesh connectivity is required.

Note: in the above, example, the vSmart controller will typically pushes the routes for all VPNs to all SDWAN edge nodes, however this may not be acceptable for some security restrictions or regularity etc. depends on the use case, or there are a lot of routes in a VPN, that is not necessarily has to be propagated to all SDWAN nodes, in which the solutions has to restrict the participation of specific SDWAN edge nodes in a particular VPN(s), here you can use a VPN membership policy to dictate this type of restriction.

Scenario-3 let’s build on the previous scenario with multiple VPNs, and assume that these two virtual networks need to communicate with certain system that reside in a different virtual network (VPN-3) this use case commonly referred to as ‘shared services’. This can be achieved by configuring a control policy that export VPN-1 and VPN-2 routes into the shared service virtual network VPN-2, and vice versa. With this export, VPN-1 and VPN-2 still separated and can not communicate with each other, unless an additional route expert is performed.

Scenario-4 what if the business decided to allow communication between two virtual networks, but traffic must pass through a security control system(s) e.g. NGFW, NGIPS etc. here we need to do some traffic engineering, or specifically what is known as ‘ service chaining’.

There are multiple ways to achieve this with Cisco SDWAN (you still can come up with your own suitable and creative way of doing it), the following are two common ways using a control policy.

Option-1: this is very similar to route leaking between two MPLS L3VPN VPNs via a fusion router/FW. As illustrated below, as the hub site, regional hub etc. where the security device like FW is sitting and connected using two interfaces to the SDWAN edge node from the service/LAN side, each interface belongs to a VPN (VPN-1 and VPN-2). The FW will advertise VPN-2 routes to the SDWAN edge node over the interface that is part of VPN-1 and vice versa, this can be done using static route as well.

Option-2: the other option  here is by configuring a service chained Firewall service (per-virtual network/VPN) as illustrated below with a control policy you can create and advertise a FW service with IP address, to steer traffic to a given next hope IP. In this scenario the following steps happens to achieve such service chained routing:

  1. The SDWAN edge node where the FW/IP/LB is connected advertise the availability of the service (configurable on that edge node) to the vSmart controllers ( can be done per VPN if the security service doing inter-VPN routing). Also, it is possible to have multiple SDWAN edge nodes to advertise the same or different service.
  2. The SDWAN edge node connected to the FW/IPS keeps adverting its typical OMP routes and TLOCs to the vSmart controllers, the TLOC here going to be the Next-Hop to reach this defined service IP after creating the policy.
  3. When the control policy is created based on destination IP prefix(s), the matched traffic that requires the security service in the path, the control vSmart policy will manipulate the Next-Hop of the OMP routes to point to the to the service landing point which is the FW/IPS interface, and this device should have the routing configured to send traffic to another interface of the SDWAN LAN interface ( if this is for routing among VPNs, route leaking/export control policy need to be created first). In this way, the traffic will pass through the security service device first for security inspection before being routed to its final destination.

Note: technically in Cisco SDWAN the same scenario above can be achieved using either control or data policy on the vSmart controller. Control policy can be used if the matching criteria is based on a destination prefix or any of its attributes, while if the matching criteria contains some source information like address, port, DSCP value, as well as a destination port of the the traffic flow, then a data policy should be used in this case.

Scenario-5: What is a global organization, needs to create regional based communication islands and policies, in which each region can communicate directly with the sites in that region directly and must go via the regional hub/HQ if communication required with another region.

With this we are moving the architecture from a flat any to any to more hierarchal one. With classical routing, if its achievable, this requires a lot of distributed routing policies. While with Cisco SDWAN, using cartelized control policies, you can limit the BFD sessions/IPSec to be formed between spoke sites in a region ( based site ID lists) as well as allow it among the hub sites , only. The matching attributes here is the TLOC. The logic here, to have two policies per region ( one for the spokes and one for the hub). For example, region APJ spokes policy that match the TLOC of all other remote sites outside APJ region (using site list) and reject it, as well as, match all routes of other remote sites outside APJ region and set the next-hop/TLOC as the APJ hub site TLOC. From the hub side, similar logic is required a policy that match the TLOC of all other remote sites outside APJ region (using site list) and reject it, excluding the other regional hubs, as well as, match all routes of other remote sites of NA region and set the next-hop/TLOC as NA hub, same for EMEAR remote sites routes  and set the next-hop/TLOC as EMEAR hub. This should create a logical topology like the one illustrated below.

Data Policy: unlike control policy that is mainly destination based routing,  Data Policy in Cisco SDWAN inspects fields in the headers of data packets, to find the source and destination addresses and ports, as well as protocol and DSCP values.

As the name imply, Data policy operates on the data plane of the Cisco SDWAN fabric, therefore, it influence how data (flows) is sent between the SDWAN edge nodes. As highlighted earlier, there are two types of Data policy, centralized data policy, which is defined at the vSmart and it controls the flow of data traffic based on the IP header fields in the data packets, while the localized Data policy, aims to control the flow of data traffic into and out of interfaces and interface queues on an SDWAN edge node.

Once matching occur based on the defined attribute(s) of a packet flow, Data policy is capable to change the next hop, apply a policer to the packets.

Because Data policy look into the data plane of traffic flows at router/site level, the Data policy is configured and applied on the vSmart controller, however, this policy then is carried out over OMP updates to the respective SDWAN edge nodes, as specified in the in the site-list where the policy is applied to. As a result, the actual Data policy operation ( matching and the associated actions) are all performed on the SDWAN edge node, in real-time while data traffic is passing.

The following are some of the flexibilities and control that can be gained by using Cisco SDWAN centralized data policy:

  • Define the sources that are allowed to send traffic to any destination outside the local site over the SDWAN Fabric.
  • Define the sources (network, host etc.) that are allowed to communicate with to a specific destination (network, host etc.) outside the local site, over the SDWAN Fabric. With this you can control the allowed source addresses and ports that are permitted to send traffic to any destination outside the local site or to a specific port at a specific destination.
  • Create explicit site-level traffic engineering, based on source and destination IPs ports and even based on application type (DPI/NBAR) and specify the desired path a the site level.

This deep packet inspection offers control over how data packets from specific applications or application families are forwarded across the network, allowing you to assign the traffic to be carried by specific tunnels.” This a manual TE, and not SLA based

To control application(s) traffic flows based application SLA t(loss, latency, etc.) over multiple paths, you need to use application-aware routing, which is covered briefly below.

Application Policy: or also known as Application-Aware Routing AAR, act differently from typically routing and forwarding, in which it specifies the performance characteristics in the form an SLA, based on pre-defined criteria/SLA of an application that trigger the policy to take effect. Then it tracks packet loss and latency on the data plane connections over the SDWAN fabric, in order to be able calculate and pick the optimal paths for data traffic in the overlay network, based on the pre-defined SLA metrics.

Application-aware routing policy is defined by means of a specially designed vSmart policy, called an app-route-policy. This policy specifies the performance characteristics in the form an SLA, which defines the required latency and loss conditions on the data plane connection that trigger the policy to take effect. Keep in mind that, AAR policy applies only on traffic that is flowing from the service/LAN to side the WAN, and if two or more tunnels match, traffic is distributed among them. Also, with AAR policy you can pick a preferred path/TLOC color for a the application(s) matched in this policy

Last but not least, it’s important for know how these different policies take effect in order.

The subsequent blog, will focus on the Cloud integration and design options and considerations

Marwan Al-shawi – CCDE No. 20130066, Google Cloud Certified Architect, AWS Certified Solutions Architect, Cisco Press author (author of the Top Cisco Certifications’ Design Books “CCDE Study Guide and the upcoming CCDP Arch 4th Edition”). He is Experienced Technical Architect. Marwan has been in the networking industry for more than 12 years and has been involved in architecting, designing, and implementing various large-scale networks, some of which are global service provider-grade networks. Marwan holds a Master of Science degree in internetworking from the University of Technology, Sydney. Marwan enjoys helping and assessing others, Therefore, he was selected as a Cisco Designated VIP by the Cisco Support Community (CSC) (official Cisco Systems forums) in 2012, and by the Solutions and Architectures subcommunity in 2014. In addition, Marwan was selected as a member of the Cisco Champions program in 2015 and 2016.