What is Tunnel MTU?
Tunnel MTU is the minimum PMTU of all the paths between source sd-wan device and remote sd-wan device after accounting sd-wan tunnel overhead. Tunnel MTU is used for making decision on fragmenting while sending over tunnel to remote sd-wan device and also for rewriting TCP MSS if tcp-adjust-mss feature is enabled on VOS.
How Tunnel MTU is calculated?
For every underlay paths between source sd-wan device and remote sd-wan device, periodic PMTU probes are sent to detect PMTU of each underlay paths. Once PMTU of all underlay paths are detected, smallest PMTU will be Tunnel MTU after accounting sd-wan tunnel overhead.
Why Tunnel MTU is lowest PMTU of all paths?
If traffic switches between one path to other path, it causes fragmentations if traffic was going over bigger PMTU path and switches to lower PMTU path. To avoid fragmentation, Tunnel MTU is lowest PMTU of all the paths between source sd-wan device and remote sd-wan device. Fragmentation is sub-optimal for performance and VOS tries to avoid fragmentation.
How often PMTU probes are sent to remote sd-wan device from local sd-wan device?
Local sd-wan device will only send probes to the remote sd-wan device path path if there is active TX data traffic over a particular WAN link.
Default is 10 mins (600 seconds) to every branch device and is configurable with a min value of 300 seconds.
PMTU probe is sent if there is any data traffic sent in the last T-10mins [Default].
How often PMTU probes are sent to remote controllers from local sd-wan device?
PMTU probes to the Controller starts with 10 mins, and it exponentially increases its timer upto 24 hrs per probe. If there is a PMTU change, then the timer is reset to 10 mins again. If no changes to the PMTU, then it is sent every 24 hrs.
How to find out what is my current tunnel-MTU?
CLI: show interfaces dynamic-tunnels
For example: The encrypted Tunnel MTU towards this Remote Branch "Hub-Snehal-22-HA2" is 1371. This is calculated based on the Path-MTU between all available WAN circuits.
How to find the PMTU?
admin@Branch-cli> show debug vsf tunnel path-info-summary site-name <Branch-Name> tenant <Tenant-Name>
How can I have a combination of links with Path MTU of 9000 and 1500 from Site-A to Site-B?
The overall Tunnel MTU will always be the lowest of all available WAN circuits between Site-A and Site-B.
If we still need Jumbo frame Tunnel MTU to be considered, we must perform one of the below 2 options:
- Shutdown the path which has low PMTU if requirement is to support Jumbo frames without fragmentation.
- Configure unique transport domain between Site-. SLA context is built only for matching transport domain between Local and Remote device.
Any Caveats with Tunnel MTU?
As we explained Tunnel MTU is minimum of PMTU of all the available paths between source sd-wan device and remote sd-wan device. If there is one path which has lower PMTU and other paths has larger PMTU, tunnel MTU is set to lowest PMTU path value. This enforces source and destination to send packets based on Tunnel MTU and cannot take benifit of some of paths with larger PMTU. Even path with no forwarding class configured in sd-wan SLA path policy is considered for tunnel MTU calculation based on PMTU of this path. Even though traffic steering policy is not configured to use certain paths, those paths will be still included in Tunnel MTU due to remote sd-wan devices may send over these paths and reverse traffic takes same path due to default path symmetricity.