How TCP optimization works in Versa


Purpose

This article explains how TCP optimization works in Versa Operating System (VOS), the modes available, how to choose between them, and how to configure, test and verify it. A sample configuration uses Auto mode to improve SCP throughput between two SD-WAN branches over a path with added latency.

Note: The configuration shown here is for reference only. Please validate it in a lab environment before deploying to production devices.

1. Overview

TCP throughput drops sharply on paths with high latency or packet loss. The sender waits a full round trip for acknowledgments, loss recovery is slow, and standard loss-based congestion control backs off aggressively. TCP optimization in VOS reduces these effects for bulk data transfers (file copies, backups, SMB, database replication and similar) across WAN links.

VOS does this with a TCP proxy architecture. One or more VOS devices in the path terminate the end host's TCP connection locally and open a separate TCP connection toward the next hop. Each connection segment is tuned and recovers from loss independently.

Note: The benefit is mostly on the transmitting side of a transfer. TCP optimization helps most with bulk data transfer over high-latency or lossy WANs. It gives little benefit on low-latency, clean paths.

2. Key Features

Feature

What it does

Latency splitting

TCP proxies split the end-to-end latency into shorter segments. Each segment runs its own loss recovery, which speeds up slow-start convergence and recovery from loss, so the connection reaches available bandwidth sooner.

Congestion control

Instead of relying only on loss-based algorithms, VOS can use BBR (Bottleneck Bandwidth and Round-trip propagation time), which models the path rather than reacting to loss. Other algorithms, such as CUBIC, are also selectable per profile.

Loss detection and recovery

VOS supports RACK (Recent Acknowledgment), which detects loss using time rather than duplicate-ACK counts. It relies on per-packet timestamps and SACK. Because duplicate-ACK counting is not used, packet reordering has much less effect on throughput.

High-performance TCP options

If the end hosts do not support Window Scaling, large buffers, SACK or Timestamps, VOS enables them on the WAN-facing segment on the hosts' behalf.

 

3. How the Session Is Split (Auto Mode)

In Auto mode, VOS devices in the path run peer discovery. They work out which device is closest to the client and which is closest to the server, then split the end-to-end session into three TCP sessions:

 Client ──(TCP 1)── VOS-A ══════(TCP 2)══════ VOS-B ──(TCP 3)── Server

        LAN profile        WAN profile (BBR/RACK)        LAN profile

1.   TCP session 1: client to the first VOS device (LAN side)

2.   TCP session 2: between the two VOS devices (WAN side). This is where the WAN profile (for example BBR and RACK) is applied.

3.   TCP session 3: second VOS device to the server (LAN side)

Prerequisite: Auto mode must be enabled on all VOS devices in the path. If the device closest to a host cannot split the session (for example, it is out of resources), the next Auto-mode device in the path takes over.

The LAN and WAN legs can use different TCP profiles.

4. TCP Optimization Modes

Mode

Type

Description

Typical use

Auto

Dual-ended

Peer discovery finds the VOS devices closest to the client and server and splits the session between them. Recommended default.

Unknown or mixed traffic patterns; transfers that are sometimes upload-heavy and sometimes download-heavy

Splice

Single-ended

Optimizes without full dual-ended termination.

Applications where a single-ended approach is enough

Proxy

Single-ended

Similar to splice, except that the first SYN is terminated locally by VOS rather than traveling end to end. It cannot be used on a hub when the proxy sits behind the hub.

Single-ended deployments on branch or spoke devices

Forward proxy

Single-ended

Optimizes data sent from clients to servers. Configure it on the VOS device closest to the client.

Upload-heavy applications

Reverse proxy

Single-ended

Optimizes data sent from servers to clients. Configure it on the VOS device closest to the server.

Download-heavy applications

 

Why Auto mode is usually the best starting point

Trade-off: Auto mode has the overhead of two proxies in the path. If you know an application is always download-heavy or always upload-heavy, a single-ended mode (reverse or forward proxy) may be enough and uses fewer resources.

5. Configuration Components

TCP optimization is configured with three building blocks:

1.   TCP profiles (Objects): define congestion control, loss detection, buffers and so on. Typically one for the LAN leg and one for the WAN leg.

2.   Address objects / address groups: identify the traffic to optimize.

3.   SD-WAN policy rule with a tcp-optimization action: references the profiles, sets the mode and sets the bypass-latency-threshold.

Example: Auto mode for SCP between two branches

Topology: Branch-1 (host 10.40.40.100) and Branch-2 (host 10.20.20.100), connected over SD-WAN with latency in the transport. Both branches run the same configuration.

TCP profiles

show configuration orgs org-services Versa objects tcp-profiles

 

lan_opt {

    description        "lan profile";

    congestion-control cubic;

    hybrid-slow-start  false;

    send-buffer        1024;

    receive-buffer     1024;

    rate-pacing        false;

}

wan_opt {

    description        "wan profile";

    congestion-control bbr;

    loss-detection     rack;

    hybrid-slow-start  false;

    send-buffer        6144;

    receive-buffer     6144;

    rate-pacing        false;

}

Address objects and group

show configuration orgs org-services Versa objects addresses

usa-host-1     { ipv4-prefix 10.40.40.100/32; }

usa-hub-host-2 { ipv4-prefix 10.20.20.100/32; }

 

show configuration orgs org-services Versa objects address-groups

usa-hosts {

    description  "for scp";

    address-list [ usa-host-1 usa-hub-host-2 ];

}

SD-WAN policy rule

show configuration orgs org-services Versa sd-wan policies Default-Policy rules tcp-optimize-scp

 

rule-disable false;

match {

    source {

        address { address-group-list [ usa-hosts ]; }

        user    { user-type any; }

    }

    destination {

        address { address-group-list [ usa-hosts ]; }

    }

    services { predefined-services-list [ TCP ]; }

}

set {

    action allow;

    lef { event never; rate-limit 10; }

    tcp-optimization {

        lan-profile              lan_opt;

        wan-profile              wan_opt;

        mode                     auto;

        bypass-latency-threshold 10;

    }

}

Parameters to note

Parameter

Meaning

lan-profile

TCP profile applied to the leg between the VOS device and the local host

wan-profile

TCP profile applied to the leg between the two VOS devices

mode auto

Dual-ended mode with automatic peer discovery

bypass-latency-threshold 10

Skip optimization when SYN to SYN-ACK latency is below 10 ms. Set it to 0 if the path has low latency but still suffers packet loss; otherwise optimization is silently skipped.

 

6. Testing and Verification

Test results

#

Condition

Result

1

No added latency, no TCP optimization

SCP transfer completed in about 1 second at about 11.9 MB/s

2

5 ms latency added, no TCP optimization

Throughput dropped to about 150 KB/s and the session stalled

3

5 ms latency added, TCP optimization configured on both branches

Transfer completed in about 1.41 seconds. The source document lists the speed as 106 KB/s, which looks inconsistent with the completion time; confirm the unit (likely MB/s) before publishing.

 

Verification commands

1. Confirm the session is TCP-terminated and optimized

show orgs org Versa sessions tcp-terminated brief

Expected: TCP OPTIMIZED = Yes and state Established.

2. Inspect each leg of the split session

show orgs org Versa sessions tcp-terminated detail

 

sessions tcp-terminated detail 0 2 231

 

source-ip        10.20.20.100

destination-ip   10.40.40.100

source-port      38064

destination-port 22

protocol         6

natted           No

sdwan            Yes

application      ssh/(predef)

tcp-optimized    Yes

client state                Established

client send-queue-count     0

client retransmit-queue-count 2

client max-send-buffer      1048576

client send-window          958080

client send-window-scale-factor 7

client send-space           31608

client max-receive-buffer   1048576

client receive-window       65567

client receive-window-scale-factor 7

client advertised-window    65567

client send-maximum-segment-size 1136

client recovery-algorithm   pipe

client loss-detection       duplicate-ack

client congestion-algorithm cubic

client congestion-state     open

client slow-start-threshold 2097120

client congestion-window    959188

client retransmission-timeout 202

client smoothed-rtt         2

client rtt-variance         50

client send-high-water-mark 32768

client receive-high-water-mark 65535

client reordering           3

client number-rtos          0

client fast-recoveries      0

client peer-id              0

client tcp-options          "SACK|TStamp|WndScale"

server state                Established

server send-queue-count     0

server retransmit-queue-count 0

server max-send-buffer      6291456

server send-window          65536

server send-window-scale-factor 8

server send-space           32768

server max-receive-buffer   6291456

server receive-window       65535

server receive-window-scale-factor 8

server advertised-window    65535

server send-maximum-segment-size 1148

server recovery-algorithm   pipe

server loss-detection       rack

server congestion-algorithm bbr

server congestion-state     open

server slow-start-threshold 2097120

server congestion-window    82656

server retransmission-timeout 295

server smoothed-rtt         95

server rtt-variance         50

server send-high-water-mark 32768

server receive-high-water-mark 65535

server reordering           3

server number-rtos          0

server fast-recoveries      0

server peer-id              104

server tcp-options          "SACK|TStamp|WndScale|PeerDisc"

The output has a client section and a server section. In an Auto-mode session, the two sections show the separate TCP legs on this device:

Field

Client leg (LAN-facing)

Server leg (WAN-facing)

Congestion algorithm

cubic (from lan_opt)

bbr (from wan_opt)

Loss detection

duplicate-ack

rack

Max send / receive buffer

1048576 (1 MB)

6291456 (6 MB)

TCP options

SACK, TStamp, WndScale

SACK, TStamp, WndScale, PeerDisc

Peer ID

0

104 (the discovered remote VOS peer)

 

What this shows:

3. Check per-rule statistics

show orgs org-services Versa sd-wan policies Default-Policy rules statistics tcp-optimization brief

rules statistics tcp-optimization brief tcp-optimize-scp

sessions-optimized        1

sessions-bypassed         2

peer-discovery-success    1

splice-success            0

bypass-configured         1

bypass-non-tcp            0

bypass-non-syn            1

bypass-tracking-failure   0

bypass-split-failure      0

bypass-already-terminated 0

bypass-no-resources       0

bypass-no-peer            0

bypass-latency            0

bypass-error              0

bypass-inv-peer-id        0

bypass-non-syn-ack        0

bypass-inv-syn-ack        0

bypass-inv-last-ack       0

client fast-recoveries     0

client average-fast-recovery-time 0

client rto-recoveries      0

client average-rto-recovery-time 0

client tlp-recoveries      0

server fast-recoveries     0

server average-fast-recovery-time 0

server rto-recoveries      0

server average-rto-recovery-time 0

Here 1 session was optimized and peer discovery succeeded. The 2 bypassed sessions are explained by bypass-configured (1) and bypass-non-syn (1). A bypass-non-syn session is one where VOS did not see the TCP SYN, so it cannot start a split.

These counters are cumulative. They start counting from a router restart, creation of the rule, or when statistics are cleared.

Bypass counter reference

Counter

Meaning / what to check

bypass-latency

Path latency was below bypass-latency-threshold. Lower the threshold (or set it to 0) if the path has loss despite low latency.

bypass-no-peer

No remote VOS peer was discovered. Check that the far-end device has a matching TCP optimization rule in an appropriate mode.

bypass-no-resources

Insufficient resources (mbuf) on the device.

bypass-non-tcp

The matched traffic was not TCP.

bypass-non-syn

The first packet seen was not a SYN (for example, a session that was already in progress when the rule took effect).

bypass-configured

Traffic matched a rule configured to bypass.

bypass-tracking-failure, bypass-split-failure, bypass-already-terminated, bypass-error, bypass-inv-peer-id, bypass-non-syn-ack, bypass-inv-syn-ack, bypass-inv-last-ack

Error or abnormal conditions during session setup or the split. Investigate if they increment.

 

7. Troubleshooting Checklist

Symptom

Likely cause and action

sessions-optimized stays at 0

Check that the rule matches the traffic (source, destination, service), that the mode is not set to bypass, and that the rule is in the correct position in the policy order.

bypass-no-peer increments

The remote VOS is not configured or not matching. Confirm Auto mode (or the correct single-ended counterpart) on the far end.

bypass-latency increments but throughput is still poor

Latency is under the threshold but loss may be present. Set bypass-latency-threshold to 0 or a lower value.

Optimization is active on both sides but the transfer is still slow

The bottleneck may be outside the network, for example the application or server tuning (such as SMB signing or buffering).

Session shows tcp-optimized No

Confirm that VOS saw the TCP three-way handshake. Sessions that were established before the rule was applied are not optimized.