Question 1. How to configure ssh over the WAN-Routing Instance?
Answer:
The easiest way is to use the workflows to allow ssh, which will automatically create all the required configurations for us, and just a re-deploy is only required for the existing Workflow, if Workflow cannot be re-deployed, please use the device template and follow [Question 2] to translate it to WAN facing interface.
Question 2. How to configure ssh over the LAN-Routing instance?
Answer:
Below is the configuration required:
###Add the tvi interfaces ###
admin@Snehal-Home-Office-cli> show configuration interfaces tvi-0/2602
description "Allow SSH paired TVI for LAN link ";
type paired;
paired-interface tvi-0/2603;
unit 0 {
enable true;
family {
inet {
address 169.254.7.210/31;
}
}
}
[ok][2021-04-24 08:28:25]
admin@Snehal-Home-Office-cli> show configuration interfaces tvi-0/2603
description "Allow SSH paired TVI for LAN link-Globalnamespace";
type paired;
paired-interface tvi-0/2602;
unit 0 {
enable true;
family {
inet {
address 169.254.7.211/31;
}
}
}
### Include the interfaces tvi-0/2602.0 and tvi-0/2603.0 to the traffic Identification under Limits of the Org: ###
admin@Snehal-Home-Office-cli(config)% show orgs org traffic-identification org Corp-Inline-Customer-1
{
traffic-identification {
using [ ptvi4 ptvi5 tvi-0/2602.0 tvi-0/2603.0 tvi-0/4.0 tvi-0/5.0 tvi-0/603.0 ];
using-networks [ LAN1 LAN2 LAN3 ];
}
}
### Create a CGNAT twice-napt-44 rule as shown below ###
admin@Snehal-Home-Office-cli(config)% show orgs org-services Corp-Inline-Customer-1 cgnat pools
HOST_BOUND_DPOOL-LAN {
address [ 169.254.7.211/32 ];
destination-port-range {
low 22;
high 22;
}
}
HOST_BOUND_SPOOL-LAN {
address [ 169.254.7.210/32 ];
routing-instance Corp-Inline-Customer-1-LAN-VR;
source-port {
allocation-scheme automatic;
random-allocation;
}
}
admin@Snehal-Home-Office-cli(config)% show orgs org-services Corp-Inline-Customer-1 cgnat rules HOST_BOUND_RULE-LAN
precedence 102;
from {
routing-instance Corp-Inline-Customer-1-LAN-VR;
destination-network [ LAN2 ];
protocol 6;
destination-port-range {
low 22;
high 22;
}
}
then {
translated {
translation-type twice-napt-44;
source-pool HOST_BOUND_SPOOL-LAN;
destination-pool HOST_BOUND_DPOOL-LAN;
}
}
### Please put the interfaces in the zone ###
admin@Snehal-Home-Office-cli(config)% show orgs org-services Corp-Inline-Customer-1 objects zones
HOST-LAN-GRT-Zone {
interface-list [ tvi-0/2603.0 ];
}
HOST-LAN-Zone {
interface-list [ tvi-0/2602.0 ];
}
### Security access policy needed here is, ###
access-policy "Allow_SSH_From_LAN" {
match {
destination {
zone {
zone-list [ "HOST-LAN-Zone" ];
}
}
services {
predefined-services-list [ "ssh" ];
}
source {
zone {
zone-list [ "Intf-LAN-Zone" ];
}
}
}
set {
action "allow";
}
}
access-policy "Allow_SSH_From_LAN_GRT" {
match {
services {
predefined-services-list [ "ssh" ];
}
source {
zone {
zone-list [ "HOST-LAN-GRT-Zone" ];
}
}
}
set {
action "allow";
}
}
## Include the LAN-VR facing interface i.e. tvi-0/2602 and if static redistribution is not included, please do the same ###
admin@Snehal-Home-Office-cli(config)% show routing-instances Corp-Inline-Customer-1-LAN-VR interfaces
interfaces [ tvi-0/2602.0 tvi-0/603.0 ];Question 3: How does a session for ssh look like?
Answer:
Question 4: Troubleshoot steps to take place in case ssh fails
Answer:
1) Please take a tcpdump on the Interface [WAN/LAN] from where the client is expected to ssh from.
admin@Snehal-Home-Office-cli> tcpdump vni-0/X filter "'port 22 -c 1000'"
Starting capture on wifi-mgmt-0/0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on _wifi, link-type EN10MB (Ethernet), capture size 262144 bytes
09:01:09.431516 18:1d:ea:b8:a2:9a > 8e:16:de:99:34:67, ethertype IPv4 (0x0800), length 56: 172.16.77.12.63386 > 172.16.77.1.22: Flags [.], ack 703044892, win 509, length 0
09:01:09.467275 8e:16:de:99:34:67 > 18:1d:ea:b8:a2:9a, ethertype IPv4 (0x0800), length 438: 172.16.77.1.22 > 172.16.77.12.63386: Flags [P.], seq 1:385, ack 0, win 460, length 384
09:01:09.507275 8e:16:de:99:34:67 > 18:1d:ea:b8:a2:9a, ethertype IPv4 (0x0800), length 294: 172.16.77.1.22 > 172.16.77.12.63386: Flags [P.], seq 385:625, ack 0, win 460, length 240
09:01:09.507283 18:1d:ea:b8:a2:9a > 8e:16:de:99:34:67, ethertype IPv4 (0x0800), length 56: 172.16.77.12.63386 > 172.16.77.1.22: Flags [.], ack 625, win 513, length 0
2) If the packets are making it to the Interface, the next step would be to see if the infmgr forwards it to the globalname space
Here we do see the traffic from the LAN-VR i.e. 169.254.7.210 is forwarded to the global namespace routing instance with IP 169.254.7.211 Port 22
[admin@Snehal-Home-Office: ~] $ sudo tcpdump -nli any host 169.254.7.210 -c 100
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
09:07:14.962172 IP 169.254.7.210.36893 > 169.254.7.211.22: Flags [.], ack 703078268, win 510, length 0
09:07:14.962307 IP 169.254.7.211.22 > 169.254.7.210.36893: Flags [P.], seq 1:161, ack 0, win 642, length 160
09:07:14.962388 IP 169.254.7.211.22 > 169.254.7.210.36893: Flags [P.], seq 161:321, ack 0, win 642, length 160
09:07:14.962445 IP 169.254.7.211.22 > 169.254.7.210.36893: Flags [P.], seq 321:497, ack 0, win 642, length 176
09:07:14.962501 IP 169.254.7.211.22 > 169.254.7.210.36893: Flags [P.], seq 497:673, ack 0, win 642, length 176
09:07:14.962559 IP 169.254.7.211.22 > 169.254.7.210.36893: Flags [P.], seq 673:849, ack 0, win 642, length 176
09:07:14.962614 IP 169.254.7.211.22 > 169.254.7.210.36893: Flags [P.], seq 849:1025, ack 0, win 642, length 176
09:07:14.962669 IP 169.254.7.211.22 > 169.254.7.210.36893: Flags [P.], seq 1025:1201, ack 0, win 642, length 176
09:07:14.962724 IP 169.254.7.211.22 > 169.254.7.210.36893: Flags [P.], seq 1201:1377, ack 0, win 642, length 176
09:07:14.962779 IP 169.254.7.211.22 > 169.254.7.210.36893: Flags [P.], seq 1377:1553, ack 0, win 642, length 176
09:07:14.962834 IP 169.254.7.211.22 > 169.254.7.210.36893: Flags [P.], seq 1553:1729, ack 0, win 642, length 176
09:07:15.017420 IP 169.254.7.210.36893 > 169.254.7.211.22: Flags [.], ack 1729, win 513, length 0
3) Please make sure that there is no restriction added here while the Appliance was onboarded to only allow it to a set of IP's or from a specific IP, which may be one of the reasons why ssh to the appliance is failing.
[admin@Snehal-Home-Office: ~] $ cat /etc/ssh/sshd_config
4) To review for any ssh key exchange algorithms issues/mismatch between the client and the servers, the log to look for is,
/var/log/syslog
To review the ssh key algorithms on the VOS,
admin@Branch-cli(config)% show confdConfig ssh
algorithms {
kex diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha256;
mac hmac-sha2-256,hmac-sha2-512;
encryption aes128-ctr,aes192-ctr,aes256-ctr;
}
clientAliveInterval 30s;
clientAliveCountMax 3;
5) If the packets are not making it to the global namespace telnet to the infmgr and check if the TAP tx/rx count is incrementing, by doing 6-8 iterations while the end-client is trying to ssh to the VOS
[admin@Snehal-Home-Office: ~] $ vsh connect infmgr
infmgr> show stats vsm
infmgr> show stats vsm
-------------------------------------------------------------------------------------------------------------------------------------------
| Inf Disc Phy-State VSN-Rx TAP-Tx TAP-Rx VSN-Tx Mirror_RX Mirror_TX Stats-Clr Proto-up/down MTU-Updt |
-------------------------------------------------------------------------------------------------------------------------------------------
| tvi-0/2603.0 0 0 3675 3675 0 0 0 0 0 0 0 |
| tvi-0/2603 0 0 0 0 3465 3465 0 0 0 0 0 |
6) If you see that intermittently ssh seems to work, and it fails only for certain tries, then it can be a reason where ssh IP may be enlisted in fail2ban
Steps to revive this is:
admin@Snehal-Home-Office-cli> show jail ssh
Status for the jail: ssh
|- filter
| |- File list: /var/log/syslog
| |- Currently failed: 0
| `- Total failed: 2
`- action
|- Currently banned: 1
| `- IP list: 169.254.7.210
`- Total banned: 1
[ok][2021-04-24 09:48:48]
If there is an IP that you see in the list, then to free the IP, please use the below command:
Sample:
admin@Snehal-Home-Office-cli> request clear jail ssh ip 169.254.7.210
169.254.7.210
As of [4/24/2021] from 21.2.1 GA, we have an option to enable to disable fail2ban and tweak the max-retries count
admin@Snehal-Home-Office-CLI(config)% show system login ban | detail
ban enabled;
admin@Snehal-Home-Office-cli(config)% show system login max-retries | details
max-retries 6;
But for older release which is released prior to 4/24/2021, below is the steps to enable/disable and increase max-retry for fail2ban:
Goto,
[admin@Snehal-Home-Office: ~] $ cat /etc/fail2ban/jail.local
# Versa Fail2Ban jail config file.
enabled = true <<<<<<<<<--- [Default: true, Options: true/false]
port = ssh
filter = sshd
logpath = /var/log/syslog
maxretry = 5 <<<<<<<<<<<---[ Default: 5]
PS: After this file update, we need to restart the versa-services. [vsh restart, please perform this operation during a maintenance window].
Note:
From 20.2 and above, we would recommend customers to move away from adding interfaces and Client-IPs to the vnf-manager for ssh purposes and instead recommend using the procedure provided in the config section of the kb.
Not recommended:
admin@Snehal-Home-Office-cli(config)% show system vnf-manager
ip-addresses [ 10.200.1.3/32 10.200.1.4/32 10.200.1.5/32 172.16.77.12/32 ]; <<---Client IP from where you ssh
vnf-mgmt-interfaces [ tvi-0/3.0 vni-0/200.0 ]; <<<---lan/wan interface for SSH
If you are having concerns or queries regarding ssh after reviewing the above kb, please feel free to open a support ticket at support@versa-networks.com