Observation: Director services will restart or it will stop and may not start automatically.

Troubleshooting: Look for logs around the time when Versa Services stopped/restarted.

In this use case, versa services stopped at 15:15 on 4th March.

Syslog (/var/log/syslog) pointing to ncsd process being terminated. 

Mar  4 15:15:55 vlab2-dcb-director1 kernel: [9769775.280901] init: ncsd main process (26777) terminated with status 19

Mar  4 15:15:55 vlab2-dcb-director1 kernel: [9769775.280921] init: ncsd main process ended, respawning

NCS logs (/var/log/vnms/ncs/ncs.log) will show error message as highlighted.

<INFO> 4-Mar-2021::15:15:52.068 vlab2-dcb-director1 ncs[26777]: - Stopping to listen for Internal IPC on 0.0.0.0:4566

<CRIT> 4-Mar-2021::15:15:52.105 vlab2-dcb-director1 ncs[26777]: - Internal error: Supervision terminated

<INFO> 4-Mar-2021::15:15:55.630 vlab2-dcb-director1 ncs[15672]: - Enabling error log

<INFO> 4-Mar-2021::15:15:55.635 vlab2-dcb-director1 ncs[15672]: - Writing error log to /var/log/vnms/ncs/ncserr.log

Cause: This issue is seen due to an internal code level error in NCS process.

Issue found in: 20.2.2

Fixed in: 21.2.1, 21.1.3, 20.2.4