Question

How to archive and restore data in Versa Analytics?

Solution

Versa Analytics platform supports periodic archiving of log data. 

You can use the archived data:

  • to recover a cluster.
  • to analyze a non-existing data in the database.


Archiving Logs

Versa Analytics nodes save the logs received from FlexVNFs as files in the local file system in the /tenant/appliance directory. After the logs in these files are transferred to database, they are moved to the backup directory. The files in the backup directory are archived periodically using a cron job. The default archival interval is one hour, and the default archive directory is /var/tmp/archive.


You can change the archive cron job interval/directory by running the following script on the Versa Analytics shell:

# sudo /opt/versa/scripts/van-scripts/log-archive-start  /var/tmp/log  /var/tmp/archive  hourly
# sudo /opt/versa/scripts/van-scripts/log-archive-stop


NOTE: The default source directory is /var/tmp/log.


Example of the archived logs in the archive directory:

versa@van2-vm2:/var/tmp/archive/tenant-Customer1/VSN0-Branch1$ ls 20170612*
-rw-r--r-- 1 root root   9816 Jun 12 01:00 20170612.0.tar.gz
-rw-r--r-- 1 root root   9366 Jun 12 02:00 20170612.1.tar.gz
::::::::::::
-rw-r--r-- 1 root root 340816 Jun 12 23:00 20170612.21.tar.gz
-rw-r--r-- 1 root root 108935 Jun 13 00:00 20170612.22.tar.gz


Restoring Logs

You can restore the archived logs by running the following script on the Versa Analytics shell:

# sudo /opt/versa/scripts/van-scripts/log-restore.py 
usage: --src <src-path>  --dst <dst-path> 
optional: --start-date <yyyy/mm/dd>
optional: --end-date <yyyy/mm/dd>
optional: --tenant <tenant> 
optional: --appliance <appliance>

NOTE:  If optional parameters are not specified, all the logs under the source directory are restored. By default, the destination directory is /var/tmp/log.

Example:

# sudo /opt/versa/scripts/van-scripts/log-restore.py --src /var/tmp/archive/ --dst /var/tmp/log --tenant tenant-Customer1 --appliance VSN0-Branch1 --start-date 2017/06/12 --end-date 2017/06/13


Transferring Archived Logs

Transferring of archived logs from Versa Analytics to an external storage is necessary to recover data if a disk fails, or a node crashes.


You can transfer the data to external storage, by running the following script, which uses the rsync utility to sync data:

# sudo /opt/versa/scripts/van-scripts/log-archive-transfer.py --src <src-path> --user <username> --dst-host <ip or host> --dst <dst-path> [--ssh-key <ssh-key-path]
Field
 Description
src
 Path of the archive directory. Default value is /var/tmp/archive.
dest
 Directory on the target where the files will be
user
 Login username on the target node.
dest-host
IP address or hostname of the remote m/c.
ssh-key path
 Path where the ssh-key is stored to access the remote node without password.


This script can be called within a cron job. The output of the script is stored in /var/log/versa/versa-log-archive-transfer.log.


Example:

# sudo /opt/versa/scripts/van-scripts/log-archive-transfer.py --src /var/tmp/archive --dst /var/tmp --dst-host 10.40.24.3 --user versa


NOTE: If a firewall is involved between the analytics node and the destination, then ensure that rsync port (873) is open.