uyuy2043 / vitalpbx_ha

VitalPBX High Availability

VitalPBX High Availability (Version 3)

High availability is a characteristic of a system which aims to ensure an agreed level of operational performance, usually uptime, for a higher than normal period.

Make a high-availability cluster out of any pair of VitalPBX servers. VitalPBX can detect a range of failures on one VitalPBX server and automatically transfer control to the other server, resulting in a telephony environment with minimal down time.

⚠️Important notes:

• Since DRBD is no longer used in version 3, it is not possible to migrate a High Availability VitalPBX from Version 2 to 3.
  
• If you are going to restore some Backup from another server that is not in HA, restore it first in the Master server before creating the HA. This should be done as the backup does not contain the firewall rules for HA to work.
  
• The VitalPBX team does not provide support for systems in an HA environment because it is not possible to determine the environment where it has been installed.

Example:

---

Prerequisites

In order to install VitalPBX in high availability you need the following:
a.- 3 IP addresses.
b.- Install VitalPBX Version 3.0 in two servers with similar characteristics.
c.- MariaDB Galera (include in VitalPBX 3).
d.- Corosync, Pacemaker, PCS and lsyncd.
e.- Root user is required for both servers to communicate.
f.- Both servers will not be able to have a proxy since this affects the communication between them.

Configurations

We will configure in each server the IP address and the host name. Go to the web interface to: Admin>System Settinngs>Network Settings.
First change the Hostname, remember press the Check button.
Disable the DHCP option and set these values

Name	Master	Standby
Hostname	vitalpbx1.local	vitalpbx2.local
IP Address	192.168.10.61	192.168.10.62
Netmask	255.255.255.0	255.255.255.0
Gateway	192.168.10.1	192.168.10.1
Primary DNS	8.8.8.8	8.8.8.8
Secondary DNS	8.8.4.4	8.8.4.4

Bind Address

In the Master server go to SETTINGS/PJSIP Settings and configure the Floating IP that we are going to use in "Bind" and "TLS Bind". Also do it in SETTINGS/SIP Settings Tab NETWORK fields "TCP Bind Address" and "TLS Bind Address".

Install Dependencies

Install the necessary dependencies on both servers

[root@vitalpbx1 ~]# yum -y install corosync pacemaker pcs lsyncd vitalpbx-high-availability
[root@vitalpbx2 ~]# yum -y install corosync pacemaker pcs lsyncd vitalpbx-high-availability

Create authorization key for the Access between the two servers without credentials

Create key in Server 1

[root@vitalpbx1 ~]# ssh-keygen -f /root/.ssh/id_rsa -t rsa -N '' >/dev/null
[root@vitalpbx1 ~]# ssh-copy-id root@192.168.10.62
Are you sure you want to continue connecting (yes/no)? yes
[email protected]'s password: (remote server root’s password)

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh '[email protected]'"
and check to make sure that only the key(s) you wanted were added. 

[root@vitalpbx1 ~]#

Create key in Server 2

[root@vitalpbx2 ~]# ssh-keygen -f /root/.ssh/id_rsa -t rsa -N '' >/dev/null
[root@vitalpbx2 ~]# ssh-copy-id root@192.168.10.61
Are you sure you want to continue connecting (yes/no)? yes
[email protected]'s password: (remote server root’s password)

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh '[email protected]'"
and check to make sure that only the key(s) you wanted were added. 

[root@vitalpbx2 ~]#

Script

Now copy and run the following script

[root@ vitalpbx1 ~]# mkdir /usr/share/vitalpbx/ha
[root@ vitalpbx1 ~]# cd /usr/share/vitalpbx/ha
[root@ vitalpbx1 ~]# wget https://raw.githubusercontent.com/VitalPBX/vitalpbx_ha/master/vpbxha.sh
[root@ vitalpbx1 ~]# chmod +x vpbxha.sh
[root@ vitalpbx1 ~]# ./vpbxha.sh

************************************************************
*  Welcome to the VitalPBX high availability installation  *
*                All options are mandatory                 *
************************************************************
IP Master................ > 192.168.10.61
IP Standby............... > 192.168.10.62
Floating IP.............. > 192.168.10.60
Floating IP Mask (SIDR).. > 24
hacluster password....... > MyPassword (any password)
************************************************************
*                   Check Information                      *
*        Make sure you have internet on both servers       *
************************************************************
Are you sure to continue with this settings? (yes,no) > yes

At the end of the installation you have to see the following message

************************************************************
*                VitalPBX Cluster OK                       *
*    Don't worry if you still see the status in Stop       *
*  sometimes you have to wait about 30 seconds for it to   *
*                 restart completely                       *
*         after 30 seconds run the command: role           *
************************************************************

 _    _ _           _ ______ ______ _    _
| |  | (_)_        | (_____ (____  \ \  / /
| |  | |_| |_  ____| |_____) )___)  ) \/ /
 \ \/ /| |  _)/ _  | |  ____/  __  ( )  (
  \  / | | |_( ( | | | |    | |__)  ) /\ \
   \/  |_|\___)_||_|_|_|    |______/_/  \_\


 Role           : Master
 Version        : 3.0.0-5
 Asterisk       : 17.6.0
 Linux Version  : CentOS Linux release 7.8.2003 (Core)
 Welcome to     : vitalpbx1.local
 Uptime         :  1:30
 Load           : Last Minute: 0.74, Last 5 Minutes: 0.30, Last 15 Minutes: 0.16
 Users          : 4 users
 IP Address     : 192.168.10.61 192.168.10.60
 Clock          : Wed 2020-08-05 09:04:19 EDT
 NTP Sync.      : no


************************************************************
*                  Servers Status                          *
************************************************************
Master
 virtual_ip     (ocf::heartbeat:IPaddr2):       Started vitalpbx1.local
 asterisk       (service:asterisk):     Started vitalpbx1.local
 lsyncd (service:lsyncd.service):       Started vitalpbx1.local

Servers Status
  vitalpbx1.local: Online
  vitalpbx2.local: Online

Change Servers Role

To execute the process of changing the role, we recommend using the following command:

[root@vitalpbx-master /]# bascul
************************************************************
*     Change the roles of servers in high availability     *
* WARNING-WARNING-WARNING-WARNING-WARNING-WARNING-WARNING  *
*All calls in progress will be lost and the system will be *
*     be in an unavailable state for a few seconds.        *
************************************************************
Are you sure to switch from vitalpbx1.local to vitalpbx2.local? (yes,no) >

This action convert the vitalpbx1.local to Standby and vitalpbx2.local to Master. If you want to return to default do the same again.

Next we will show a short video how high availability works in VitalPBX

Recommendations

If you have to turn off both servers at the same time, we recommend that you start by turning off the one in Standby and then the Master
If the two servers stopped abruptly, always start first that you think you have the most up-to-date information and a few minutes later the other server
If you want to update the version of VitalPBX we recommend you do it first on Server 1, then do a bascul and do it again on Server 2

Dahdi with Xorcom HA Hardware - Only install if you are using physical telephony cards

If you are going to install Dhadi with Xorcom HA Hardware we recommend you to execute the following commands in the Server 1

[root@vitalpbx1 ~]# systemctl stop dahdi
[root@vitalpbx1 ~]# systemctl disable dahdi
[root@vitalpbx1 ~]# pcs resource create dahdi service:dahdi op monitor interval=30s
[root@vitalpbx1 ~]# pcs cluster cib fs_cfg
[root@vitalpbx1 ~]# pcs cluster cib-push fs_cfg --config
[root@vitalpbx1 ~]# pcs -f fs_cfg constraint colocation add dahdi with virtual_ip INFINITY
[root@vitalpbx1 ~]# pcs -f fs_cfg constraint order lsyncd then dahdi
[root@vitalpbx1 ~]# pcs cluster cib-push fs_cfg --config

and in the Server 2

[root@vitalpbx2 ~]# systemctl stop dahdi
[root@vitalpbx2 ~]# systemctl disable dahdi

Update VitalPBX version

To update VitalPBX to the latest version just follow the following steps:
1.- From your browser, go to ip 192.168.10.60
2.- Update VitalPBX from the interface
3.- Execute the following command in Master console

[root@vitalpbx1 /]# bascul

4.- From your browser, go to ip 192.168.10.60 again
5.- Update VitalPBX from the interface
6.- Execute the following command in Master console

[root@vitalpbx1 /]# bascul

Some useful commands

• bascul, is used to change roles between high availability servers. If all is well, a confirmation question should appear if we wish to execute the action.
• role, shows the status of the current server. If all is well you should return Masters or Slaves.
• pcs resource refresh --full, to poll all resources even if the status is unknown, enter the following command.
• pcs cluster unstandby host, in some cases the bascul command does not finish tilting, which causes one of the servers to be in standby (stop), with this command the state is restored to normal.

If the databases lose synchronization

[root@vitalpbx1 /]# mysql -uroot
mysql> STOP SLAVE;
mysql> SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 1;
mysql> START SLAVE;

More Information

If you want more information that will help you solve problems about High Availability in VitalPBX we invite you to see the following manual
High Availability Manual, step by step

CONGRATULATIONS, you have installed and tested the high availability in VitalPBX 3
👍