PBS setup master node

Home > CentOS > CentOS 7.x > CentOS 7.x Rocks cluster 7.0 > PBS setup master node

Setup new PBS Master node

To setup PBS master node use:

1. Download pbs packages

   wget http://wpc.23a7.iotacdn.net/8023A7/origin2/rl/PBS-Open/pbspro_19.1.2.centos_7.zip
   unzip pbspro_19.1.2.centos_7.zip

2. Install PBS on master server

   cd pbspro_19.1.2.centos_7
   yum localinstall -y ./pbspro-server-19.1.2-0.x86_64.rpm

3. Edit pbs conf file ('/etc/pbs.conf') so that it has following contents:

   PBS_EXEC=/opt/pbs
   PBS_SERVER=<rocks-master-hostname>    //Ex: master
   PBS_START_SERVER=1
   PBS_START_SCHED=1
   PBS_START_COMM=1
   PBS_START_MOM=1
   PBS_HOME=/var/spool/pbs
   PBS_CORE_LIMIT=unlimited
   PBS_SCP=/bin/scp
   PBS_LEAF_NAME=<rocks-master-hostname>
   

   Here rocks-master-hostname can be only hostname such as rocks2 which resolves to master node private IP

   Notice that PBS_START_MOM=1 in above output so that we can execute jobs on master also. If job execution on master is not desired then that can be left as 0

   Notice PBS_LEAF_NAME has been appended manually, it is not present by default

4. Change the path in current running system using:

   source /etc/profile.d/pbs.sh
   source /etc/pbs.conf

   Note that in case of rocks the above will add pbs path after current paths. Hence if we run some command which is common to SGE () and PBS then SGE command will run unless we exit from current terminal and reopen new terminal. This can be validated using echo $PATH. /opt/pbs/bin should be before /opt/gridengine/bin in path output

5. Enter master hostname in mom config file '/var/spool/pbs/mom_priv/config' as:

   $clienthost <rocks-master-hostname>

   For example

   $clienthost master

   Here rocks-master-hostname can be only hostname such as rocks2 which resolves to master node private IP

6. In pbs server, needs to specify all the compute/gpu nodes at '/var/spool/pbs/server_priv/nodes'. Example listing

   compute-0-0
   compute-0-1
   rocksmaster

   If there are 4 hosts then output of below command can help in getting host list. For additional hosts change the tail -4 command appropriately.

   rocks list host | cut -b 1-12 | sed 's/://' | tail -4

7. Start the pbs

   systemctl start pbs

Refer:

• http://community.pbspro.org/t/step-by-step-install-document/814/10
• https://github.com/PBSPro/pbspro/blob/master/INSTALL
• http://community.pbspro.org/t/qstat-cannot-connect-to-server-single-node-cluster/1504

Fixing rocks scheduler to use private names

As per rocks architecture, master node has two hostnames. One is public hostname and another one is private hostname

In OpenPBS, all the communication is happening through hostname only. We have to configure OpenPBS with private hostname.

Whenever rocks master restarts, OpenPbs service will start automatically. pbs_sched starts with public hostname. So we need to start the pbs_sched using private hostname

To ensure that PBS works properly with rocks use:

1. Configure clientfile also on master (Typically master is used as client for submitting jobs) '/var/spool/pbs/sched_priv/clientfile'

   $clienthost <master-hostname>

2. Kill pbs_sched

   killall -9 pbs_sched

3. Start pbs_sched by the following method

   /opt/pbs/sbin/pbs_sched -c /var/spool/pbs/sched_priv/clientfile

4. To automate all the three steps create '/root/restart-pbs.sh' with:

   #!/bin/bash
   
   systemctl restart pbs
   killall -9 pbs_sched
   /opt/pbs/sbin/pbs_sched -c /var/spool/pbs/sched_priv/clientfile
   

5. Make the file executable

   chmod +x /root/restart-pbs.sh

   Use this script instead of "systemctl restart pbs" on master.

6. Also add below to '/etc/rc.d/rc.local' so that cluster works properly after reboot

   /root/restart-pbs.sh

Refer:

• https://community.openpbs.org/t/proper-way-to-configure-pbs-on-multiple-nic-system/1508/2

Check installed PBS version

To check installed PBS version use:

pbs-config --version

Refer:

• https://serverfault.com/questions/570491/how-to-find-out-the-pbs-version-installed-in-a-cluster

Home > CentOS > CentOS 7.x > CentOS 7.x Rocks cluster 7.0 > PBS setup master node