RHEL GFS2 Clustered Filesystem on vSphere

Setup Clustered Storage

1. Add LUN as RDM, use new SCSI controller and set controller sharing to Physical and disk compatibility mode to virtual.


When adding the disk to second node, specify existing disk and locate RDM file on first node.

2. Install VMware tools on each node
3. Install the needed packages:

High Availability
Resilient Storage
High Availability Management (head node only)

4. Turn off firewall
5. Turn off selinux
6. Change password for ricci
7. Set following service to start on boot and start the services


8. Edit /etc/hosts and add in each host

9. Browse https://<head node>:8084, login in using root credentials.

10. Create cluster


11. Check cluster status using command clustat


13. Check node status using cman_tool status


14. Enable Clustered LVM using command lvmconf

lvmconf –enable-cluster

15. Create physical volume using pvcreate

pvcreate /dev/sdb

16. Create clustered volume group using vgcreate

vgcreate -c y alf_data /dev/sdb

17. Create logical volume using lvcreate

lvcreate -l 512002 -n alf_data-lv alf_data

18. Make gfs2 filesystem

mkfs.gfs2 -p lock_dlm -t bhaalfcluster:alf_data -j 4 /dev/mapper/alf_data-alf_data—lv

19. Update /etc/fstab to mount the filesystem to /var/alf_data at startup

echo “/dev/mapper/alf_data-alf_data–lv /var/alf_data gfs2 defaults,noatime,nodiratime 0 0” >> /etc/fstab

Setup Fencing

vCenter server: bhavc01
vCenter Fencing Account: fence (Local account on bhavc01)
1st node of 2 node cluster: bhaalfapp1.bhadmz (Named bhaalfapp1 in vCenter)
2nd node of 2 node cluster: bhaalfapp2.bhadmz (Named bhaalfapp2 in vCenter)

1, Get the UUID of your two nodes

a) On bhaalfapp1.bhadmz, run the following:

fence_vmware_soap –ip –username fence –password Pa55word -z –action list | grep BHAALFAPP1

fence_vmware_soap –ip –username fence –password Pa55word -z –action list | grep BHAALFAPP2

These gives:


b) Fencing can be tested using the following:

fence_vmware_soap –ip –username fence –password Pa55word -z –action off –uuid 420f1c6e-0109-a456-4e87-b827a52f8e9e

Recent Tasks in vCenter should show fence login shut down the system.

2, Adding Fencing in LUCI:

a) Log into LUCI and select your cluster and then select the Fence Devices tab. Select Add.

Fill out the form exactly as shown:

Fence type: VMWare (SOAP Interface)
Name: bhavc01 (Just needs to be descriptive)
IP Address or Hostname:
IP Port (optional): blank
Login: fence
password: Pa55word

Leave the rest as they are or blank


b) Go back to Manage Clusters in LUCI and select bhaalfapp1 by clicking on the name. At the bottom of the screen select Add Fence Method. Give it a name, bhavc01 in this case. Submit change. Now select Add Fence Instance that appears inside the method box and fill it out exactly as described below:

Select the fencing device configured in step a

VM Name: Leave blank
VM UUID: 420f13e6-4f98-ee4d-5462-03aad4cfb5c2 (The UUID copied from earlier)
Use SSL: Check it on. Fencing will not work without this checked.


Do the above for bhaalfapp2 

How To Increase GFS and LVM Size
LVM Configuration

  1. Initialize LUNs with pvcreate.e.g. pvcreate /dev/sdd
  2. Add a device into the Volume group with vgextende.g. vgextend alfdata-gfs /dev/sdd
  3. Sync the LVM Metadataclvmd -R
  4. Extend the size of the Logical Volume with lvextende.g. lvextend –l 1036286 /dev/alfdata-gfs/lvol0
  5. Sync the LVM Metadata clvmd –R

GFS Configuration

  1. Grow the existing GFS file system into the additional space gfs_growe.g. gfs2_grow –v /dev/mapper/alfdata—gfs-lvol10


Tags: , ,

No comments yet.

Leave a Reply