Nethence Newdoc Olddoc Lab Your IP BBDock  


Warning: those guides are mostly obsolete, please have a look at the new documentation.

UnixWindowsOracleObsoleteHardwareDIYMechanicsScriptsConfigs

Veritas Storage Foundation Cluster File System v4.1 administration
 
 
Monitoring
Happy happy
Check everything's fine,
ps ax | grep vx
hastatus -summary
cfscluster status
vxdctl -c mode
Note. all systems should be RUNNING
Note. all systems of the cvm group should be ONLINE
Note. other possible commands,
#vxparms
#hares -display | grep -i state
#hasys -display | grep -i sysstate
 
 
Initialize Disk Group
Quoting /etc/vx/bin/vxdisksetup,
# Determine if the device is a SCSI device or not; only SCSI devices are
# supported as CDS disks. Determine the starting disk address and size
# of the disk (for use as the WHOLEDISK slice on Solaris labeled disks
# or the extended partition on non-Solaris disks), including any CDS
# disk restrictions for non-Solaris disks.
Note. you need e.g. SAN or vmware shared raw disk. Software iSCSI may not work.
 
Check you've got a shared disk available among the nodes,
#find /sys/devices/platform/host* -name "block*"
fdisk -l
 
Check the shared disk is 'online' on all the nodes,
vxdisk scandisks
vxdctl enable
vxdisk -o alldgs list
#vxdisk -eo alldgs list
 
On only one node, check disk geometry and make sure ncyl isn't equal to zero (which may happen with software iscsi),
/usr/lib/vxvm/diag.d/vxscsi -g sdc
#hdparm -i /dev/sdb | grep SCSI
 
On the master node, initialize a disk,
vxdctl -c mode
vxdisksetup -i sdc
Note. format=cdsdisk is the default
Note. '-f' forces a disk to be initialized even if it is a member of a deported disk group
 
On the master node, create a disk group with it,
vxdg -s init group0 sdc
#vxdg -g group0 set activation=sharedwrite
vxdg list
vxdisk -o alldgs list
#vxdisk -eo alldgs list
#vxdisk -g group0 list
#vxdisk -eg group0 list
Note. alternatively,
#vxdg init group0 group0_disk1=sdb
Note. to add disks to the disk group,
#vxdg -g ABCdg adddisk ABCdg02=c0t0d2
#vxdg -g ABCdg adddisk ABCdg09=c0t0d9
Note. we could also use vxdiskadd interactively for that, but heh, the hard way is always better,
#vxdiskadd sdc
Note. vxdg list should return e.g. this for group0,
group0 enabled,shared,cds 1279463494.34.vrts1.example.net
Note. vxdisk list should return e.g. this for group0,
sdc auto:cdsdisk sdc group0 online shared
Note. to destroy a DG,
#vxdg destroy group0
 
 
Create a volume
On only one node, create a new volume with that disk (50GB here),
vxassist maxsize
vxassist make vol0 104820736
Note. other example,
#vxassist make mirvol 500m layout=mirror,log mirror=ctlr \!ctlr:c1
Note. to remove a volume,
#vxassist remove volume vol0
 
You should now be able to access the volume from any node in the cluster.
 
 
Filesystem
On the master node,
ls -l /dev/vx/dsk/group0/vol0
mkfs.vxfs /dev/vx/dsk/group0/vol0
mkdir -p /mnt/vxfs
mount.vxfs -o log,rw,cluster,crw /dev/vx/dsk/group0/vol0 /mnt/vxfs
Note. -o largefiles (>1TB file capable) is the default, check with,
fsadm /dev/vx/dsk/group0/vol0
 
On the other nodes,
ls -l /dev/vx/dsk/group0/vol0
mkdir -p /mnt/vxfs
mount.vxfs -o log,rw,cluster,crw /dev/vx/dsk/group0/vol0 /mnt/vxfs
 
Ref. http://seer.entsupport.symantec.com/docs/300537.htm
 
 
Usage
Start/stop a volume,
vxvol start vol0
vxvol stop vol0
 
Start/stop all volumes,
vxvol startall
vxvol stopall
 
Make sure the disk group is imported,
vxdg import group0
 
Make sure all volumes on the disk group are started,
vxvol -g group0 startall
 
To remove a volume and recursively (-r) its plexes and subdisks,
umount ...
vxvol stop vol0
vxedit -r rm vol0
 
To remove a disk group,
vxdg list
vxdg destroy group0
 
 
Misc
Note. to uninstall SFCFS,
cd /opt/VRTS/install
./uninstallinfr
 
 
Troubbleshooting
Watch the logs,
cd /var/VRTSvcs/log
ls -ltr
tail -n 500 -F engine_A.log
 
VxVM vxdisksetup ERROR V-5-2-3461 sdb: Disk does not have enough cylinders for 'cdsdisk' format
The 'vxdisksetup' scripts calls vxscsi, which returns ncyl=0 (cylinders), check yourself,
/usr/lib/vxvm/diag.d/vxscsi -g sdc
You're proably using software iSCSI which doesn't seem to be enought. Try with a real SAN or with VMware shared raw disks.
 
VxVM vxdg ERROR V-5-1-585 Disk group group0: cannot create: Cannot find disk on slave node.
Here's the workaround. On the failing node (vxdisk list show an error on the disk),
vxdisk offline sdc
blockdev --rereadpt /dev/vx/dmp/sdc
vxdisk online sdc
vxdisk scandisks
check again (should be online),
vxdisk list
Ref. http://seer.entsupport.symantec.com/docs/282930.htm
 
ERROR: V-3-24996
Disassociate the volume,
vxvol -g dg -f dis vol
Make the volume active by starting it,
vxvol -g dg start vol
Run a consistency check on the existing filesystem,
fsck -F vxfs /dev/vx/rdsk/dg/vol
Reassociate the volume to the root volume group,
vxvol -g dg assoc rvg vol
Mount the filesystem within the Operating System,
mount -F vxfs /dev/vx/dsk/dg/vol /mntpt
 
Ref. http://seer.entsupport.symantec.com/docs/311556.htm
 
 
Tuning
Note. mount option for applications that O_SYNC but only require write data sync,
-o convosync=dsync
 
Refs.
http://www.mail-archive.com/veritas-vx@mailman.eng.auburn.edu/msg02378.html
 
 
References
http://sfdoccentral.symantec.com/
http://sfdoccentral.symantec.com/Storage_Foundation_HA_41_Linux.html
 
Cheatsheets
http://www.cuddletech.com/veritas/vxcheatsheet.txt
http://www.blight.com/~rick/veritas/cheat.html
 
Usage references
http://uw714doc.sco.com/en/ODM_VMadmin/sag-6.html
http://www.darklab.net/resources/veritas-cluster-server-debug.html
http://cortiz.org/2008/08/13/veritas-storage-creating-dg-volumes-and-filesystem/
http://www.camelrichard.org/topics/VeritasVM/Volume_Manager_CLI_Examples
http://adminschoice.com/veritas-volume-manager-vxassist
http://www.cuddletech.com/veritas/vxassist/x41.html
 

(obsolete, see the new doc)