Nethence Newdoc Olddoc Lab Your IP BBDock  

Warning: those guides are mostly obsolete, please have a look at the new documentation.


Veritas Storage Foundation Cluster File System v4.1 administration
Happy happy
Check everything's fine,
ps ax | grep vx
hastatus -summary
cfscluster status
vxdctl -c mode
Note. all systems should be RUNNING
Note. all systems of the cvm group should be ONLINE
Note. other possible commands,
#hares -display | grep -i state
#hasys -display | grep -i sysstate
Initialize Disk Group
Quoting /etc/vx/bin/vxdisksetup,
# Determine if the device is a SCSI device or not; only SCSI devices are
# supported as CDS disks. Determine the starting disk address and size
# of the disk (for use as the WHOLEDISK slice on Solaris labeled disks
# or the extended partition on non-Solaris disks), including any CDS
# disk restrictions for non-Solaris disks.
Note. you need e.g. SAN or vmware shared raw disk. Software iSCSI may not work.
Check you've got a shared disk available among the nodes,
#find /sys/devices/platform/host* -name "block*"
fdisk -l
Check the shared disk is 'online' on all the nodes,
vxdisk scandisks
vxdctl enable
vxdisk -o alldgs list
#vxdisk -eo alldgs list
On only one node, check disk geometry and make sure ncyl isn't equal to zero (which may happen with software iscsi),
/usr/lib/vxvm/diag.d/vxscsi -g sdc
#hdparm -i /dev/sdb | grep SCSI
On the master node, initialize a disk,
vxdctl -c mode
vxdisksetup -i sdc
Note. format=cdsdisk is the default
Note. '-f' forces a disk to be initialized even if it is a member of a deported disk group
On the master node, create a disk group with it,
vxdg -s init group0 sdc
#vxdg -g group0 set activation=sharedwrite
vxdg list
vxdisk -o alldgs list
#vxdisk -eo alldgs list
#vxdisk -g group0 list
#vxdisk -eg group0 list
Note. alternatively,
#vxdg init group0 group0_disk1=sdb
Note. to add disks to the disk group,
#vxdg -g ABCdg adddisk ABCdg02=c0t0d2
#vxdg -g ABCdg adddisk ABCdg09=c0t0d9
Note. we could also use vxdiskadd interactively for that, but heh, the hard way is always better,
#vxdiskadd sdc
Note. vxdg list should return e.g. this for group0,
group0 enabled,shared,cds
Note. vxdisk list should return e.g. this for group0,
sdc auto:cdsdisk sdc group0 online shared
Note. to destroy a DG,
#vxdg destroy group0
Create a volume
On only one node, create a new volume with that disk (50GB here),
vxassist maxsize
vxassist make vol0 104820736
Note. other example,
#vxassist make mirvol 500m layout=mirror,log mirror=ctlr \!ctlr:c1
Note. to remove a volume,
#vxassist remove volume vol0
You should now be able to access the volume from any node in the cluster.
On the master node,
ls -l /dev/vx/dsk/group0/vol0
mkfs.vxfs /dev/vx/dsk/group0/vol0
mkdir -p /mnt/vxfs
mount.vxfs -o log,rw,cluster,crw /dev/vx/dsk/group0/vol0 /mnt/vxfs
Note. -o largefiles (>1TB file capable) is the default, check with,
fsadm /dev/vx/dsk/group0/vol0
On the other nodes,
ls -l /dev/vx/dsk/group0/vol0
mkdir -p /mnt/vxfs
mount.vxfs -o log,rw,cluster,crw /dev/vx/dsk/group0/vol0 /mnt/vxfs
Start/stop a volume,
vxvol start vol0
vxvol stop vol0
Start/stop all volumes,
vxvol startall
vxvol stopall
Make sure the disk group is imported,
vxdg import group0
Make sure all volumes on the disk group are started,
vxvol -g group0 startall
To remove a volume and recursively (-r) its plexes and subdisks,
umount ...
vxvol stop vol0
vxedit -r rm vol0
To remove a disk group,
vxdg list
vxdg destroy group0
Note. to uninstall SFCFS,
cd /opt/VRTS/install
Watch the logs,
cd /var/VRTSvcs/log
ls -ltr
tail -n 500 -F engine_A.log
VxVM vxdisksetup ERROR V-5-2-3461 sdb: Disk does not have enough cylinders for 'cdsdisk' format
The 'vxdisksetup' scripts calls vxscsi, which returns ncyl=0 (cylinders), check yourself,
/usr/lib/vxvm/diag.d/vxscsi -g sdc
You're proably using software iSCSI which doesn't seem to be enought. Try with a real SAN or with VMware shared raw disks.
VxVM vxdg ERROR V-5-1-585 Disk group group0: cannot create: Cannot find disk on slave node.
Here's the workaround. On the failing node (vxdisk list show an error on the disk),
vxdisk offline sdc
blockdev --rereadpt /dev/vx/dmp/sdc
vxdisk online sdc
vxdisk scandisks
check again (should be online),
vxdisk list
ERROR: V-3-24996
Disassociate the volume,
vxvol -g dg -f dis vol
Make the volume active by starting it,
vxvol -g dg start vol
Run a consistency check on the existing filesystem,
fsck -F vxfs /dev/vx/rdsk/dg/vol
Reassociate the volume to the root volume group,
vxvol -g dg assoc rvg vol
Mount the filesystem within the Operating System,
mount -F vxfs /dev/vx/dsk/dg/vol /mntpt
Note. mount option for applications that O_SYNC but only require write data sync,
-o convosync=dsync
Usage references

(obsolete, see the new doc)