this is obsolete doc -- see instead

Veritas Storage Foundation Cluster File System v4.1 administration 




Happy happy 

Check everything's fine, 

ps ax | grep vx
hastatus -summary
cfscluster status
vxdctl -c mode

Note. all systems should be RUNNING 

Note. all systems of the cvm group should be ONLINE 

Note. other possible commands, 

#hares -display | grep -i state
#hasys -display | grep -i sysstate



Initialize Disk Group 

Quoting /etc/vx/bin/vxdisksetup, 

# Determine if the device is a SCSI device or not; only SCSI devices are
# supported as CDS disks. Determine the starting disk address and size
# of the disk (for use as the WHOLEDISK slice on Solaris labeled disks
# or the extended partition on non-Solaris disks), including any CDS
# disk restrictions for non-Solaris disks.

Note. you need e.g. SAN or vmware shared raw disk. Software iSCSI may not work. 


Check you've got a shared disk available among the nodes, 

#find /sys/devices/platform/host* -name "block*"
fdisk -l


Check the shared disk is 'online' on all the nodes, 

vxdisk scandisks
vxdctl enable
vxdisk -o alldgs list
#vxdisk -eo alldgs list


On only one node, check disk geometry and make sure ncyl isn't equal to zero (which may happen with software iscsi), 

/usr/lib/vxvm/diag.d/vxscsi -g sdc
#hdparm  -i /dev/sdb | grep SCSI


On the master node, initialize a disk, 

vxdctl -c mode
vxdisksetup -i sdc

Note. format=cdsdisk is the default 

Note. '-f' forces a disk to be initialized even if it is a member of a deported disk group 


On the master node, create a disk group with it, 

vxdg -s init group0 sdc
#vxdg -g group0 set activation=sharedwrite
vxdg list
vxdisk -o alldgs list
#vxdisk -eo alldgs list
#vxdisk -g group0 list
#vxdisk -eg group0 list

Note. alternatively, 

#vxdg init group0 group0_disk1=sdb

Note. to add disks to the disk group, 

#vxdg -g ABCdg adddisk ABCdg02=c0t0d2
#vxdg -g ABCdg adddisk ABCdg09=c0t0d9

Note. we could also use vxdiskadd interactively for that, but heh, the hard way is always better, 

#vxdiskadd sdc

Note. vxdg list should return e.g. this for group0, 

group0       enabled,shared,cds

Note. vxdisk list should return e.g. this for group0, 

sdc          auto:cdsdisk    sdc          group0       online shared

Note. to destroy a DG, 

#vxdg destroy group0



Create a volume 

On only one node, create a new volume with that disk (50GB here), 

vxassist maxsize
vxassist make vol0 104820736

Note. other example, 

#vxassist make mirvol 500m layout=mirror,log mirror=ctlr \!ctlr:c1

Note. to remove a volume, 

#vxassist remove volume vol0


You should now be able to access the volume from any node in the cluster. 




On the master node, 

ls -l /dev/vx/dsk/group0/vol0
mkfs.vxfs /dev/vx/dsk/group0/vol0
mkdir -p /mnt/vxfs
mount.vxfs -o log,rw,cluster,crw /dev/vx/dsk/group0/vol0 /mnt/vxfs

Note. -o largefiles (>1TB file capable) is the default, check with, 

fsadm /dev/vx/dsk/group0/vol0


On the other nodes, 

ls -l /dev/vx/dsk/group0/vol0
mkdir -p /mnt/vxfs
mount.vxfs -o log,rw,cluster,crw /dev/vx/dsk/group0/vol0 /mnt/vxfs






Start/stop a volume, 

vxvol start vol0
vxvol stop vol0


Start/stop all volumes, 

vxvol startall
vxvol stopall


Make sure the disk group is imported, 

vxdg import group0


Make sure all volumes on the disk group are started, 

vxvol -g group0 startall


To remove a volume and recursively (-r) its plexes and subdisks, 

umount ...
vxvol stop vol0
vxedit -r rm vol0


To remove a disk group, 

vxdg list
vxdg destroy group0




Note. to uninstall SFCFS, 

cd /opt/VRTS/install




Watch the logs, 

cd /var/VRTSvcs/log
ls -ltr
tail -n 500 -F engine_A.log


VxVM vxdisksetup ERROR V-5-2-3461 sdb: Disk does not have enough cylinders for 'cdsdisk' format 

The 'vxdisksetup' scripts calls vxscsi, which returns ncyl=0 (cylinders), check yourself, 

/usr/lib/vxvm/diag.d/vxscsi -g sdc

You're proably using software iSCSI which doesn't seem to be enought. Try with a real SAN or with VMware shared raw disks. 


VxVM vxdg ERROR V-5-1-585 Disk group group0: cannot create: Cannot find disk on slave node. 

Here's the workaround. On the failing node (vxdisk list show an error on the disk), 

vxdisk offline sdc
blockdev --rereadpt /dev/vx/dmp/sdc
vxdisk online sdc
vxdisk scandisks

check again (should be online), 

vxdisk list



ERROR: V-3-24996 

Disassociate the volume, 

vxvol -g dg -f dis vol

Make the volume active by starting it, 

vxvol -g dg start vol

Run a consistency check on the existing filesystem, 

fsck -F vxfs /dev/vx/rdsk/dg/vol

Reassociate the volume to the root volume group, 

vxvol -g dg assoc rvg vol

Mount the filesystem within the Operating System, 

mount -F vxfs /dev/vx/dsk/dg/vol /mntpt 






Note. mount option for applications that O_SYNC but only require write data sync, 

-o convosync=dsync









Usage references