this is obsolete doc -- see http://doc.nethence.com/ instead
XEN4 RHEL6 dom0 (pv_ops)
Requirements
Make sure the development tools package group and those additional packages are installed,
yum groupinstall "Development tools"
yum install \
openssl-devel \
uuid-devel \
libX11-devel \
ncurses-devel \
dev86 \
iasl \
libuuid-devel \
python-devel \
texinfo
Note. among the development tools, you'll have gcc, make, zlib-devel
XEN compilation and installation
Fetch (http://xen.org/products/xen_source.html) and build the xen microkernel, xen tools and xen stubdom (hvm ioemu),
cd ~/
#wget http://bits.xensource.com/oss-xen/release/4.1.1/xen-4.1.1.tar.gz
#tar xzf xen-4.1.1.tar.gz
#cd xen-4.1.1/
wget http://bits.xensource.com/oss-xen/release/4.1.2/xen-4.1.2.tar.gz
tar xzf xen-4.1.2.tar.gz
cd xen-4.1.2/
make xen && make tools && make stubdom
Put the XEN micro-kernel in place and install the userland tools,
#cp xen/xen.gz /boot/xen-4.1.1.gz
cp xen/xen.gz /boot/xen-4.1.2.gz
make install-tools
make install-stubdom
Dom0 kernel compilation and installation
Checkout Jeremy's repo (https://github.com/mirrors/linux-2.6 while kernel.org is down),
cd ~/
#git clone git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
git clone git://github.com/jsgf/linux-xen.git
cd linux-xen/
git branch
git pull
Note. we're on the next-2.6.32 branch by default.
Configure your kernel,
make mrproper
make defconfig
cat >> .config <<EOF9
CONFIG_EXT4_FS=y
CONFIG_EXT4_FS_POSIX_ACL=y
CONFIG_BRIDGE=y
CONFIG_NFSD=y
CONFIG_NFSD_V3=y
CONFIG_NFSD_V3_ACL=y
EOF9
make silentoldconfig
# (press enter for the defaults)
cat >> .config <<EOF9
CONFIG_DUMMY=m
CONFIG_TUN=m
CONFIG_BONDING=m
EOF9
make silentoldconfig
cat >> .config <<EOF9
CONFIG_PARAVIRT_GUEST=y
CONFIG_XEN=y
CONFIG_XEN_DOM0=y
CONFIG_XEN_NETDEV_BACKEND=y
CONFIG_XEN_BLKDEV_BACKEND=y
CONFIG_XEN_BLKDEV_TAP=y
CONFIG_XEN_GNTDEV=y
CONFIG_XEN_PLATFORM_PCI=y
EOF9
make silentoldconfig
# (press enter for the defaults)
Add the needed drivers for your hardware. For example for a Dell XS11-VX8 mini-server,
cat >> .config <<EOF9
CONFIG_PATA_VIA=y
CONFIG_I2C_VIAPRO=y
CONFIG_FB_VIA=y
CONFIG_E1000E=y
EOF9
make silentoldconfig
Note. disk, bus, video and network drivers.
Compile your kernel,
time nice make
ls -l /lib/modules/
make modules_install
cp arch/x86_64/boot/bzImage /boot/vmlinuz-2.6.32.46xen
mkdir -p /data/share/
cp arch/x86_64/boot/bzImage /data/share/vmlinuz-2.6.32.46xen
Note. the kernel you just compiled is both, dom0 and domU capable.
Rebuild the initramfs,
cd /boot/
dracut initramfs-2.6.32.46xen.img 2.6.32.46
Check the filenames,
#ls -l xen-4.1.1.gz
ls -l xen-4.1.2.gz
ls -l vmlinuz-2.6.32.46xen
ls -l initramfs-2.6.32.46xen.img
make the new kernel bootable,
vi grub/grub.conf
change/add,
default=1
[...]
title XEN 4.1.2 with Linux 2.6.32.46xen
root (hd0,0)
kernel /xen-4.1.2.gz dom0_mem=512M
module /vmlinuz-2.6.32.46xen ro root=/dev/sda2 md=
module /initramfs-2.6.32.46xen.img
Note. assuming sda1 for /boot and sda2 for the root mount point.
Eventually fix some useful kernel source symlinks,
cd /usr/src/
ln -s ../../root/linux-xen
ln -s ../../root/linux-xen linux
Disable KSM and reboot,
cd ~/
chkconfig ksm off
chkconfig ksmtuned off
shutdown -r now
once rebooted, check that you're actually running the new kernel on top of xen,
uname -r
xm info
Dom0 configuration
Mount XENFS,
cat >> /etc/fstab <<EOF9
xenfs /proc/xen xenfs defaults 0 0
EOF9
mount /proc/xen
check,
mount | grep xen
Eventually edit your screen configuration,
cd ~/
cat >> .screenrc <<EOF9
screen -t "xen" 1 tail -F /var/log/xen/xend.log
screen -t "xendebug" 1 tail -F /var/log/xen/xend-debug.log
EOF9
XEN network configuration
Clean up the config file,
cd /etc/xen/
mv xend-config.sxp xend-config.sxp.dist
sed '/^#/d; /^$/d' xend-config.sxp.dist > xend-config.sxp
vi xend-config.sxp
change/check,
(network-script network-bridge)
Note. we're keeping the default (bridge for dom0 and guests). For guests, you can always force alternate bridges in their configs
Note. if you're using a standalone xen dom0 without live migration, you may disable this to secure the 8002 port,
#(xend-relocation-server yes)
#(xend-relocation-hosts-allow '^localhost$ ^localhost\\.localdomain$')
Rebuild bridge-utils
On RHEL6 there's an issue with bridge-utils (IGMP snooping conflicts with the custom dom0 kernel). You need to rebuild it without it.
Prepare your environment,
cd ~/
yum install gcc make rpm-build
mkdir -p rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
ls -l .rpmmacros
echo '%_topdir %(echo $HOME)/rpmbuild' > .rpmmacros
groupadd mockbuild
useradd -g mockbuild mockbuild
Note. redhat-rpm-config isn't mandatory
Install the src RPM and tweak the spec file,
wget ftp://ftp.free.fr/mirrors/ftp.centos.org/6.0/os/SRPMS/Packages/bridge-utils-1.2-9.el6.src.rpm
rpm -ivh bridge-utils-1.2-9.el6.src.rpm
cd ~/rpmbuild/SPECS/
cp bridge-utils.spec bridge-utils.spec.dist
vi bridge-utils.spec
remove those two lines from it,
Patch4: bridge-utils-1.2-Add_IGMP_snooping_support.patch
%patch4 -p1 -b .Add_IGMP_snooping_support
you can now build it with IGMP snooping disabled,
yum install libsysfs-devel
rpmbuild -ba bridge-utils.spec
Install the thing,
cd ~/rpmbuild/RPMS/x86_64/
rpm -e bridge-utils
rpm -ivh bridge-utils-1.2-9.el6.x86_64.rpm
Refs.
http://wiki.centos.org/HowTos/RebuildSRPM
http://wiki.centos.org/HowTos/SetupRpmBuildEnvironment
Ready to go
Start the xen daemons (xencommons now needs to be started before xend),
chkconfig xencommons on
chkconfig xend on
service xencommons start
service xend start
Troubleshooting
If you get this error when in the xen logs and when doing 'brctl show',
/sys/class/net/virbr0/bridge: No such file or directory
==> make sure you rebuilt bridge-utils w/o IGMP snooping.
If you get this error in the xen logs,
xend start "cat: /sys/bus/scsi/devices/host0/vendor: No such file or directory"
==> make sure you started xencommons BEFORE xend.
Update
Eventually uninstall the previous version before you start from scratch again,
cd ~/
cd xen-4.1.1/
make uninstall
cd ../
rm -rf xen-4.1.1/
References
http://wiki.xen.org/xenwiki/RHEL6Xen4Tutorial
http://wiki.xen.org/xenwiki/XenParavirtOps
http://wiki.xen.org/xenwiki/XenSerialConsole
http://wiki.xen.org/xenwiki/MigrationGuideToXen4.1%2B
http://wiki.xen.org/xenwiki/XenCommonProblems
http://wiki.gcu-squad.org/doku.php?id=linux:dediboxv3_debian6_xen4