Problems with booting Xen guests after going to version 3

If you are experiencing bootup problems with your Xen v3 guests, it may very be that you have not completely prepared your guests with the Xen v3 modules.

You see, with Xen v2, most people compiled their own kernels, both for the xen0/dom0 and xenU/domU. Most admins compiled in the ext3 support or necessary filesystem drivers directly into the XenU kernel, and were able to boot the Virtual Machines with no hiccups.
With Xen v3, the kernel is the same with both dom0 and domU, and everything is modular. Most admins even have to make sure to include a initrd to even boot their dom0. How do you do the same with the domU ??

Follow these two steps:

1. Add the following in your xmconfig file for that domU:
   

   ramdisk="/boot/initrd-2.6.18-1.2869.fc6xen.img"

   
   
2. Tar up the Xen modules, mount your domU image on a loopback and copy the modules to the domU image.
   

   cd /lib/modules
   tar -czvf ./2.6.18-1.2869.fc6xen-modules.tgz ./2.6.18-1.2869.fc6xen/
   mkdir /mnt/guest-os
   mount -o loop /someclient_image /mnt/guest-os
   cd /mnt/guest-os
   tar -xzvf /lib/modules/2.6.18-1.2869.fc6xen.tgz
   cd /
   umount /mnt/guest-os

   
   

That should be all, you should now be able to start your domU.
Note: This article takes for granted that your actual kernel name is vmlinux-2.6.18-1.2869.fc6xen :-) YMMV

Compiling XNU (OSX Kernel)

We are looking into getting OCF into the OSX kernel. That means we have to first be able to compile the XNU/Darwin kernel. I followed this howto.

Note a few points:

• Make sure to install GCC 3.3 on a OSX machine.


• Get the building utilities pre-built here.


• Get the Xnu kernel tarball, either PPC or Intel.

So, all of this is rather straightforward, BUT I had install both GCC 4.0 and GCC3.3, and I think that XNU is allergic to GCC4.0

I ended up relinking all the /usr/bin/gcc, /usr/bin/c++, and such to their gcc 3.3 equivalent. Once done, the compile went rather well, until the very end and then ended up complaining about -lcc_kext not found. The trick is to modify makedefs/MakeInc.def and to change:

# 
# Default runtime libraries to be linked with the kernel
#
export LD_KERNEL_LIBS   = -lcc_kext

with

# 
# Default runtime libraries to be linked with the kernel
#
export LD_KERNEL_LIBS   = -L/usr/lib/gcc/darwin/3.3/ -lcc_kext

That's it.

Here are a few links that might interest you for understanding of building of the XNU kernel:
MacOSX Forge
OpenDarwin
O'Reilly
Building Intel xnu with darwinbuild