I'm setting up an internet facing server which would have a limited number of services exposed. I'm also testing out the capabilities of Xen. Because the hardware I'm testing on is a 3 year old Dell 1U server with no hardware
RAID0, I can't use
VMWare ESXi (the free one) which would be a simple click and run.
So Xen it is. Installed CentOS 5.2, with software
RAID0 (thank you mdadm!), and ofcourse with the Xen kernel enabled. Installing a new CentOS image was relatively straightforward;
- Use virt-install
- If you are having problems in setting the ISO image as the source, you can
- mount -o loop Centos.DVD.iso /var/www/html/centos/
- use http://[localLANIP]/centos as the repository
So back to the issue at hand. The Physical server has two network cards. One is allocated as a trusted LAN card (
eth0), and the other card is the untrusted WAN card (
eth1). Normally when you install your server, you have access to both ports.
However if you want the
Dom0, the Xen physical server which hosts the child Virtual Machines (
DomU) to be insulated from the internet, you need eth1 to be invisible to
Dom0.
To do this, you will need to mess around abit. Parts of this guide comes from
here and
here. And you will need a kernel module called
pciback. To test that you have
pciback in your system, do this:
# modprobe pciback
# lsmod | grep pci
pciback 29389 0
Wonderful, it comes by default with the stock CentOS 5.2 distro. Next you will have to find out which PCI address to look out for.
# lspci |grep Ethernet
00:03.0 Ethernet controller: 3Com Corporation 3c905C
00:10.0 Host bridge: Broadcom I/O Bridge with Gigabit Ethernet
00:10.2 Host bridge: Broadcom I/O Bridge with Gigabit Ethernet
01:03.0 Ethernet controller: Intel Corporation Ethernet Pro 100
The one I was interested in is the 3Com card. So remember
00:03.0 . I tried the kernel comand arguments as described in the
URL above, but that didn't work. Fortunately in Linuxland, there is always more than one way of doing things.
First, modify
/etc/modprobe.conf manually
# cat modprobe.conf
alias eth0 e100
# 090217 yky Hiding eth1 from Dom0 to be revealed to DomU
#alias eth1 3c59x
options pciback hide=(0000:00:03.0)
Basically
pciback "seizes" any PCI devices before the kernel can get to it. Which is why it has to be preloaded early on. Unfortunately
pciback is a dynamic module residing in the filesystem, which may not be in the ramdisk when the kernel is loading. So you will need to add it to the ramdisk by running this command.
# mkinitrd -f --preload=pciback /boot/initrd-$(uname -r).img $(uname -r)
# ls -la /boot/initrd*
-rw------- 1 root root 2280203 Feb 17 17:42 initrd-2.6.18-92.el5xen.img
Now you are ready to lose
eth1 from
Dom0. Reboot. To confirm this to yourself, do this:
# dmesg | grep eth
e100: eth0: e100_probe: addr 0xfcf00000, irq 18, MAC addr 00:90:27:D3:A8:BC
e100: eth0: e100_watchdog: link up, 100Mbps, full-duplex
eth0: no IPv6 routers present
e100: peth0: e100_watchdog: link up, 100Mbps, full-duplex
device peth0 entered promiscuous mode
xenbr0: port 2(peth0) entering learning state
xenbr0: port 2(peth0) entering forwarding state
# dmesg | grep pciback
pciback 0000:00:03.0: seizing device
pciback: vpci: 0000:00:03.0: assign to virtual slot 0
# ls -l /sys/bus/pci/drivers/pciback/
lrwxrwxrwx 1 root root 0 Feb 17 17:52 0000:00:03.0 -> ../../../../devices/pci0000:00/0000:00:03.0
...
Previously
eth1 would appear where
eth0 did. Now not so.
eth1 has been
seized.
So the next step is to make
00:03.0 available to the
DomU guest VM. To do so, your Xen config file should look something like this:
# cat /etc/xen/DomU1
name = "DomU1"
memory = 178
bootloader = "/usr/bin/pygrub"
disk = [ "tap:aio:/xen/DomU1/DomU1.img,xvda,w" ]
vif = [ "mac=00:00:ee:24:9a:73,bridge=xenbr0" ]
pci = [ "00:03.0" ]
The
vif ... bridge=xenbr0 will provide a virtual NIC to the
DomU which can be connected to the trusted LAN.
The
pci = [ "00:03.0" ] defines the PCI address which is accessible to the
DomU to probe. On a successful bootup, it should detect it as a pure 3Com card as
Dom0 used to do.
You can now configure
eth1 as per normal. Dont forget to bring up them firewalls!
yk.