Tag Archives: GNU/Linux

Switch your KVM from regular bridge to Open vSwitch

This howto focusses on the steps that you have to take to switch from the regular bridge to Open vSwitch.

 My SETUP

My server runs Debian 6.0.6 Squeeze. It has only 1 external IP address. That IP address is bound to eth0.

It currenlty has a br0 configured with an internal (RFC1918) ip address. All virtual machines are connected to that bridge. I’m using iptables and natting to make sure that all traffic on port 80 is forwarded to my webserver, port 25 to the mailserver etc.

Bridge configuration in /etc/network/interfaces

auto br0
iface br0 inet static
  address 172.16.11.1
  network 172.16.11.0
  netmask 255.255.255.0 
  broadcast 172.16.11.255
  bridge_stp on
  bridgefd 0 
  bridge_maxwait 0 
  pre-up brctl addbr br0
  post-down brctl delbr br0
  post-up /etc/network/firewallscript.sh

Building the packages

Since Debian doesn’t provide any packages for Open-vSwitch on Squeeze, I decided to create that package myself.

I’m going to use the latest version since I’m interested in the latest features as well. Let’s start by cloning the source tree.

 git clone git://openvswitch.org/openvswitch

Building the package itself. I’ve omitted the output.

cd openvswitch
./boot.sh
./configure
dpkg-buildpackage -b

This resulted in the following list of packages being created.

rivy@shell:~/src/openvswitch/src/openvswitch$ ls -l ../
total 13736
drwxr-xr-x 19 rivy rivy 4096 Nov 20 15:07 openvswitch
-rw-r--r-- 1 rivy rivy 272814 Nov 20 15:07 openvswitch-brcompat_1.9.90-1_amd64.deb
-rw-r--r-- 1 rivy rivy 664470 Nov 20 15:07 openvswitch-common_1.9.90-1_amd64.deb
-rw-r--r-- 1 rivy rivy 297856 Nov 20 15:07 openvswitch-controller_1.9.90-1_amd64.deb
-rw-r--r-- 1 rivy rivy 2328744 Nov 20 15:07 openvswitch-datapath-dkms_1.9.90-1_all.deb
-rw-r--r-- 1 rivy rivy 2395538 Nov 20 15:07 openvswitch-datapath-source_1.9.90-1_all.deb
-rw-r--r-- 1 rivy rivy 6181626 Nov 20 15:07 openvswitch-dbg_1.9.90-1_amd64.deb
-rw-r--r-- 1 rivy rivy 32656 Nov 20 15:07 openvswitch-ipsec_1.9.90-1_amd64.deb
-rw-r--r-- 1 rivy rivy 26310 Nov 20 15:07 openvswitch-pki_1.9.90-1_all.deb
-rw-r--r-- 1 rivy rivy 1610624 Nov 20 15:07 openvswitch-switch_1.9.90-1_amd64.deb
-rw-r--r-- 1 rivy rivy 45264 Nov 20 15:07 openvswitch-test_1.9.90-1_all.deb
-rw-r--r-- 1 rivy rivy 4902 Nov 20 15:07 openvswitch_1.9.90-1_amd64.changes
-rw-r--r-- 1 rivy rivy 48778 Nov 20 15:07 ovsdbmonitor_1.9.90-1_all.deb
-rw-r--r-- 1 rivy rivy 84260 Nov 20 15:07 python-openvswitch_1.9.90-1_all.deb
rivy@shell:~/src/openvswitch/src/openvswitch$

In case you don’t want to build the package. openvswitch_1.9.90-1_amd64.tgz.

Installing and configuring

Now install those packages on the server that has KVM running.

dpkg -i openvswitch-*

Make sure all dependencies are met. If not, run ‘apt-get -f install’ and rerun the installation.

Now, make sure we provide compatibility with the Linux bridge. Uncomment and change ‘no’ to ‘yes’ in /etc/default/openvswitch-switch

BRCOMPAT=yes

Make sure the module is built and will be loadable at boot time.

module-assistant auto-install openvswitch-datapath

Adapt the bridge settings in /etc/network/interfaces

#auto br0
iface br0 inet static
  address 172.16.11.1
  network 172.16.11.0
  netmask 255.255.255.0 
  broadcast 172.16.11.255
  bridge_stp on
  bridgefd 0 
  bridge_maxwait 0 
#  pre-up ovs-vsctl add-br br0 ( not needed as the switch config is kept in DB)
#  post-down ovs-vsctl del-br br0
  post-up /etc/network/firewallscript.sh

Replace the existing ifup scripts with scripts that make use of the new Open vSwitch. Therefor we keep a copy of the old files and create 2 new files.

cd /etc/kvm
mv kvm-ifdown kvm-ifdown-original
mv kvm-ifup kvm-ifup-original

Contents of new  /etc/kvm/kvm-ifup

#!/bin/sh
switch='br0'
/sbin/ifconfig $1 0.0.0.0 up
ovs-vsctl --if-exists del-port ${switch} $1
ovs-vsctl add-port ${switch} $1

Contents of new /etc/kvm/kvm-ifdown

#!/bin/sh
switch='br0'
/sbin/ifconfig $1 0.0.0.0 up
ovs-vsctl del-port ${switch} $1

Make sure that the new module is loaded before ‘bridge’. I did this by adding ‘openvswitch’ to ‘/etc/modules’.

echo 'openvswitch' >> /etc/modules

Next step shutdown all guests and remove the old bridge. When the bridge is removed, you can unload the old ‘bridge’ module and load the new ‘openvswitch’ module.

virsh list
virsh shutdown <all your guests>
ifconfig br0 down
rmmod bridge
modprobe openvswitch
service openvswitch-controller start
service openvswitch-switch start
service networking restart
service libvirt-bin restart

If all goes well, you guests should be using the new Open vSwitch.

To check if this is correct, issue the following command. You should have a vnetX interface for each guest.

# ovs-vsctl show
c9900989-6f68-4da6-b739-23b106efcab5
 Bridge "br0"
   Port "vnet6"
     Interface "vnet6"
   Port "vnet5"
     Interface "vnet5"
   Port "vnet4"
     Interface "vnet4"
   Port "vnet3"
     Interface "vnet3"
   Port "vnet0"
     Interface "vnet0"
   Port "vnet1"
     Interface "vnet1"
   Port "br0"
     Interface "br0"
       type: internal
   Port "vnet2"
     Interface "vnet2"
 ovs_version: "1.9.90"