Tag Archives: GNU/Linux

Configuring a SPAN or mirror port on Open vSwitch

This howto describes how to configure a mirror port on your Open vSwitch. The goal is to install a new guest to act as IDS/IPS system. This guest is configured with 2 virtual network interfaces. The first interface will have an IP address and will be used to manage the guest. The other interfaces will be connected to the mirror port on Open vSwitch. This means that it will see all mirrored traffic.

My setup

Host OS : Ubuntu Quantal Quetzal 12.04 with libvirtd
Networking : The virtual machines are all connected to a OpenvSwitch bridge and are using RFC 1918 ip addresses. Since I only have a single external IP, my host runs a firewall that NATs certain ports towards the virtual machines.

XML configuration of guest

This is a copy of the interface declarations in the config file.

<interface type='bridge'>
 <mac address='52:54:bb:bb:11:11'/>
 <source bridge='ovsbr0'/>
 <virtualport type='openvswitch'>
 </virtualport>
 <model type='virtio'/>
</interface>
<interface type='bridge'>
 <mac address='52:54:bb:bb:11:12'/>
 <source bridge='ovsbr0'/>
 <virtualport type='openvswitch'>
 </virtualport>
 <model type='virtio'/>
</interface>

Since the MAC address has to be unique, we can use that as an identifier to configure the mirror port on Open vSwitch.

Configuring the mirror port on Open vSwitch

This script should be named ‘qemu’ and should be placed in ‘/var/libvirt/hooks’.

#!/bin/bash
# Written by Thomas Elsen
# You can use this at your own risk.
#
# The following to variables should be set before using the script.
# MAC containts the mac address from the interface that will receive
# all mirrored traffic.
MAC="52:54:bb:bb:11:12"
#GUEST should point to the name of the guest
GUEST="ids"

if [ $1 = $GUEST ];
then
        if [ $2 = 'started' ];
        then
                IFACE=`ifconfig | grep $MAC | awk '{print $1;}'`
                ovs-vsctl clear bridge ovsbr0 mirrors
                ovs-vsctl -- --id=@m create mirror name=mirror0 -- add bridge ovsbr0 mirrors @m -- --id=@capt get Port $IFACE -- set mirror mirror0 output_port=@capt select_all=1
                exit 0
        fi
fi

echo "Nothing to do : $1 $2" | logger
exit 0

After installing the script, make sure to set the 2 variables to the right values and give it the right permissions.

# chmod 755 /etc/libvirt/hooks/qemu

Using the above script will make sure that the mirror port is created when the guest is started. To make sure that libvirtd will use this new script, we have to restart it.

# /etc/init.d/libvirt-bin restart

Next step

In the next article I’ll use this new guest to run snort. Snort is an Open Source IDS sensor.

Creating a KVM virtual machine using CLI

This tutorial explains how to create a new KVM virtual machine on Ubuntu Linux using only the CLI (command line).

My setup

Host OS : Ubuntu Quantal Quetzal 12.04 with libvirtd
Networking : The KVM virtual machines are all connected to a OpenvSwitch bridge and are using RFC 1918 ip addresses. Since I only have a single external IP, my host runs a firewall that NATs certain ports towards the virtual machines.

Procedure

Create the disk

You can create a sparse file with the following command. The reserved space will be 12 GB in size. This method is often called ‘thin provisioning’.

# truncate -s 12G disk.img

Optional : Extract xml from existing KVM virtual machine

If you already have other machines running, shutdown a machine and extract its xml file.

virsh dumpxml <existing_machine> > newmachine.xml

You can also use this xml file as a start. As you can see, I set the boot device to the cdrom and pointed the cdrom to a bootable iso image. Note that in this example, I made use of an OpenvSwitch bridge for network connectivity.

Adapt the xml file

    • Change the uuid to a unique value.
# uuidgen
78dac724-9fdd-4202-a27e-52cdbf491ada
    • Change the MAC address to a unique value.
# echo 52:54:$(dd if=/dev/urandom count=1 2>/dev/null | md5sum | sed 's/^\(..\)\(..\)\(..\)\(..\).*$/\1:\2:\3:\4/')
52:54:f6:65:52:39

Create the new KVM virtual machine guest

Now we will import the xml file into our system. After importing it, you can remove the file.

# virsh create image.xml
Domain lucid created from image.xml

Please verify if you got the same output. Errors will be printed at this point. Please do not proceed until you ran this command successfully.
After running this command, the new KVM virtual machine guest will be running.

Connect to the interface

At this point, a vnc server should be running on a port on our host system. That vnc server can be used to manage/install the guest. Since the port is dynamically allocated, we have to use the following command to get the port.

# virsh dumpxml image | grep vnc
    <graphics type='vnc' port='5907' autoport='yes' listen='127.0.0.1'>

In this case, the dynamic port is 5907. We can connect from the local machine to that port using vncviewer.

# vncviewer localhost 5907

Post installation

After installing the machine, don’t forget configuration to make sure the system boots from the hard disc.

# virsh edit image

For ‘boot dev’, change ‘cdrom’ into ‘hd’.

Optional : Autostart the new KVM virtual machine

If you want to start the guest whenever the hosts system boots, issue this command.

# virsh autostart image

IPv6 : ping: sendmsg: Network is down

SETUP

I have 2 linux machines which are directly connected by means of an OpenVPN tunnel using tap interfaces. This means that 1 interface of each machine sits in the same virtual layer 2 network.

Interface of machine A:

root@machineA ~ # ip -6 ad show dev ovsbr1
6: ovsbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 
    inet6 2001:470:7071::1/64 scope global 
       valid_lft forever preferred_lft forever

Interface of machine B:

root@machineB:~# ip -6 ad show tap0
10: tap0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qlen 100
    inet6 2001:470:7071::2/128 scope global 
       valid_lft forever preferred_lft forever

Both machines can ping each other.

root@machineA ~ # ping6 -c 1 2001:470:7071::2
PING 2001:470:7071::2(2001:470:7071::2) 56 data bytes
64 bytes from 2001:470:7071::2: icmp_seq=1 ttl=64 time=34.9 ms

--- 2001:470:7071::2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 34.949/34.949/34.949/0.000 ms

And the other way around:

root@machineB:~# ping6 -c 1 2001:470:7071::1
PING 2001:470:7071::1 (2001:470:7071::1): 56 data bytes
64 bytes from 2001:470:7071::1: seq=0 ttl=255 time=34.233 ms

--- 2001:470:7071::1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 34.233/34.233/34.233 ms

Also, just to prove that both machines are really pinging each other, I’ll print the mac addresses and neighbour caches.

root@machineA ~ # ip link show dev ovsbr1
6: ovsbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT 
    link/ether 62:b0:18:37:1d:47 brd ff:ff:ff:ff:ff:ff
root@machineA ~ # ip -6 neigh
2001:470:7071::2 dev ovsbr1 lladdr 26:d5:0d:eb:93:72 router REACHABLE
root@machineA ~ #

And same info for machine B:

root@machineB:~# ip link show dev tap0
10: tap0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN mode DEFAULT qlen 100
    link/ether 26:d5:0d:eb:93:72 brd ff:ff:ff:ff:ff:ff
root@machineB:~# ip -6 neigh
2001:470:7071::1 dev tap0 lladdr 62:b0:18:37:1d:47 router REACHABLE
root@machineB:~#

FYI : neighbor cache in IPv6 is similar to arp cache in IPv4.

MachineA has a route for 2001:470:7071:200::/56 pointing towards machineB

root@machineA ~ # ip -6 route  | grep 200::
2001:470:7071:200::/56 via 2001:470:7071::2 dev ovsbr1  metric 1 
root@machineA ~ # ip -6 route get 2001:470:7071:200::5
2001:470:7071:200::5 from :: via 2001:470:7071::2 dev ovsbr1  src 2001:470:7071::1  metric 0 
    cache 
root@machineA ~ #

Some extra info from machine A:
– Ubuntu Quantal. Linux kernel 3.5.0.
– The bridge is an OpenvSwitch bridge.

Problem

On machineA, I can’t connect or ping to hosts behind machineB. This is what happens.

root@machineA ~ # ping6 -c 1 2001:470:7071:200::5
PING 2001:470:7071:200::51(2001:470:7071:200::5) 56 data bytes
ping: sendmsg: Network is down
^C
--- 2001:470:7071:200::5 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

root@machineA ~ #

I also checked with tcpdump, but nothing is send over the interface when I try to ping.

Solution

None yet. If you have an idea, let me know.
[update on 2013/02/26] Started working after a reboot. Too bad I didn’t find the real reason.