SETUP
I have 2 linux machines which are directly connected by means of an OpenVPN tunnel using tap interfaces. This means that 1 interface of each machine sits in the same virtual layer 2 network.
Interface of machine A:
root@machineA ~ # ip -6 ad show dev ovsbr1
6: ovsbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
inet6 2001:470:7071::1/64 scope global
valid_lft forever preferred_lft forever
Interface of machine B:
root@machineB:~# ip -6 ad show tap0
10: tap0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qlen 100
inet6 2001:470:7071::2/128 scope global
valid_lft forever preferred_lft forever
Both machines can ping each other.
root@machineA ~ # ping6 -c 1 2001:470:7071::2 PING 2001:470:7071::2(2001:470:7071::2) 56 data bytes 64 bytes from 2001:470:7071::2: icmp_seq=1 ttl=64 time=34.9 ms --- 2001:470:7071::2 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 34.949/34.949/34.949/0.000 ms
And the other way around:
root@machineB:~# ping6 -c 1 2001:470:7071::1 PING 2001:470:7071::1 (2001:470:7071::1): 56 data bytes 64 bytes from 2001:470:7071::1: seq=0 ttl=255 time=34.233 ms --- 2001:470:7071::1 ping statistics --- 1 packets transmitted, 1 packets received, 0% packet loss round-trip min/avg/max = 34.233/34.233/34.233 ms
Also, just to prove that both machines are really pinging each other, I’ll print the mac addresses and neighbour caches.
root@machineA ~ # ip link show dev ovsbr1
6: ovsbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT
link/ether 62:b0:18:37:1d:47 brd ff:ff:ff:ff:ff:ff
root@machineA ~ # ip -6 neigh
2001:470:7071::2 dev ovsbr1 lladdr 26:d5:0d:eb:93:72 router REACHABLE
root@machineA ~ #
And same info for machine B:
root@machineB:~# ip link show dev tap0
10: tap0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN mode DEFAULT qlen 100
link/ether 26:d5:0d:eb:93:72 brd ff:ff:ff:ff:ff:ff
root@machineB:~# ip -6 neigh
2001:470:7071::1 dev tap0 lladdr 62:b0:18:37:1d:47 router REACHABLE
root@machineB:~#
FYI : neighbor cache in IPv6 is similar to arp cache in IPv4.
MachineA has a route for 2001:470:7071:200::/56 pointing towards machineB
root@machineA ~ # ip -6 route | grep 200::
2001:470:7071:200::/56 via 2001:470:7071::2 dev ovsbr1 metric 1
root@machineA ~ # ip -6 route get 2001:470:7071:200::5
2001:470:7071:200::5 from :: via 2001:470:7071::2 dev ovsbr1 src 2001:470:7071::1 metric 0
cache
root@machineA ~ #
Some extra info from machine A:
– Ubuntu Quantal. Linux kernel 3.5.0.
– The bridge is an OpenvSwitch bridge.
Problem
On machineA, I can’t connect or ping to hosts behind machineB. This is what happens.
root@machineA ~ # ping6 -c 1 2001:470:7071:200::5 PING 2001:470:7071:200::51(2001:470:7071:200::5) 56 data bytes ping: sendmsg: Network is down ^C --- 2001:470:7071:200::5 ping statistics --- 1 packets transmitted, 0 received, 100% packet loss, time 0ms root@machineA ~ #
I also checked with tcpdump, but nothing is send over the interface when I try to ping.
Solution
None yet. If you have an idea, let me know.
[update on 2013/02/26] Started working after a reboot. Too bad I didn’t find the real reason.
This same thing keeps happening to me. My virtual machine host suddenly loses the ability to contact one of its VMs, and ping gives this ‘Network is down’ thing. But i can still reach the VM with IPv4, and the VM can reach the other VMs on the box with IPv6. (but not the internet, which makes sense because that routes through the VM host)
It’s driving me nuts. I hate when errors happen with no error message. Rebooting (and temporarally taking down all my VMs in the process) can fix the problem, but it comes back sometimes.
I ran into the same problem. Funnily enough, I ran into it once after a reboot.
Maybe this problem report is related: https://lkml.org/lkml/2012/3/25/13 It doesn’t have any solution or explanation though.
The link mentions that the problem goes away when the routing table is modified. In my case, the problem also goes away when do “ip -6 route flush cache”. To (hopefully) work around the problem, I added a script to my network configuration which calls this command some time after bringing the bridge for my VMs up.
I found out that “ip -6 route flush cache” doesn’t solve the issue. I’m not sure why I’ve believed it did.
However, I found something else: The following doesn’t work:
brctl addbr br0
ip link set br0 up
ip addr add 1::1/128 dev br0
ip route add 2::2/128 dev br0
ip route add 2::/64 via 2::2
brctl addif br0 vm1
Then, ping6 gives “ping: sendmsg: Network is down” for everything in 2::/64 except for 2::2.
However, the following does work:
brctl addbr br0
brctl addif br0 vm1
ip link set br0 up
ip addr add 1::1/128 dev br0
ip route add 2::2/128 dev br0
ip route add 2::/64 via 2::2
Note that the device vm1 is added to the bridge br0 before the routes are added to br0. With this order, I do not get any errors. I find this rather strange.
I use Ubuntu 12.04 with kernel 3.2.0-55.