Starting about 3 months ago, I started seeing some odd looking messages in my logs basically once a minute from OpenVPN.

$ journalctl -b -n6 -u openvpn@server
-- Logs begin at Sat 2012-11-10 12:22:39 CST, end at Mon 2013-02-25 21:53:30 CST. --
Feb 25 21:41:15 toofishes openvpn@server[31141]: IP packet with unknown IP version=15 seen
Feb 25 21:42:15 toofishes openvpn@server[31141]: IP packet with unknown IP version=15 seen
Feb 25 21:43:15 toofishes openvpn@server[31141]: IP packet with unknown IP version=15 seen
Feb 25 21:44:15 toofishes openvpn@server[31141]: IP packet with unknown IP version=15 seen
Feb 25 21:45:15 toofishes openvpn@server[31141]: IP packet with unknown IP version=15 seen
Feb 25 21:46:16 toofishes openvpn@server[31141]: IP packet with unknown IP version=15 seen

Now that Munin supports zooming, I was able to pin this down to a pretty specific point in time when the odd behavior started. This is a graph of tun0 traffic. Luckily I don't often send traffic to or from the VPN server itself; it is usually from client to client so I was able to see a difference of just a bits a minute in traffic.

OpenVPN tun0 weird traffic

Searching the Internet didn't yield much on this, unfortuately, so I punted on it for a while until it started bugging me tonight. What the heck was this? One related post started to dig into it, but "fixed" it by switching to a different non-Linode, Ubuntu-provided kernel. Although not a solution I was going to use, this realization was the key to understanding this whole situation and how to fix it.

  1. Linode's kernel has a builtin AoE block device driver as of 3.5.3.
  2. AoE has an EtherType of 0x88A2, which was noticed after doing a tcpdump on both the eth0 and lo interfaces and seeing the once-per-minute packet.
  3. Ebtables doesn't work to block this, which is what I tried first.
  4. By default, this sends out a "discover" query on all interfaces every 60 seconds, with no obvious way to disable this.

The solution I arrived at was to tell this silly built-in AoE driver to send discover packets to only the lo interface, where they will effectively be ignored. This skips both the public eth0 interface and the tun0 interface (which is used by OpenVPN).

echo 'lo' > /sys/module/aoe/parameters/aoe_iflist

This totally removed the bogus traffic from ever showing on tun0 and OpenVPN is no longer logging messages once per minute about a protocol it cannot understand.