touch-packages team mailing list archive

[Adam Ryczkowski <adam.ryczkowski@xxxxxxxxxxxxxx>]

After consulting with Serge Hallyn, the original author of the upstart
script that governs creation of the lxc-net bridge, I came up with the
following solution:

1. Turn off all lxc containers and the lxc-net (sudo service lxc-net stop)
2. Remove (or move away) the file /etc/init/lxc-net.conf
3. Create the file /etc/init/lxc-net.conf with the following contents:

description "lxc network"
author "Serge Hallyn <serge.hallyn@xxxxxxxxxxxxx>"

start on starting lxc
stop on stopped lxc

env USE_LXC_BRIDGE="true"
env LXC_BRIDGE="lxcbr0"
env LXC_ADDR="10.0.3.1"
env LXC_NETMASK="255.255.255.0"
env LXC_NETWORK="10.0.3.0/24"
env varrun="/run/lxc"
env LXC_DOMAIN=""

pre-start script
    [ -f /etc/default/lxc ] && . /etc/default/lxc

    [ "x$USE_LXC_BRIDGE" = "xtrue" ] || { stop; exit 0; }

    use_iptables_lock="-w"
    iptables -w -L -n > /dev/null 2>&1 || use_iptables_lock=""
    cleanup() {
        # dnsmasq failed to start, clean up the bridge
        iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 67 -j ACCEPT
        iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 67 -j ACCEPT
        iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 53 -j ACCEPT
        iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 53 -j ACCEPT
        iptables $use_iptables_lock -D FORWARD -i ${LXC_BRIDGE} -j ACCEPT
        iptables $use_iptables_lock -D FORWARD -o ${LXC_BRIDGE} -j ACCEPT
        iptables $use_iptables_lock -t nat -D POSTROUTING -s ${LXC_NETWORK} ! -d ${LXC_NETWORK} -j MASQUERADE || true
        iptables $use_iptables_lock -t mangle -D POSTROUTING -o ${LXC_BRIDGE} -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
        ifconfig ${LXC_BRIDGE} down || true
        brctl delbr ${LXC_BRIDGE} || true
    }
    if [ -d /sys/class/net/${LXC_BRIDGE} ]; then
        if [ ! -f ${varrun}/network_up ]; then
            # bridge exists, but we didn't start it
            stop;
        fi
        exit 0;
    fi

    # set up the lxc network
    brctl addbr ${LXC_BRIDGE} || { echo "Missing bridge support in kernel"; stop; exit 0; }
    echo 1 > /proc/sys/net/ipv4/ip_forward
    mkdir -p ${varrun}
    ifconfig ${LXC_BRIDGE} ${LXC_ADDR} netmask ${LXC_NETMASK} up
    iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p udp --dport 67 -j ACCEPT
    iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p tcp --dport 67 -j ACCEPT
    iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p udp --dport 53 -j ACCEPT
    iptables $use_iptables_lock -I INPUT -i ${LXC_BRIDGE} -p tcp --dport 53 -j ACCEPT
    iptables $use_iptables_lock -I FORWARD -i ${LXC_BRIDGE} -j ACCEPT
    iptables $use_iptables_lock -I FORWARD -o ${LXC_BRIDGE} -j ACCEPT
    iptables $use_iptables_lock -t nat -A POSTROUTING -s ${LXC_NETWORK} ! -d ${LXC_NETWORK} -j MASQUERADE
    iptables $use_iptables_lock -t mangle -A POSTROUTING -o ${LXC_BRIDGE} -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill

    touch ${varrun}/network_up
end script

post-stop script
    [ -f /etc/default/lxc ] && . /etc/default/lxc
    [ -f "${varrun}/network_up" ] || exit 0;
    # if $LXC_BRIDGE has attached interfaces, don't shut it down
    ls /sys/class/net/${LXC_BRIDGE}/brif/* > /dev/null 2>&1 && exit 0;

    if [ -d /sys/class/net/${LXC_BRIDGE} ]; then
        use_iptables_lock="-w"
        iptables -w -L -n > /dev/null 2>&1 || use_iptables_lock=""
        ifconfig ${LXC_BRIDGE} down
        iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 67 -j ACCEPT
        iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 67 -j ACCEPT
        iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p udp --dport 53 -j ACCEPT
        iptables $use_iptables_lock -D INPUT -i ${LXC_BRIDGE} -p tcp --dport 53 -j ACCEPT
        iptables $use_iptables_lock -D FORWARD -i ${LXC_BRIDGE} -j ACCEPT
        iptables $use_iptables_lock -D FORWARD -o ${LXC_BRIDGE} -j ACCEPT
        iptables $use_iptables_lock -t nat -D POSTROUTING -s ${LXC_NETWORK} ! -d ${LXC_NETWORK} -j MASQUERADE || true
        iptables $use_iptables_lock -t mangle -D POSTROUTING -o ${LXC_BRIDGE} -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
        pid=`cat ${varrun}/dnsmasq.pid 2>/dev/null` && kill -9 $pid || true
        rm -f ${varrun}/dnsmasq.pid
        brctl delbr ${LXC_BRIDGE}
    fi
    rm -f ${varrun}/network_up
end script

4. Create the file /etc/init/lxc-dnsmasq.conf with the following
contents:

description "lxc dnsmasq service"
author "Adam Ryczkowski, ispired by Serge Hallyn <serge.hallyn@xxxxxxxxxxxxx>"

expect fork

start on started lxc-net
stop on stopped lxc-net

env USE_LXC_BRIDGE="true"
env LXC_BRIDGE="lxcbr0"
env LXC_ADDR="10.0.3.1"
env LXC_NETMASK="255.255.255.0"
env LXC_NETWORK="10.0.3.0/24"
env LXC_DHCP_RANGE="10.0.3.2,10.0.3.254"
env LXC_DHCP_MAX="253"
env LXC_DHCP_CONFILE=""
env varrun="/run/lxc-dnsmasq"
env LXC_DOMAIN=""

pre-start script
    [ -f /etc/default/lxc ] && . /etc/default/lxc

    [ "x$USE_LXC_BRIDGE" = "xtrue" ] || { stop; exit 0; }

    if [ ! -d ${varrun} ]; then
        mkdir -p ${varrun}
    fi
    opts="$LXC_DOMAIN_ARG -u lxc-dnsmasq --strict-order --bind-interfaces --pid-file=${varrun}/dnsmasq.pid --conf-file=${LXC_DHCP_CONFILE} --listen-address ${LXC_ADDR} --dhcp-range ${LXC_DHCP_RANGE} --dhcp-lease-max=${LXC_DHCP_MAX} --dhcp-no-override --except-interface=lo --interface=${LXC_BRIDGE} --dhcp-leasefile=/var/lib/misc/dnsmasq.${LXC_BRIDGE}.leases --dhcp-authoritative --keep-in-foreground"

    /usr/sbin/dnsmasq $opts &

end script

post-stop script
    if [ -f ${varrun}/dnsmasq.pid ]; then
        PID=`cat ${varrun}/dnsmasq.pid`
        kill $PID
    fi
end script

5. Start the lxc-net again.


After that, if user wants to force dnsmasq to re-read its configuration, all he need to do is to "sudo service lxc-dnsmasq restart"

As far as I am concerned the problem is solved.

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to lxc in Ubuntu.
https://bugs.launchpad.net/bugs/1389849

Title:
  sudo service lxc-net restart does not reload dnsmasq when there is a
  container running

Status in “lxc” package in Ubuntu:
  New

Bug description:
  I know, that I can assign static IP manually, using
  /etc/network/interfaces.

  I also know, that I can read the MAC address of the LXC container
  (e.g. by looking for lxc.network.hwaddr entry in /var/lib/lxc
  /<container-name>/config and assign the IP based using entries dhcp-
  host=<mac-addr>,10.0.3.3 in /etc/dnsmasq.d/<some file>.

  In the file /etc/default/lxc-net I read

  # Uncomment the next line if you'd like to use a conf-file for the lxcbr0
  # dnsmasq.  For instance, you can use 'dhcp-host=mail1,10.0.3.100' to have
  # container 'mail1' always get ip address 10.0.3.100.
  #LXC_DHCP_CONFILE=/etc/lxc/dnsmasq.conf

  That would suit my needs; unfortunately doing so has no effect - at
  least not until the next computer reboot.

  I would expect that `sudo service lxc-net restart` forces the dnsmasq
  to reload - unfortunately it never works.

  To reproduce:
  On Ubuntu Trusty 14.04 64bit,
  1. install package lxc,
  2. create a container (e.g. sudo lxc-create -n mycontainer -t ubuntu -- -r trusty)
  3. start it (sudo lxc-start -d -n mycontainer)
  4. create another container (e.g. sudo lxc-create -n my2ndcontainer -t ubuntu -- -r trusty)
  5. edit /etc/default/lxc-net to uncomment the LXC_DHCP_CONFILE
  6. edit /etc/lxc/dnsmasq.conf to contain a line like `dhcp-host=my2ndcontainer,10.0.3.142`
  7. sudo service lxc-net restart
  8. start the 2nd container (sudo lxc-start -d -n my2ndcontainer)

  The steps 2 and 3 are optional.

  The 2nd container never gets the ip 10.0.3.142, but it keeps the
  assigned dynamic IP

  Walkaround 1:
  Turn off the computer and test again tomorrow.

  Walkaround 2 (more serious, but works only if steps 2 and 3 are
  skipped):

  name=my2ndcontainer
  sudo lxc-stop -n $name >/dev/null
  sudo service lxc-net stop >/dev/null
  if [ -d /sys/class/net/$internalif ]; then
     sudo brctl delbr $internalif >/dev/null #Why? See below.
  fi
  sudo rm /var/lib/misc/dnsmasq.$internalif.leases
  sudo service lxc-net start >/dev/null
  sudo lxc-start -d -n $name >/dev/null
  sleep 5

  Unfortunately, there is a bug (feature?) in the /etc/init/lxc-net.conf
  in Ubuntu 14.04 that prevents reloading the dnsmasq unless the bridge
  device is down for the host.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1389849/+subscriptions