touch-packages team mailing list archive

[Eric Desrochers <eric.desrochers@xxxxxxxxxxxxx>]

I also noticed the situation can be reproduced at boot if the value of
"net.core.rmem_default" is too low.

I reproduced it by only lowering the "net.core.rmem_default = 2000"
value with 6 network interface at boot.

ntpd[851]: ntp_io: estimated max descriptors: 1024, initial socket boundary: 16
ntpd[851]: Listen and drop on 0 v4wildcard 0.0.0.0 UDP 123
ntpd[851]: Listen and drop on 1 v6wildcard :: UDP 123
ntpd[851]: Listen normally on 2 eth1 192.168.1.10 UDP 123
ntpd[851]: Listen normally on 3 eth2 192.168.2.10 UDP 123
ntpd[851]: Listen normally on 4 eth3 192.168.3.10 UDP 123
ntpd[851]: Listen normally on 5 eth4 192.168.4.10 UDP 123
ntpd[851]: Listen normally on 6 eth5 192.168.5.10 UDP 123
ntpd[851]: Listen normally on 7 eth6 192.168.6.10 UDP 123
ntpd[851]: peers refreshed
ntpd[851]: Listening on routing socket on fd #24 for interface updates
ntpd[851]: Deferring DNS for 0.ubuntu.pool.ntp.org 1
ntpd[851]: Deferring DNS for 1.ubuntu.pool.ntp.org 1
ntpd[851]: Deferring DNS for 2.ubuntu.pool.ntp.org 1
ntpd[851]: Deferring DNS for 3.ubuntu.pool.ntp.org 1
ntpd[851]: Deferring DNS for ntp.ubuntu.com 1
ntpd[864]: signal_no_reset: signal 17 had flags 4000000
===> ntpd[851]: i/o error on routing socket No buffer space available - disabling <===

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to ntp in Ubuntu.
https://bugs.launchpad.net/bugs/1481388

Title:
  NTP : Use-after-free in routing socket code after dropping root

Status in ntp package in Ubuntu:
  Confirmed
Status in ntp source package in Precise:
  In Progress
Status in ntp source package in Trusty:
  Confirmed
Status in ntp source package in Vivid:
  Confirmed
Status in ntp source package in Wily:
  Confirmed

Bug description:
  [Impact]

   * User experienced repeated segfaults at the same instruction pointer

  i/o error on routing socket No buffer space available - disabling
  segfault at 31 ip 0000000000000031 sp 00007ffff9f11788 error 14 in libpthread-2.15.so[7f967a5d9000+18000] 

  The remove_ and delete_ functions remove the current element from the
  asyncio_reader_list, and free it, respectively.

  We then return back to the loop at the top, wherein the asyncio_reader variable still points at the now-freed element, whose contents are (in theory) now scrambled
  by having link pointers, etc, from internal malloc state overlaying the data.

  [Test Case]

  You can easily reproduce the bug by :

  - Lowering the sysctl value net.core.rmem_max

  $ sysctl -w net.core.wmem_max=<LOWER_VALUE>
  This sets the max OS send buffer size for all types of connections.

  - Adding multiple network interfaces and static routes.

  [Regression Potential]

  None expected since the fix is already available upstream
  (https://github.com/ntp-project/ntp.git) and Debian package.

  If after installing the patch, user are receiving this kind of message in /var/log/syslog : "routing socket reports: No buffer space available".
  The next step, would be to increase the "net.core.rmem_max" and "net.core.wmem_max" values equally until the "routing socket reports: No buffer space available" message no longer showed up.

  [Other Info]

  NTP upstream (https://github.com/ntp-project/ntp.git)
  [Bug 2224] Use-after-free in routing socket code after dropping root. - Commit: d6df9d3
  [Bug 2890] Ignore ENOBUFS on routing netlink socket. - Commit: db47bd4 

  The use-after-free bug has been fix in Debian release (closes: #795315)
  Will submit the ignore-ENOBUFS-on-routing-netlink-socket in Debian in the next days.

  [Original Description]

  We have 1 server (among hundreds) that its ntp service is crashing.

  A few minute/seconds after a start attempts we can see the following in syslog:
  ntpd[2729]: peers refreshed
  ntpd[2729]: Listening on routing socket on fd #49 for interface updates
  ntpd[2729]: i/o error on routing socket No buffer space available - disabling
  kernel: [157516.495224] ntpd[2729]: segfault at 31 ip 0000000000000031 sp 00007ffff9f11788 error 14 in libpthread-2.15.so[7f967a5d9000+18000]

  OS: Ubuntu 12.04.4 LTS
  Kernel: 3.11.0-19-generic

  I tried to compare it to other servers, and the only thing I could find that is different is that while it's up (before it crashes) I can see the following when running "lsof | grep ntp":
  ntpd 2729 ntp 49u sock 0,7 0t0 2473952565 can't identify protocol.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ntp/+bug/1481388/+subscriptions