yahoo-eng-team team mailing list archive

[Matt Riedemann <mriedem@xxxxxxxxxx>]

Marked as high priority since it's nova-net specific at this point and
that's what the related fixes are for.

** Also affects: nova/kilo
   Importance: Undecided
       Status: New

** Also affects: nova/liberty
   Importance: Undecided
       Status: New

** Changed in: nova/kilo
       Status: New => In Progress

** Changed in: nova/liberty
       Status: New => In Progress

** Changed in: nova/kilo
     Assignee: (unassigned) => Sean Dague (sdague)

** Changed in: nova/liberty
     Assignee: (unassigned) => Sean Dague (sdague)

** Changed in: nova
   Importance: Undecided => High

** Changed in: nova/kilo
   Importance: Undecided => High

** Changed in: nova/liberty
   Importance: Undecided => High

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1532809

Title:
  Gate failures when DHCP lease cannot be acquired

Status in OpenStack Compute (nova):
  In Progress
Status in OpenStack Compute (nova) kilo series:
  In Progress
Status in OpenStack Compute (nova) liberty series:
  In Progress

Bug description:
  Example from:
  http://logs.openstack.org/97/265697/1/check/gate-grenade-dsvm/6eeced7/console.html#_2016-01-11_07_42_30_838

  Logstash query:
  message:"No lease, failing" AND voting:1

  dhcp_release for an ip/mac does not seem to reach dnsmasq (or it fails
  to act on it - "unknown lease") as i don't see entries in syslog for
  it.

  Logs from nova network:
  dims@dims-mac:~/junk/6eeced7$ grep dhcp_release old/screen-n-net.txt.gz | grep 10.1.0.3 | grep CMD
  2016-01-11 07:25:35.548 DEBUG oslo_concurrency.processutils [req-62aaa0b9-093e-4f28-805d-d4bf3008bfe6 tempest-ServersTestJSON-1206086292 tempest-ServersTestJSON-1551541405] CMD "sudo nova-rootwrap /etc/nova/rootwrap.conf dhcp_release br100 10.1.0.3 fa:16:3e:32:51:c3" returned: 0 in 0.117s execute /usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:297
  2016-01-11 07:25:51.259 DEBUG oslo_concurrency.processutils [req-31115ffa-8d43-4621-bb2e-351d6cd4bcef tempest-ServerActionsTestJSON-357128318 tempest-ServerActionsTestJSON-854742699] CMD "sudo nova-rootwrap /etc/nova/rootwrap.conf dhcp_release br100 10.1.0.3 fa:16:3e:a4:f0:11" returned: 0 in 0.108s execute /usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:297
  2016-01-11 07:26:35.357 DEBUG oslo_concurrency.processutils [req-c32a216e-d909-41a0-a0bc-d5eb7a21c048 tempest-TestVolumeBootPattern-46217374 tempest-TestVolumeBootPattern-1056816637] CMD "sudo nova-rootwrap /etc/nova/rootwrap.conf dhcp_release br100 10.1.0.3 fa:16:3e:ed:de:f6" returned: 0 in 0.110s execute /usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py:297

  Logs from syslog:
  dims@dims-mac:~/junk$ grep 10.1.0.3 syslog.txt.gz
  Jan 11 07:25:35 devstack-trusty-rax-iad-7090830 dnsmasq-dhcp[7798]: DHCPRELEASE(br100) 10.1.0.3 fa:16:3e:32:51:c3 unknown lease
  Jan 11 07:25:51 devstack-trusty-rax-iad-7090830 dnsmasq-dhcp[7798]: DHCPRELEASE(br100) 10.1.0.3 fa:16:3e:a4:f0:11 unknown lease
  Jan 11 07:26:24 devstack-trusty-rax-iad-7090830 dnsmasq-dhcp[7798]: DHCPOFFER(br100) 10.1.0.3 fa:16:3e:ed:de:f6
  Jan 11 07:26:24 devstack-trusty-rax-iad-7090830 dnsmasq-dhcp[7798]: DHCPREQUEST(br100) 10.1.0.3 fa:16:3e:ed:de:f6
  Jan 11 07:26:24 devstack-trusty-rax-iad-7090830 dnsmasq-dhcp[7798]: DHCPACK(br100) 10.1.0.3 fa:16:3e:ed:de:f6 tempest
  Jan 11 07:27:34 devstack-trusty-rax-iad-7090830 object-auditor: Object audit (ALL). Since Mon Jan 11 07:27:34 2016: Locally: 1 passed, 0 quarantined, 0 errors files/sec: 2.03 , bytes/sec: 10119063.16, Total time: 0.49, Auditing time: 0.00, Rate: 0.00
  Jan 11 07:39:12 devstack-trusty-rax-iad-7090830 dnsmasq-dhcp[7798]: not using configured address 10.1.0.3 because it is leased to fa:16:3e:ed:de:f6
  Jan 11 07:40:12 devstack-trusty-rax-iad-7090830 dnsmasq-dhcp[7798]: not using configured address 10.1.0.3 because it is leased to fa:16:3e:ed:de:f6
  Jan 11 07:41:12 devstack-trusty-rax-iad-7090830 dnsmasq-dhcp[7798]: not using configured address 10.1.0.3 because it is leased to fa:16:3e:ed:de:f6
  Jan 11 07:42:26 devstack-trusty-rax-iad-7090830 dnsmasq-dhcp[7798]: DHCPRELEASE(br100) 10.1.0.3 fa:16:3e:fe:1f:36 unknown lease

  Net, The test that runs the ssh against the vm fails with the "No
  lease, failing" in its serial console

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1532809/+subscriptions