yahoo-eng-team team mailing list archive

[Matt Riedemann <mriedem.os@xxxxxxxxx>]

Public bug reported:

I noticed today that when shutting down an instance, like for a snapshot
test, it's sometimes taking a full 60 seconds to wait for the graceful
shutdown before just destroying the guest:

http://logs.openstack.org/56/656656/12/check/nova-multi-
cell/6a403ba/compute1/logs/screen-n-cpu.txt.gz#_May_16_19_47_26_017509

May 16 19:47:26.017509 ubuntu-bionic-rax-dfw-0006201136 nova-
compute[17635]: INFO nova.virt.libvirt.driver [None req-bde576dc-
5de2-4113-982d-025c65b97e8d tempest-MigrationsAdminTest-391674513
tempest-MigrationsAdminTest-391674513] [instance:
21c45b79-b0f4-48a8-8949-9f480beb5864] Instance failed to shutdown in 60
seconds.

http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22Instance%20failed%20to%20shutdown%20in%2060%20seconds.%5C%22%20AND%20tags%3A%5C%22screen-n-cpu.txt%5C%22&from=7d

This could contribute to overall job/test timeouts since a lot of
tempest test wait operations are using the build_timeout config and set
to 196 seconds. I'm not sure why these guests aren't stopping, but they
should be since they shouldn't be doing anything. Looking at logstash
affected branches it seems this started in Stein.

This is with libvirt 4.0.0-1ubuntu8.10 and qemu 1:2.11+dfsg-1ubuntu7.13.

I don't see any errors in the libvirtd logs, the shutdown request
returns with code 0.

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: gate libvirt

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1829896

Title:
  libvirt: "Instance failed to shutdown in 60 seconds." in the gate

Status in OpenStack Compute (nova):
  New

Bug description:
  I noticed today that when shutting down an instance, like for a
  snapshot test, it's sometimes taking a full 60 seconds to wait for the
  graceful shutdown before just destroying the guest:

  http://logs.openstack.org/56/656656/12/check/nova-multi-
  cell/6a403ba/compute1/logs/screen-n-cpu.txt.gz#_May_16_19_47_26_017509

  May 16 19:47:26.017509 ubuntu-bionic-rax-dfw-0006201136 nova-
  compute[17635]: INFO nova.virt.libvirt.driver [None req-bde576dc-
  5de2-4113-982d-025c65b97e8d tempest-MigrationsAdminTest-391674513
  tempest-MigrationsAdminTest-391674513] [instance:
  21c45b79-b0f4-48a8-8949-9f480beb5864] Instance failed to shutdown in
  60 seconds.

  http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22Instance%20failed%20to%20shutdown%20in%2060%20seconds.%5C%22%20AND%20tags%3A%5C%22screen-n-cpu.txt%5C%22&from=7d

  This could contribute to overall job/test timeouts since a lot of
  tempest test wait operations are using the build_timeout config and
  set to 196 seconds. I'm not sure why these guests aren't stopping, but
  they should be since they shouldn't be doing anything. Looking at
  logstash affected branches it seems this started in Stein.

  This is with libvirt 4.0.0-1ubuntu8.10 and qemu 1:2.11+dfsg-
  1ubuntu7.13.

  I don't see any errors in the libvirtd logs, the shutdown request
  returns with code 0.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1829896/+subscriptions