yahoo-eng-team team mailing list archive

Thread
Date

[Bug 1137060] Re: RPC timeouts (when using raw image backend for example)

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: "Gary W. Smith" <gary.smith@xxxxxxxx>
Date: Thu, 14 Sep 2017 15:52:38 -0000
Reply-to: Bug 1137060 <1137060@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx

This bug was last updated over 4 years ago, and as there have
been many changes to both nova and horizon since then, this is
getting marked as Invalid. If the issue still exists, please 
feel free to reopen it.

** Changed in: horizon
       Status: Confirmed => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Dashboard (Horizon).
https://bugs.launchpad.net/bugs/1137060

Title:
  RPC timeouts (when using raw image backend for example)

Status in OpenStack Dashboard (Horizon):
  Invalid

Bug description:
  Release found: Folsom

  Environment:
  A single node running all services
  Ubuntu 12.04
  Using Ubuntu cloud archive packages

  Steps to reproduce:
  1. Upload a big image, for example a windows 7 image of 8GB (I guess any big image will do, but we verified this with a big win7 image)
  2. Add the following flag to nova.conf: libvirt_images_type=raw and restart nova-compute
  3. Use horizon to launch 20 instances at once, using the win7 image

  Current result:
  Only 3..7 out of 20 instances are able to launch successfuly, all other instances become in error state because of rpc timeout of nova-network

  Expected result:
  Higher ratio of success preferably 20, maybe by limiting the amount of allowed instances that can be started when raw backend is being used. Or maybe this should be done by limiting system io for the spawn instance process?

  Additional info:
  If we change to libvirt_images_type=default the exact same hardware/setup can launch 20 instances without any rpc timeouts. It seems that the big image copy part is causing heavy load, but when using qcow2 it uses a backing file so it doesn't have to copy the whole image.

  Related error log which shows the RPC timeout:
  http://paste.openstack.org/show/32687/

  When you use nova-network in multi_host mode the same issue may be
  experiencied when launching lots of instances. So another way to
  reproduce is to deploy several nodes all running nova-network
  multi_host and then launch 20 x N instances where N == amount of
  compute nodes.

To manage notifications about this bug go to:
https://bugs.launchpad.net/horizon/+bug/1137060/+subscriptions