yahoo-eng-team team mailing list archive

[Adam Gandelman <1327406@xxxxxxxxxxxxxxxxxx>]

** Also affects: nova/icehouse
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1327406

Title:
  The One And Only network is variously visible

Status in OpenStack Compute (Nova):
  Fix Released
Status in OpenStack Compute (nova) icehouse series:
  New

Bug description:
  I am testing with the templates in
  https://review.openstack.org/#/c/97366/

  I can create a stack.  I can use `curl` to hit the webhooks to scale
  up and down the old-style group and to scale down the new-style group;
  those all work.  What fails is hitting the webhook to scale up the
  new-style group.  Here is a typescript showing the failure:

  $ curl -X POST
  'http://10.10.0.125:8000/v1/signal/arn%3Aopenstack%3Aheat%3A%3A39675672862f4bd08505bfe1283773e0%3Astacks%2Ftest4
  %2F3cd6160b-
  d8c5-48f1-a527-4c7df9205fc3%2Fresources%2FNewScaleUpPolicy?Timestamp=2014-06-06T19%3A45%3A27Z&SignatureMethod=HmacSHA256&AWSAccessKeyId=35678396d987432f87cda8e4c6cdbfb5&SignatureVersion=2&Signature=W3aJQ6SR7O5lLOxLEQndbzNB%2FUhefr1W7qO9zNZ%2BHVs%3D'

  <ErrorResponse><Error><Message>The request processing has failed due to an internal error:Remote error: ResourceFailure Error: Nested stack UPDATE failed: Error: Resource CREATE failed: NotFound: No Network matching {'label': u'private'}. (HTTP 404)
  [u'Traceback (most recent call last):\n', u'  File "/opt/stack/heat/heat/engine/service.py", line 61, in wrapped\n    return func(self, ctx, *args, **kwargs)\n', u'  File "/opt/stack/heat/heat/engine/service.py", line 911, in resource_signal\n    stack[resource_name].signal(details)\n', u'  File "/opt/stack/heat/heat/engine/resource.py", line 879, in signal\n    raise failure\n', u"ResourceFailure: Error: Nested stack UPDATE failed: Error: Resource CREATE failed: NotFound: No Network matching {'label': u'private'}. (HTTP 404)\n"].</Message><Code>InternalFailure</Code><Type>Server</Type></Error></ErrorResponse>

  The original sin looks like this in the heat engine log:

  2014-06-06 17:39:20.013 28692 DEBUG urllib3.connectionpool [req-2391a9ea-46d6-46f0-9a7b-cf999a8697e9 ] "GET /v2/39675672862f4bd08505bfe1283773e0/os-networks HTTP/1.1" 200 16 _make_request /usr/lib/python2.7/dist-packages/urllib3/connectionpool.py:415
  2014-06-06 17:39:20.014 28692 ERROR heat.engine.resource [req-2391a9ea-46d6-46f0-9a7b-cf999a8697e9 None] CREATE : Server "my_instance" Stack "test1-new_style-qidqbd5nrk44-43e7l57kqf5w-4t3xdjrfrr7s" [20523269-0ebb-45b8-ad59-75f55607f3bd]
  2014-06-06 17:39:20.014 28692 TRACE heat.engine.resource Traceback (most recent call last):
  2014-06-06 17:39:20.014 28692 TRACE heat.engine.resource   File "/opt/stack/heat/heat/engine/resource.py", line 383, in _do_action
  2014-06-06 17:39:20.014 28692 TRACE heat.engine.resource     handle())
  2014-06-06 17:39:20.014 28692 TRACE heat.engine.resource   File "/opt/stack/heat/heat/engine/resources/server.py", line 493, in handle_create
  2014-06-06 17:39:20.014 28692 TRACE heat.engine.resource     nics = self._build_nics(self.properties.get(self.NETWORKS))
  2014-06-06 17:39:20.014 28692 TRACE heat.engine.resource   File "/opt/stack/heat/heat/engine/resources/server.py", line 597, in _build_nics
  2014-06-06 17:39:20.014 28692 TRACE heat.engine.resource     network = self.nova().networks.find(label=label_or_uuid)
  2014-06-06 17:39:20.014 28692 TRACE heat.engine.resource   File "/opt/stack/python-novaclient/novaclient/base.py", line 194, in find
  2014-06-06 17:39:20.014 28692 TRACE heat.engine.resource     raise exceptions.NotFound(msg)
  2014-06-06 17:39:20.014 28692 TRACE heat.engine.resource NotFound: No Network matching {'label': u'private'}. (HTTP 404)

  Private debug logging reveals that in the scale-up case, the call to
  "GET /v2/{tenant-id}/os-networks HTTP/1.1" returns with response code
  200 and an empty list of networks.  Comparing with the corresponding
  call when the stack is being created shows no difference in the calls
  --- because the normal logging omits the headers --- even though the
  results differ (when the stack is being created, the result contains
  the correct list of networks).  Turning on HTTP debug logging in the
  client reveals that the X-Auth-Token headers differ.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1327406/+subscriptions