yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1556758] Re: Instance create error because of timeout

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: zhaozhilong <zhaozhilong@xxxxxxxxxxxxxxx>
Date: Mon, 14 Mar 2016 06:37:31 -0000
Reply-to: Bug 1556758 <1556758@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
nova-api get an error :

2016-03-14 14:35:48.479 ERROR nova.api.openstack.extensions [req-ce1e8c97-3a63-47e3-bcd0-cb2ff7d1d6d2 admin demo] Unexpected exception in API method
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions Traceback (most recent call last):
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions   File "/opt/stack/nova/nova/api/openstack/extensions.py", line 478, in wrapped
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions     return f(*args, **kwargs)
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions   File "/opt/stack/nova/nova/api/validation/__init__.py", line 73, in wrapper
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions     return func(*args, **kwargs)
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions   File "/opt/stack/nova/nova/api/validation/__init__.py", line 73, in wrapper
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions     return func(*args, **kwargs)
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions   File "/opt/stack/nova/nova/api/validation/__init__.py", line 73, in wrapper
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions     return func(*args, **kwargs)
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions   File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 630, in create
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions     **create_kwargs)
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions   File "/opt/stack/nova/nova/hooks.py", line 149, in inner
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions     rv = f(*args, **kwargs)
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions   File "/opt/stack/nova/nova/compute/api.py", line 1497, in create
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions     check_server_group_quota=check_server_group_quota)
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions   File "/opt/stack/nova/nova/compute/api.py", line 1079, in _create_instance
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions     reservation_id, max_count)
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions   File "/opt/stack/nova/nova/compute/api.py", line 829, in _validate_and_build_base_options
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions     requested_networks, max_count)
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions   File "/opt/stack/nova/nova/compute/api.py", line 448, in _check_requested_networks
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions     max_count)
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions   File "/opt/stack/nova/nova/network/api.py", line 49, in wrapped
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions     return func(self, context, *args, **kwargs)
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions   File "/opt/stack/nova/nova/network/api.py", line 399, in validate_networks
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions     requested_networks)
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions   File "/opt/stack/nova/nova/network/rpcapi.py", line 212, in validate_networks
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions     return self.client.call(ctxt, 'validate_networks', networks=networks)
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 413, in call
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions     return self.prepare().call(ctxt, method, **kwargs)
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 158, in call
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions     retry=self.retry)
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 90, in _send
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions     timeout=timeout, retry=retry)
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 470, in send
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions     retry=retry)
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 459, in _send
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions     result = self._waiter.wait(msg_id, timeout)
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 342, in wait
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions     message = self.waiters.get(msg_id, timeout=timeout)
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 244, in get
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions     'to message ID %s' % msg_id)
2016-03-14 14:35:48.479 TRACE nova.api.openstack.extensions MessagingTimeout: Timed out waiting for a reply to message ID 1b0fb47698c74924ab04060b0d8d1d4f


** Changed in: nova
       Status: New => Invalid

** Changed in: heat
       Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1556758

Title:
  Instance create error because of  timeout

Status in heat:
  Invalid
Status in OpenStack Compute (nova):
  Invalid

Bug description:
  2016-03-14 14:19:47.620 TRACE heat.engine.resource Traceback (most recent call last):
  2016-03-14 14:19:47.620 TRACE heat.engine.resource   File "/opt/stack/heat/heat/engine/resource.py", line 688, in _action_recorder
  2016-03-14 14:19:47.620 TRACE heat.engine.resource     yield
  1). my yaml file as blow:
  heat_template_version: 2015-10-15
   
  description: HOT template for one interconnected VMs with floating ips.
   
  parameters:
    image_id:
      type: string
      description: Image Name
      default: cirros-0.3.4-x86_64-uec
   
    public_net:
      type: string
      description: public network id
      default: a059ae2f-0eed-468f-b2a2-a0427f621da1 
   
  resources:
    private_net:
      type: OS::Neutron::Net
      properties:
        name: private-net
        
    private_subnet:
      type: OS::Neutron::Subnet
      properties:
        network_id: { get_resource: private_net }
        cidr: 172.16.2.0/24
        gateway_ip: 172.16.2.1
   
    server1:
      type: OS::Nova::Server
      properties:
        name: Server1
        image: { get_param: image_id }
        flavor: m1.tiny
        networks:
          - network: {get_resource: private_net }

  
  2). when i try to create an instance, i get an error.(since )
  2016-03-14 14:28:36.237 TRACE heat.engine.resource Traceback (most recent call last):
  2016-03-14 14:28:36.237 TRACE heat.engine.resource   File "/opt/stack/heat/heat/engine/resource.py", line 688, in _action_recorder
  2016-03-14 14:28:36.237 TRACE heat.engine.resource     yield
  2016-03-14 14:28:36.237 TRACE heat.engine.resource   File "/opt/stack/heat/heat/engine/resource.py", line 759, in _do_action
  2016-03-14 14:28:36.237 TRACE heat.engine.resource     yield self.action_handler_task(action, args=handler_args)
  2016-03-14 14:28:36.237 TRACE heat.engine.resource   File "/opt/stack/heat/heat/engine/scheduler.py", line 297, in wrapper
  2016-03-14 14:28:36.237 TRACE heat.engine.resource     step = next(subtask)
  2016-03-14 14:28:36.237 TRACE heat.engine.resource   File "/opt/stack/heat/heat/engine/resource.py", line 730, in action_handler_task
  2016-03-14 14:28:36.237 TRACE heat.engine.resource     handler_data = handler(*args)
  2016-03-14 14:28:36.237 TRACE heat.engine.resource   File "/opt/stack/heat/heat/engine/resources/openstack/nova/server.py", line 822, in handle_create
  2016-03-14 14:28:36.237 TRACE heat.engine.resource     admin_pass=admin_pass)
  2016-03-14 14:28:36.237 TRACE heat.engine.resource   File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1038, in create
  2016-03-14 14:28:36.237 TRACE heat.engine.resource     **boot_kwargs)
  2016-03-14 14:28:36.237 TRACE heat.engine.resource   File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 555, in _boot
  2016-03-14 14:28:36.237 TRACE heat.engine.resource     return_raw=return_raw, **kwargs)
  2016-03-14 14:28:36.237 TRACE heat.engine.resource   File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 302, in _create
  2016-03-14 14:28:36.237 TRACE heat.engine.resource     _resp, body = self.api.client.post(url, body=body)
  2016-03-14 14:28:36.237 TRACE heat.engine.resource   File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 451, in post
  2016-03-14 14:28:36.237 TRACE heat.engine.resource     return self._cs_request(url, 'POST', **kwargs)
  2016-03-14 14:28:36.237 TRACE heat.engine.resource   File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 426, in _cs_request
  2016-03-14 14:28:36.237 TRACE heat.engine.resource     resp, body = self._time_request(url, method, **kwargs)
  2016-03-14 14:28:36.237 TRACE heat.engine.resource   File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 399, in _time_request
  2016-03-14 14:28:36.237 TRACE heat.engine.resource     resp, body = self.request(url, method, **kwargs)
  2016-03-14 14:28:36.237 TRACE heat.engine.resource   File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 393, in request
  2016-03-14 14:28:36.237 TRACE heat.engine.resource     raise exceptions.from_response(resp, body, url, method)
  2016-03-14 14:28:36.237 TRACE heat.engine.resource ClientException: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  2016-03-14 14:28:36.237 TRACE heat.engine.resource <class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-d1fb3f88-9623-47b6-b3cd-74703e2ae89e)

To manage notifications about this bug go to:
https://bugs.launchpad.net/heat/+bug/1556758/+subscriptions
References

[Bug 1556758] [NEW] Instance create error because of timeout
From: zhaozhilong, 2016-03-14