yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1321508] Re: Unable to create an instance from remote machine using novaclient

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Sean Dague <sean@xxxxxxxxx>
Date: Wed, 01 Apr 2015 13:02:56 -0000
Reply-to: Bug 1321508 <1321508@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
I'm assuming this is now addressed, haven't seen this in a while

** Changed in: nova
       Status: Incomplete => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1321508

Title:
  Unable to create an instance from remote machine using novaclient

Status in OpenStack Compute (Nova):
  Fix Released

Bug description:
  When attempting to boot an instance from a remote host using
  novaclient and API access file downloaded via dashboard I'm unable to
  create instances due to an error in attempting to retrieve networks.
  This is reproducible via devstack on both Precise and Trusty, and I've
  verified that using the same creds file on the actual compute host is
  succesful.

  It looks like this might be related to the following commit:
  https://github.com/openstack/nova/commit/869b435dca27e06f4160b781d86bba708475866a

  Nova-API Trace:
  ```````````````````````````````````````````````````````````````````````````

  etag: 51bc16b900bf0f814bb6c0c3dd8f0790
  x-image-meta-is_public: True
  x-image-meta-min_ram: 0
  x-image-meta-owner: 8109ccec0fec4106ad3f005fd76130bb
  x-image-meta-updated_at: 2014-05-21T00:19:30
  content-type: text/html; charset=UTF-8
  x-openstack-request-id: req-35660f52-3922-456b-b44c-6d65d8f696fa
  x-image-meta-disk_format: qcow2
  x-image-meta-name: Fedora-x86_64-20-20131211.1-sda
   from (pid=25312) log_http_response /opt/stack/python-glanceclient/glanceclient/common/http.py:153
  2014-05-20 18:53:38.456 ERROR nova.api.openstack [req-44e54528-f86b-4771-a7a6-91f6af164058 demo demo] Caught error: No networks defined.
  Traceback (most recent call last):

    File "/opt/stack/nova/nova/conductor/manager.py", line 597, in _object_dispatch
      return getattr(target, method)(context, *args, **kwargs)

    File "/opt/stack/nova/nova/objects/base.py", line 115, in wrapper
      result = fn(cls, context, *args, **kwargs)

    File "/opt/stack/nova/nova/objects/network.py", line 183, in get_by_uuids
      project_only)

    File "/opt/stack/nova/nova/db/api.py", line 1003, in network_get_all_by_uuids
      project_only=project_only)

    File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 164, in wrapper
      return f(*args, **kwargs)

    File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 2600, in network_get_all_by_uuids
      raise exception.NoNetworksFound()

  NoNetworksFound: No networks defined.

  Traceback (most recent call last):

    File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply
      incoming.message))

    File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch
      return self._do_dispatch(endpoint, method, ctxt, args)

    File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch
      result = getattr(endpoint, method)(ctxt, **new_args)

    File "/opt/stack/nova/nova/network/manager.py", line 1351, in validate_networks
      self._get_networks_by_uuids(context, network_uuids)

    File "/opt/stack/nova/nova/network/manager.py", line 1377, in _get_networks_by_uuids
      context, network_uuids, project_only="allow_none")

    File "/opt/stack/nova/nova/objects/base.py", line 113, in wrapper
      args, kwargs)

    File "/opt/stack/nova/nova/conductor/rpcapi.py", line 355, in object_class_action
      objver=objver, args=args, kwargs=kwargs)

    File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/client.py", line 150, in call
      wait_for_reply=True, timeout=timeout)

    File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/transport.py", line 90, in _send
      timeout=timeout)

    File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py", line 386, in send
      return self._send(target, ctxt, message, wait_for_reply, timeout)

    File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py", line 379, in _send
      raise result

  NoNetworksFound_Remote: No networks defined.
  Traceback (most recent call last):

    File "/opt/stack/nova/nova/conductor/manager.py", line 597, in _object_dispatch
      return getattr(target, method)(context, *args, **kwargs)

    File "/opt/stack/nova/nova/objects/base.py", line 115, in wrapper
      result = fn(cls, context, *args, **kwargs)

    File "/opt/stack/nova/nova/objects/network.py", line 183, in get_by_uuids
      project_only)

    File "/opt/stack/nova/nova/db/api.py", line 1003, in network_get_all_by_uuids
      project_only=project_only)

    File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 164, in wrapper
      return f(*args, **kwargs)

    File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 2600, in network_get_all_by_uuids
      raise exception.NoNetworksFound()

  NoNetworksFound: No networks defined.

  
  2014-05-20 18:53:38.456 TRACE nova.api.openstack Traceback (most recent call last):
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/__init__.py", line 125, in __call__
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     return req.get_response(self.application)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1320, in send
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     application, catch_exc_info=False)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1284, in call_application
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     app_iter = application(self.environ, start_response)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     return resp(environ, start_response)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/python-keystoneclient/keystoneclient/middleware/auth_token.py", line 632, in __call__
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     return self.app(env, start_response)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     return resp(environ, start_response)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     return resp(environ, start_response)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/routes/middleware.py", line 131, in __call__
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     response = self.app(environ, start_response)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     return resp(environ, start_response)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     return self.func(req, *args, **kwargs)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 917, in __call__
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     content_type, body, accept)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 983, in _process_stack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     action_result = self.dispatch(meth, request, action_args)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 1067, in dispatch
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     return method(req=request, **action_args)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 956, in create
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     legacy_bdm=legacy_bdm)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/hooks.py", line 103, in inner
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     rv = f(*args, **kwargs)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/compute/api.py", line 1333, in create
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     legacy_bdm=legacy_bdm)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/compute/api.py", line 976, in _create_instance
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     max_count)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/compute/api.py", line 747, in _validate_and_build_base_options
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     requested_networks, max_count)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/compute/api.py", line 463, in _check_requested_networks
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     max_count)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/network/api.py", line 46, in wrapped
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     return func(self, context, *args, **kwargs)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/network/api.py", line 371, in validate_networks
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     requested_networks)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/network/rpcapi.py", line 230, in validate_networks
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     return self.client.call(ctxt, 'validate_networks', networks=networks)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/client.py", line 361, in call
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     return self.prepare().call(ctxt, method, **kwargs)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/client.py", line 150, in call
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     wait_for_reply=True, timeout=timeout)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/transport.py", line 90, in _send
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     timeout=timeout)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py", line 386, in send
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     return self._send(target, ctxt, message, wait_for_reply, timeout)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py", line 379, in _send
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     raise result
  2014-05-20 18:53:38.456 TRACE nova.api.openstack NoNetworksFound_Remote: No networks defined.
  2014-05-20 18:53:38.456 TRACE nova.api.openstack Traceback (most recent call last):
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/conductor/manager.py", line 597, in _object_dispatch
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     return getattr(target, method)(context, *args, **kwargs)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/objects/base.py", line 115, in wrapper
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     result = fn(cls, context, *args, **kwargs)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/objects/network.py", line 183, in get_by_uuids
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     project_only)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/db/api.py", line 1003, in network_get_all_by_uuids
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     project_only=project_only)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 164, in wrapper
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     return f(*args, **kwargs)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 2600, in network_get_all_by_uuids
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     raise exception.NoNetworksFound()
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack NoNetworksFound: No networks defined.
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack Traceback (most recent call last):
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     incoming.message))
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     return self._do_dispatch(endpoint, method, ctxt, args)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     result = getattr(endpoint, method)(ctxt, **new_args)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     result = getattr(endpoint, method)(ctxt, **new_args)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/network/manager.py", line 1351, in validate_networks
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     self._get_networks_by_uuids(context, network_uuids)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/network/manager.py", line 1377, in _get_networks_by_uuids
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     context, network_uuids, project_only="allow_none")
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/objects/base.py", line 113, in wrapper
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     args, kwargs)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/conductor/rpcapi.py", line 355, in object_class_action
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     objver=objver, args=args, kwargs=kwargs)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/client.py", line 150, in call
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     wait_for_reply=True, timeout=timeout)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/transport.py", line 90, in _send
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     timeout=timeout)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py", line 386, in send
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     return self._send(target, ctxt, message, wait_for_reply, timeout)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py", line 379, in _send
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     raise result
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack NoNetworksFound_Remote: No networks defined.
  2014-05-20 18:53:38.456 TRACE nova.api.openstack Traceback (most recent call last):
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/conductor/manager.py", line 597, in _object_dispatch
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     return getattr(target, method)(context, *args, **kwargs)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/objects/base.py", line 115, in wrapper
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     result = fn(cls, context, *args, **kwargs)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/objects/network.py", line 183, in get_by_uuids
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     project_only)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/db/api.py", line 1003, in network_get_all_by_uuids
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     project_only=project_only)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 164, in wrapper
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     return f(*args, **kwargs)
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack   File "/opt/stack/nova/nova/db/sqlalchemy/api.py", line 2600, in network_get_all_by_uuids
  2014-05-20 18:53:38.456 TRACE nova.api.openstack     raise exception.NoNetworksFound()
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack NoNetworksFound: No networks defined.
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.456 TRACE nova.api.openstack
  2014-05-20 18:53:38.494 INFO nova.api.openstack [req-44e54528-f86b-4771-a7a6-91f6af164058 demo demo] http://172.16.140.247:8774/v2/3859a6e8a68f4e2db486a9fd31bbcc62/servers returned with HTTP 404
  2014-05-20 18:53:38.495 DEBUG nova.api.openstack.wsgi [req-44e54528-f86b-4771-a7a6-91f6af164058 demo demo] Returning 404 to user: The resource could not be found. from (pid=25312) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:1209
  2014-05-20 18:53:38.496 INFO nova.osapi_compute.wsgi.server [req-44e54528-f86b-4771-a7a6-91f6af164058 demo demo] 192.168.66.4 "POST /v2/3859a6e8a68f4e2db486a9fd31bbcc62/servers HTTP/1.1" status: 404 len: 272 time: 0.1439140

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1321508/+subscriptions
References

[Bug 1321508] [NEW] Unable to create an instance from remote machine using novaclient
From: John Griffith, 2014-05-21