openstack team mailing list archive

Thread
Date

Re: VM creation failure

To: JuanFra Rodriguez Cardoso <juanfra.rodriguez.cardoso@xxxxxxxxx>
From: Javier Alvarez <javier.alvarez@xxxxxx>
Date: Tue, 26 Feb 2013 09:46:10 +0100
Cc: openstack@xxxxxxxxxxxxxxxxxxx
In-reply-to: <CAGXO4kzCVyk5Ek7JPZNDL4UcKGUaNRKAFyBcdB8cetw84wcBJw@mail.gmail.com>
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130106 Thunderbird/17.0.2

Hello JuanFra,

How can I check that? I have rabbitmq-server running on the controllernode and I have set rabbit_host property in the .conf files. Also, if Irestart nova-compute service, I get the following message:

2013-02-26 09:43:52 INFO nova.rpc.common [-] Connected to AMQP server on172.16.8.1:5672


Is there any other test I could do to verify everything is properly working?

Thanks,

Javi

On 25/02/13 20:09, JuanFra Rodriguez Cardoso wrote:

Hi:

Have you ensured your messaging queue is working correctly?

Regards,
JuanFra

2013/2/25 Javier Alvarez <javier.alvarez@xxxxxx<mailto:javier.alvarez@xxxxxx>>


    Hello,

    I'm trying to setup OpenStack Essex on Debian and I'm getting an
    error when creating new VMs. The problem arises when trying to
    allocate the network of the VM, but I'm not sure why it's
    happening, I'm using nova-network, and that's the log output:

    2013-02-25 18:20:28 AUDIT nova.compute.manager
    [req-4d211944-bd5c-4e88-a8df-c4a9167676e2
    36a7e60b4d134307b61e06949d33735e 9558f53959a04cc992c3f8b6d91bfb9f]
    [instance: 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] Starting instance...
    2013-02-25 18:20:40 INFO nova.virt.libvirt.connection [-]
    Compute_service record updated for bscgrid21
    2013-02-25 18:20:40 INFO nova.compute.manager [-] Updating host status
    2013-02-25 18:21:28 ERROR nova.rpc.common
    [req-4d211944-bd5c-4e88-a8df-c4a9167676e2
    36a7e60b4d134307b61e06949d33735e 9558f53959a04cc992c3f8b6d91bfb9f]
    Timed out waiting for RPC response: timed out
    2013-02-25 18:21:28 TRACE nova.rpc.common Traceback (most recent
    call last):
    2013-02-25 18:21:28 TRACE nova.rpc.common   File
    "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line
    490, in ensure
    2013-02-25 18:21:28 TRACE nova.rpc.common     return method(*args,
    **kwargs)
    2013-02-25 18:21:28 TRACE nova.rpc.common   File
    "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line
    567, in _consume
    2013-02-25 18:21:28 TRACE nova.rpc.common     return
    self.connection.drain_events(timeout=timeout)
    2013-02-25 18:21:28 TRACE nova.rpc.common   File
    "/usr/lib/python2.7/dist-packages/kombu/connection.py", line 167,
    in drain_events
    2013-02-25 18:21:28 TRACE nova.rpc.common     return
    self.transport.drain_events(self.connection, **kwargs)
    2013-02-25 18:21:28 TRACE nova.rpc.common   File
    "/usr/lib/python2.7/dist-packages/kombu/transport/amqplib.py",
    line 262, in drain_events
    2013-02-25 18:21:28 TRACE nova.rpc.common     return
    connection.drain_events(**kwargs)
    2013-02-25 18:21:28 TRACE nova.rpc.common   File
    "/usr/lib/python2.7/dist-packages/kombu/transport/amqplib.py",
    line 94, in drain_events
    2013-02-25 18:21:28 TRACE nova.rpc.common     return
    self.wait_multi(self.channels.values(), timeout=timeout)
    2013-02-25 18:21:28 TRACE nova.rpc.common   File
    "/usr/lib/python2.7/dist-packages/kombu/transport/amqplib.py",
    line 100, in wait_multi
    2013-02-25 18:21:28 TRACE nova.rpc.common chanmap.keys(),
    allowed_methods, timeout=timeout)
    2013-02-25 18:21:28 TRACE nova.rpc.common   File
    "/usr/lib/python2.7/dist-packages/kombu/transport/amqplib.py",
    line 159, in _wait_multiple
    2013-02-25 18:21:28 TRACE nova.rpc.common     channel, method_sig,
    args, content = read_timeout(timeout)
    2013-02-25 18:21:28 TRACE nova.rpc.common   File
    "/usr/lib/python2.7/dist-packages/kombu/transport/amqplib.py",
    line 132, in read_timeout
    2013-02-25 18:21:28 TRACE nova.rpc.common     return
    self.method_reader.read_method()
    2013-02-25 18:21:28 TRACE nova.rpc.common   File
    "/usr/lib/python2.7/dist-packages/amqplib/client_0_8/method_framing.py",
    line 221, in read_method
    2013-02-25 18:21:28 TRACE nova.rpc.common     raise m
    2013-02-25 18:21:28 TRACE nova.rpc.common timeout: timed out
    2013-02-25 18:21:28 TRACE nova.rpc.common
    2013-02-25 18:21:28 ERROR nova.compute.manager
    [req-4d211944-bd5c-4e88-a8df-c4a9167676e2
    36a7e60b4d134307b61e06949d33735e 9558f53959a04cc992c3f8b6d91bfb9f]
    [instance: 8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] Instance failed
    network setup
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] Traceback (most recent call
    last):
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]   File
    "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line
    571, in _allocate_network
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]
    requested_networks=requested_networks)
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]   File
    "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 178,
    in allocate_for_instance
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]     'args': args})
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]   File
    "/usr/lib/python2.7/dist-packages/nova/rpc/__init__.py", line 68,
    in call
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]     return
    _get_impl().call(context, topic, msg, timeout)
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]   File
    "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line
    674, in call
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]     return
    rpc_amqp.call(context, topic, msg, timeout, Connection.pool)
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]   File
    "/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line 343, in call
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]     rv = list(rv)
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]   File
    "/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line 304, in
    __iter__
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]     self.done()
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]   File
    "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]     self.gen.next()
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]   File
    "/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line 301, in
    __iter__
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] self._iterator.next()
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]   File
    "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line
    572, in iterconsume
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]     yield
    self.ensure(_error_callback, _consume)
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]   File
    "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line
    503, in ensure
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]     error_callback(e)
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]   File
    "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line
    553, in _error_callback
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]     raise rpc_common.Timeout()
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60] Timeout: Timeout while
    waiting on RPC response.
    2013-02-25 18:21:28 TRACE nova.compute.manager [instance:
    8c5b2d4a-c715-4f4d-8d74-a6bdccbe1b60]
    2013-02-25 18:21:28 ERROR nova.rpc.amqp
    [req-4d211944-bd5c-4e88-a8df-c4a9167676e2
    36a7e60b4d134307b61e06949d33735e 9558f53959a04cc992c3f8b6d91bfb9f]
    Exception during message handling
    2013-02-25 18:21:28 TRACE nova.rpc.amqp Traceback (most recent
    call last):
    2013-02-25 18:21:28 TRACE nova.rpc.amqp   File
    "/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line 253, in
    _process_data
    2013-02-25 18:21:28 TRACE nova.rpc.amqp     rval =
    node_func(context=ctxt, **node_args)
    2013-02-25 18:21:28 TRACE nova.rpc.amqp   File
    "/usr/lib/python2.7/dist-packages/nova/exception.py", line 114, in
    wrapped
    2013-02-25 18:21:28 TRACE nova.rpc.amqp     return f(*args, **kw)
    2013-02-25 18:21:28 TRACE nova.rpc.amqp   File
    "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line
    183, in decorated_function
    2013-02-25 18:21:28 TRACE nova.rpc.amqp     sys.exc_info())
    2013-02-25 18:21:28 TRACE nova.rpc.amqp   File
    "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
    2013-02-25 18:21:28 TRACE nova.rpc.amqp     self.gen.next()
    2013-02-25 18:21:28 TRACE nova.rpc.amqp   File
    "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line
    177, in decorated_function
    2013-02-25 18:21:28 TRACE nova.rpc.amqp     return function(self,
    context, instance_uuid, *args, **kwargs)
    2013-02-25 18:21:28 TRACE nova.rpc.amqp   File
    "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line
    663, in run_instance
    2013-02-25 18:21:28 TRACE nova.rpc.amqp do_run_instance()
    2013-02-25 18:21:28 TRACE nova.rpc.amqp   File
    "/usr/lib/python2.7/dist-packages/nova/utils.py", line 945, in inner
    2013-02-25 18:21:28 TRACE nova.rpc.amqp     retval = f(*args,
    **kwargs)
    2013-02-25 18:21:28 TRACE nova.rpc.amqp   File
    "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line
    662, in do_run_instance
    2013-02-25 18:21:28 TRACE nova.rpc.amqp
    self._run_instance(context, instance_uuid, **kwargs)
    2013-02-25 18:21:28 TRACE nova.rpc.amqp   File
    "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line
    463, in _run_instance
    2013-02-25 18:21:28 TRACE nova.rpc.amqp
    self._set_instance_error_state(context, instance_uuid)
    2013-02-25 18:21:28 TRACE nova.rpc.amqp   File
    "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
    2013-02-25 18:21:28 TRACE nova.rpc.amqp     self.gen.next()
    2013-02-25 18:21:28 TRACE nova.rpc.amqp   File
    "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line
    436, in _run_instance
    2013-02-25 18:21:28 TRACE nova.rpc.amqp requested_networks)
    2013-02-25 18:21:28 TRACE nova.rpc.amqp   File
    "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line
    571, in _allocate_network
    2013-02-25 18:21:28 TRACE nova.rpc.amqp
    requested_networks=requested_networks)
    2013-02-25 18:21:28 TRACE nova.rpc.amqp   File
    "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 178,
    in allocate_for_instance
    2013-02-25 18:21:28 TRACE nova.rpc.amqp     'args': args})
    2013-02-25 18:21:28 TRACE nova.rpc.amqp   File
    "/usr/lib/python2.7/dist-packages/nova/rpc/__init__.py", line 68,
    in call
    2013-02-25 18:21:28 TRACE nova.rpc.amqp     return
    _get_impl().call(context, topic, msg, timeout)
    2013-02-25 18:21:28 TRACE nova.rpc.amqp   File
    "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line
    674, in call
    2013-02-25 18:21:28 TRACE nova.rpc.amqp     return
    rpc_amqp.call(context, topic, msg, timeout, Connection.pool)
    2013-02-25 18:21:28 TRACE nova.rpc.amqp   File
    "/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line 343, in call
    2013-02-25 18:21:28 TRACE nova.rpc.amqp     rv = list(rv)
    2013-02-25 18:21:28 TRACE nova.rpc.amqp   File
    "/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line 304, in
    __iter__
    2013-02-25 18:21:28 TRACE nova.rpc.amqp     self.done()
    2013-02-25 18:21:28 TRACE nova.rpc.amqp   File
    "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
    2013-02-25 18:21:28 TRACE nova.rpc.amqp     self.gen.next()
    2013-02-25 18:21:28 TRACE nova.rpc.amqp   File
    "/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line 301, in
    __iter__
    2013-02-25 18:21:28 TRACE nova.rpc.amqp self._iterator.next()
    2013-02-25 18:21:28 TRACE nova.rpc.amqp   File
    "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line
    572, in iterconsume
    2013-02-25 18:21:28 TRACE nova.rpc.amqp     yield
    self.ensure(_error_callback, _consume)
    2013-02-25 18:21:28 TRACE nova.rpc.amqp   File
    "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line
    503, in ensure
    2013-02-25 18:21:28 TRACE nova.rpc.amqp error_callback(e)
    2013-02-25 18:21:28 TRACE nova.rpc.amqp   File
    "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line
    553, in _error_callback
    2013-02-25 18:21:28 TRACE nova.rpc.amqp     raise rpc_common.Timeout()
    2013-02-25 18:21:28 TRACE nova.rpc.amqp Timeout: Timeout while
    waiting on RPC response.
    2013-02-25 18:21:28 TRACE nova.rpc.amqp

    Thanks in advance,

    Javi

--Javier Álvarez Cid-Fuentes

    Grid Computing and Clusters Group
    Barcelona Supercomputing Center (BSC-CNS)
    Tel. (+34) 93 413 72 46


    WARNING / LEGAL TEXT: This message is intended only for the use of the
    individual or entity to which it is addressed and may contain
    information which is privileged, confidential, proprietary, or exempt
    from disclosure under applicable law. If you are not the intended
    recipient or the person responsible for delivering the message to the
    intended recipient, you are strictly prohibited from disclosing,
    distributing, copying, or in any way using this message. If you have
    received this communication in error, please notify the sender and
    destroy and delete any copies you may have received.

    http://www.bsc.es/disclaimer

    _______________________________________________
    Mailing list: https://launchpad.net/~openstack
    <https://launchpad.net/%7Eopenstack>
    Post to     : openstack@xxxxxxxxxxxxxxxxxxx
    <mailto:openstack@xxxxxxxxxxxxxxxxxxx>
    Unsubscribe : https://launchpad.net/~openstack
    <https://launchpad.net/%7Eopenstack>
    More help   : https://help.launchpad.net/ListHelp



--
Javier Álvarez Cid-Fuentes
Grid Computing and Clusters Group
Barcelona Supercomputing Center (BSC-CNS)
Tel. (+34) 93 413 72 46



WARNING / LEGAL TEXT: This message is intended only for the use of the
individual or entity to which it is addressed and may contain
information which is privileged, confidential, proprietary, or exempt
from disclosure under applicable law. If you are not the intended
recipient or the person responsible for delivering the message to the
intended recipient, you are strictly prohibited from disclosing,
distributing, copying, or in any way using this message. If you have
received this communication in error, please notify the sender and
destroy and delete any copies you may have received.

http://www.bsc.es/disclaimer

Follow ups

Re: VM creation failure
From: Unmesh Gurjar, 2013-02-26

References

VM creation failure
From: Javier Alvarez, 2013-02-25
Re: VM creation failure
From: JuanFra Rodriguez Cardoso, 2013-02-25