openstack team mailing list archive

Thread
Date
A computte node error IOError: Socket closed,help me

To: openstack@xxxxxxxxxxxxxxxxxxx
From: 王鹏 <breakwindwp@xxxxxxxxx>
Date: Wed, 8 Aug 2012 13:01:46 +0800
Hi，everyone：
      I add a compute node(install ntp&nova-compute only,copy config from
controller),use nova-manage service list is ok.
     But,when I create a instance in compute node.
     It show instance error.
     I find error from /var/log/nova/nova/nova-scheduler.log,as follow:
2012-08-08 10:57:07 ERROR nova.rpc.common [-] Failed to consume message
from queue: Socket closed
2012-08-08 10:57:07 TRACE nova.rpc.common Traceback (most recent call last):
2012-08-08 10:57:07 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 490, in
ensure
2012-08-08 10:57:07 TRACE nova.rpc.common return method(*args, **kwargs)
2012-08-08 10:57:07 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 567, in
_consume
2012-08-08 10:57:07 TRACE nova.rpc.common return
self.connection.drain_events(timeout=timeout)
2012-08-08 10:57:07 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/kombu/connection.py", line 175, in
drain_events
2012-08-08 10:57:07 TRACE nova.rpc.common return
self.transport.drain_events(self.connection, **kwargs)
2012-08-08 10:57:07 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 238,
in drain_events
2012-08-08 10:57:07 TRACE nova.rpc.common return
connection.drain_events(**kwargs)
2012-08-08 10:57:07 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 57,
in drain_events
2012-08-08 10:57:07 TRACE nova.rpc.common return
self.wait_multi(self.channels.values(), timeout=timeout)
2012-08-08 10:57:07 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 63,
in wait_multi
2012-08-08 10:57:07 TRACE nova.rpc.common chanmap.keys(), allowed_methods,
timeout=timeout)
2012-08-08 10:57:07 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 120,
in _wait_multiple
2012-08-08 10:57:07 TRACE nova.rpc.common channel, method_sig, args,
content = read_timeout(timeout)
2012-08-08 10:57:07 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 88,
in read_timeout
2012-08-08 10:57:07 TRACE nova.rpc.common return
self.method_reader.read_method()
2012-08-08 10:57:07 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/amqplib/client_0_8/method_framing.py",
line 221, in read_method
2012-08-08 10:57:07 TRACE nova.rpc.common raise m
2012-08-08 10:57:07 TRACE nova.rpc.common IOError: Socket closed
2012-08-08 10:57:07 TRACE nova.rpc.common
2012-08-08 10:57:07 INFO nova.rpc.common [-] Reconnecting to AMQP server on
172.18.32.7:5672
2012-08-08 10:57:07 ERROR nova.rpc.common [-] AMQP server on
172.18.32.7:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in
1 seconds.
2012-08-08 10:57:07 TRACE nova.rpc.common Traceback (most recent call last):
2012-08-08 10:57:07 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 446, in
reconnect
2012-08-08 10:57:07 TRACE nova.rpc.common self._connect()
2012-08-08 10:57:07 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 423, in
_connect
2012-08-08 10:57:07 TRACE nova.rpc.common self.connection.connect()
2012-08-08 10:57:07 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/kombu/connection.py", line 154, in connect
2012-08-08 10:57:07 TRACE nova.rpc.common return self.connection
2012-08-08 10:57:07 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/kombu/connection.py", line 560, in
connection
2012-08-08 10:57:07 TRACE nova.rpc.common self._connection =
self._establish_connection()
2012-08-08 10:57:07 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/kombu/connection.py", line 521, in
_establish_connection
2012-08-08 10:57:07 TRACE nova.rpc.common conn =
self.transport.establish_connection()
2012-08-08 10:57:07 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 255,
in establish_connection
2012-08-08 10:57:07 TRACE nova.rpc.common
connect_timeout=conninfo.connect_timeout)
2012-08-08 10:57:07 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", line 52,
in __init__
2012-08-08 10:57:07 TRACE nova.rpc.common super(Connection,
self).__init__(*args, **kwargs)
2012-08-08 10:57:07 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/amqplib/client_0_8/connection.py", line
129, in __init__
2012-08-08 10:57:07 TRACE nova.rpc.common self.transport =
create_transport(host, connect_timeout, ssl)
2012-08-08 10:57:07 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/amqplib/client_0_8/transport.py", line
281, in create_transport
2012-08-08 10:57:07 TRACE nova.rpc.common return TCPTransport(host,
connect_timeout)
2012-08-08 10:57:07 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/amqplib/client_0_8/transport.py", line
85, in __init__
2012-08-08 10:57:07 TRACE nova.rpc.common raise socket.error, msg
2012-08-08 10:57:07 TRACE nova.rpc.common error: [Errno 111] ECONNREFUSED
2012-08-08 10:57:07 TRACE nova.rpc.common
2012-08-08 10:57:10 AUDIT nova.service [-] Starting scheduler node (version
2012.1-LOCALBRANCH:LOCALREVISION)
2012-08-08 10:57:10 INFO nova.rpc.common
[req-5ae8f2f4-6e4d-49d7-968c-e0a2f4547152 None None] Connected to AMQP
server on 172.18.32.7:5672
2012-08-08 11:00:11 INFO nova.rpc.common
[req-7ef3bced-fc4e-4812-a96c-4c32a37ea047 6ed9496e07724c969e1e470e9ea4621e
f4c42d279f124477a487c57f2a96d2df] Connected to AMQP server on
172.18.32.7:5672
2012-08-08 11:16:43 ERROR nova.rpc.common [-] Failed to consume message
from queue: Socket closed
2012-08-08 11:16:43 TRACE nova.rpc.common Traceback (most recent call last):
2012-08-08 11:16:43 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 490, in
ensure
2012-08-08 11:16:43 TRACE nova.rpc.common return method(*args, **kwargs)
2012-08-08 11:16:43 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 567, in
_consume
:$
2012-08-08 12:43:21 TRACE nova.rpc.common super(Connection,
self).__init__(*args, **kwargs)
2012-08-08 12:43:21 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/amqplib/client_0_8/connection.py", line
129, in __init__
2012-08-08 12:43:21 TRACE nova.rpc.common self.transport =
create_transport(host, connect_timeout, ssl)
2012-08-08 12:43:21 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/amqplib/client_0_8/transport.py", line
281, in create_transport
2012-08-08 12:43:21 TRACE nova.rpc.common return TCPTransport(host,
connect_timeout)
2012-08-08 12:43:21 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/amqplib/client_0_8/transport.py", line
85, in __init__
2012-08-08 12:43:21 TRACE nova.rpc.common raise socket.error, msg
2012-08-08 12:43:21 TRACE nova.rpc.common error: [Errno 111] ECONNREFUSED
2012-08-08 12:43:21 TRACE nova.rpc.common
2012-08-08 12:43:24 INFO nova.rpc.common [-] Reconnecting to AMQP server on
172.18.32.7:5672
2012-08-08 12:43:24 INFO nova.rpc.common [-] Connected to AMQP server on
172.18.32.7:5672
2012-08-08 12:44:25 ERROR nova.rpc.common
[req-8e9ddc98-d9a3-43b0-9fe8-d376243fc79b 6ed9496e07724c969e1e470e9ea4621e
f4c42d279f124477a487c57f2a96d2df] Failed to publish message to topic
'compute.keystone': Socket closed
2012-08-08 12:44:25 TRACE nova.rpc.common Traceback (most recent call last):
2012-08-08 12:44:25 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 490, in
ensure
2012-08-08 12:44:25 TRACE nova.rpc.common return method(*args, **kwargs)
2012-08-08 12:44:25 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 593, in
_publish
2012-08-08 12:44:25 TRACE nova.rpc.common publisher = cls(self.channel,
topic, **kwargs)
2012-08-08 12:44:25 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 287, in
__init__
2012-08-08 12:44:25 TRACE nova.rpc.common **options)
2012-08-08 12:44:25 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 239, in
__init__
2012-08-08 12:44:25 TRACE nova.rpc.common self.reconnect(channel)
2012-08-08 12:44:25 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 246, in
reconnect
2012-08-08 12:44:25 TRACE nova.rpc.common channel=channel,
routing_key=self.routing_key)
2012-08-08 12:44:25 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/kombu/messaging.py", line 85, in __init__
2012-08-08 12:44:25 TRACE nova.rpc.common self.declare()
2012-08-08 12:44:25 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/kombu/messaging.py", line 98, in declare
2012-08-08 12:44:25 TRACE nova.rpc.common self.exchange.declare()
2012-08-08 12:44:25 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/kombu/entity.py", line 154, in declare
2012-08-08 12:44:25 TRACE nova.rpc.common nowait=nowait)
2012-08-08 12:44:25 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/kombu/syn.py", line 23, in blocking
2012-08-08 12:44:25 TRACE nova.rpc.common return __sync_current(fun, *args,
**kwargs)
2012-08-08 12:44:25 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/kombu/syn.py", line 49, in __eblocking__
2012-08-08 12:44:25 TRACE nova.rpc.common return spawn(fun, *args,
**kwargs).wait()
2012-08-08 12:44:25 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 166, in
wait
2012-08-08 12:44:25 TRACE nova.rpc.common return self._exit_event.wait()
2012-08-08 12:44:25 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait
2012-08-08 12:44:25 TRACE nova.rpc.common return hubs.get_hub().switch()
2012-08-08 12:44:25 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 177, in switch
2012-08-08 12:44:25 TRACE nova.rpc.common return self.greenlet.switch()
2012-08-08 12:44:25 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 192, in
main
2012-08-08 12:44:25 TRACE nova.rpc.common result = function(*args, **kwargs)
2012-08-08 12:44:25 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/amqplib/client_0_8/channel.py", line 843,
in exchange_declare
2012-08-08 12:44:25 TRACE nova.rpc.common (40, 11), #
Channel.exchange_declare_ok
2012-08-08 12:44:25 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/amqplib/client_0_8/abstract_channel.py",
line 95, in wait
2012-08-08 12:44:25 TRACE nova.rpc.common self.channel_id, allowed_methods)
2012-08-08 12:44:25 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/amqplib/client_0_8/connection.py", line
202, in _wait_method
2012-08-08 12:44:25 TRACE nova.rpc.common self.method_reader.read_method()
2012-08-08 12:44:25 TRACE nova.rpc.common File
"/usr/lib/python2.7/dist-packages/amqplib/client_0_8/method_framing.py",
line 221, in read_method
2012-08-08 12:44:25 TRACE nova.rpc.common raise m
2012-08-08 12:44:25 TRACE nova.rpc.common IOError: Socket closed

please help me!
thank you
Follow ups

Re: A computte node error IOError: Socket closed, help me
From: Qiu Zhigang, 2012-08-08