← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #38592

[Bug 1496774] [NEW] metadata-agent fills up log-files with " TypeError: 'NoneType' object has no attribute '__getitem__'"

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

Sience Monday, the log file of metadata-agent are getting full with the
following error / trace:

2015-09-17 11:06:00.187 8277 ERROR oslo_messaging._drivers.impl_rabbit [-] Failed to consume message from queue: 'NoneType' object has no attribute '__getitem__'
2015-09-17 11:06:00.187 8276 ERROR oslo_messaging._drivers.impl_rabbit [-] Failed to consume message from queue: 'NoneType' object has no attribute '__getitem__'
2015-09-17 11:06:00.186 8267 ERROR oslo_messaging._drivers.amqpdriver [-] Failed to process incoming message, retrying...
2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver Traceback (most recent call last):
2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 197, in poll
2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     self.conn.consume(limit=1)
2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 1172, in consume
2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     six.next(it)
2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 1083, in iterconsume
2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     error_callback=_error_callback)
2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 870, in ensure
2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     ret, channel = autoretry_method()
2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 453, in _ensured
2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     return fun(*args, **kwargs)
2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 520, in __call__^C
2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     self.revive(create_channel())
2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 251, in channel
2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     chan = self.transport.create_channel(self.connection)^C
2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 91, in create_channel
2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     return connection.channel()
2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 279, in channel
2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     return self.channels[channel_id]
2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver TypeError: 'NoneType' object has no attribute '__getitem__'
2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver

There was no change in the configuration before, i'm not exactly sure
why it does that, the configuration looks fine and all good.

I have also a 100% load on the server.

** Affects: neutron
     Importance: Undecided
         Status: New


** Tags: metadata typeerror

** Description changed:

  Sience Monday, the log file of metadata-agent are getting full with the
  following error / trace:
  
  2015-09-17 11:06:00.187 8277 ERROR oslo_messaging._drivers.impl_rabbit [-] Failed to consume message from queue: 'NoneType' object has no attribute '__getitem__'
  2015-09-17 11:06:00.187 8276 ERROR oslo_messaging._drivers.impl_rabbit [-] Failed to consume message from queue: 'NoneType' object has no attribute '__getitem__'
  2015-09-17 11:06:00.186 8267 ERROR oslo_messaging._drivers.amqpdriver [-] Failed to process incoming message, retrying...
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver Traceback (most recent call last):
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 197, in poll
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     self.conn.consume(limit=1)
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 1172, in consume
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     six.next(it)
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 1083, in iterconsume
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     error_callback=_error_callback)
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 870, in ensure
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     ret, channel = autoretry_method()
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 453, in _ensured
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     return fun(*args, **kwargs)
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 520, in __call__^C
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     self.revive(create_channel())
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 251, in channel
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     chan = self.transport.create_channel(self.connection)^C
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 91, in create_channel
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     return connection.channel()
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 279, in channel
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     return self.channels[channel_id]
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver TypeError: 'NoneType' object has no attribute '__getitem__'
- 2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver 
+ 2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver
  
+ There was no change in the configuration before, i'm not exactly sure
+ why it does that, the configuration looks fine and all good.
  
- There was no change in the configuration before, i'm not exactly sure why it does that, the configuration looks fine and all good.
+ I have also a 100% load on the server.

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1496774

Title:
  metadata-agent fills up log-files with " TypeError: 'NoneType' object
  has no attribute '__getitem__'"

Status in neutron:
  New

Bug description:
  Sience Monday, the log file of metadata-agent are getting full with
  the following error / trace:

  2015-09-17 11:06:00.187 8277 ERROR oslo_messaging._drivers.impl_rabbit [-] Failed to consume message from queue: 'NoneType' object has no attribute '__getitem__'
  2015-09-17 11:06:00.187 8276 ERROR oslo_messaging._drivers.impl_rabbit [-] Failed to consume message from queue: 'NoneType' object has no attribute '__getitem__'
  2015-09-17 11:06:00.186 8267 ERROR oslo_messaging._drivers.amqpdriver [-] Failed to process incoming message, retrying...
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver Traceback (most recent call last):
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 197, in poll
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     self.conn.consume(limit=1)
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 1172, in consume
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     six.next(it)
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 1083, in iterconsume
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     error_callback=_error_callback)
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 870, in ensure
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     ret, channel = autoretry_method()
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 453, in _ensured
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     return fun(*args, **kwargs)
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 520, in __call__^C
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     self.revive(create_channel())
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 251, in channel
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     chan = self.transport.create_channel(self.connection)^C
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 91, in create_channel
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     return connection.channel()
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver   File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 279, in channel
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver     return self.channels[channel_id]
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver TypeError: 'NoneType' object has no attribute '__getitem__'
  2015-09-17 11:06:00.186 8267 TRACE oslo_messaging._drivers.amqpdriver

  There was no change in the configuration before, i'm not exactly sure
  why it does that, the configuration looks fine and all good.

  I have also a 100% load on the server.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1496774/+subscriptions

Follow ups

  Thread Previous • Date Previous • • Thread Next