← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #66247

[Bug 1707284] [NEW] Extend attached volume fails with "VolumePathsNotFound: Could not find any paths for the volume." in os-brick iscsi connector

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

Seen here:

http://logs.openstack.org/78/480778/2/check/gate-tempest-dsvm-neutron-
full-ubuntu-
xenial/999910b/logs/screen-n-cpu.txt.gz?level=TRACE#_Jul_27_07_06_43_460444

Jul 27 07:06:43.460444 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server [req-3e8818f0-952a-4242-a294-0319bae0121a req-b68879c5-8886-4a58-ae10-7a3804868df7 service nova] Exception during message handling: VolumePathsNotFound: Could not find any paths for the volume.
Jul 27 07:06:43.460651 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
Jul 27 07:06:43.460831 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
Jul 27 07:06:43.461013 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
Jul 27 07:06:43.461314 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
Jul 27 07:06:43.461483 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
Jul 27 07:06:43.461653 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
Jul 27 07:06:43.461816 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
Jul 27 07:06:43.461989 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/exception_wrapper.py", line 76, in wrapped
Jul 27 07:06:43.462153 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     function_name, call_dict, binary)
Jul 27 07:06:43.462324 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
Jul 27 07:06:43.462484 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     self.force_reraise()
Jul 27 07:06:43.462653 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
Jul 27 07:06:43.462818 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
Jul 27 07:06:43.462988 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/exception_wrapper.py", line 67, in wrapped
Jul 27 07:06:43.463151 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     return f(self, context, *args, **kw)
Jul 27 07:06:43.463373 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/compute/manager.py", line 6907, in external_instance_event
Jul 27 07:06:43.463543 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     self.extend_volume(context, instance, event.tag)
Jul 27 07:06:43.463800 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/compute/utils.py", line 864, in decorated_function
Jul 27 07:06:43.463993 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
Jul 27 07:06:43.464156 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/compute/manager.py", line 211, in decorated_function
Jul 27 07:06:43.464316 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     kwargs['instance'], e, sys.exc_info())
Jul 27 07:06:43.464476 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
Jul 27 07:06:43.464633 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     self.force_reraise()
Jul 27 07:06:43.464793 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
Jul 27 07:06:43.464963 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
Jul 27 07:06:43.465122 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/compute/manager.py", line 199, in decorated_function
Jul 27 07:06:43.465293 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
Jul 27 07:06:43.465460 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/compute/manager.py", line 6868, in extend_volume
Jul 27 07:06:43.465625 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     instance)
Jul 27 07:06:43.465783 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1412, in extend_volume
Jul 27 07:06:43.465948 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     new_size = self._extend_volume(connection_info, instance)
Jul 27 07:06:43.466118 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1173, in _extend_volume
Jul 27 07:06:43.466278 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     return vol_driver.extend_volume(connection_info, instance)
Jul 27 07:06:43.467246 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/virt/libvirt/volume/iscsi.py", line 85, in extend_volume
Jul 27 07:06:43.467453 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     new_size = self.connector.extend_volume(connection_info['data'])
Jul 27 07:06:43.467708 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/os_brick/utils.py", line 150, in trace_logging_wrapper
Jul 27 07:06:43.467884 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     result = f(*args, **kwargs)
Jul 27 07:06:43.468070 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 271, in inner
Jul 27 07:06:43.468249 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     return f(*args, **kwargs)
Jul 27 07:06:43.468418 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/os_brick/initiator/connectors/iscsi.py", line 472, in extend_volume
Jul 27 07:06:43.468581 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     raise exception.VolumePathsNotFound()
Jul 27 07:06:43.468741 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server VolumePathsNotFound: Could not find any paths for the volume.
Jul 27 07:06:43.468910 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server

** Affects: nova
     Importance: Medium
         Status: Confirmed


** Tags: extend iscsi volumes

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1707284

Title:
  Extend attached volume fails with "VolumePathsNotFound: Could not find
  any paths for the volume." in os-brick iscsi connector

Status in OpenStack Compute (nova):
  Confirmed

Bug description:
  Seen here:

  http://logs.openstack.org/78/480778/2/check/gate-tempest-dsvm-neutron-
  full-ubuntu-
  xenial/999910b/logs/screen-n-cpu.txt.gz?level=TRACE#_Jul_27_07_06_43_460444

  Jul 27 07:06:43.460444 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server [req-3e8818f0-952a-4242-a294-0319bae0121a req-b68879c5-8886-4a58-ae10-7a3804868df7 service nova] Exception during message handling: VolumePathsNotFound: Could not find any paths for the volume.
  Jul 27 07:06:43.460651 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
  Jul 27 07:06:43.460831 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
  Jul 27 07:06:43.461013 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
  Jul 27 07:06:43.461314 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
  Jul 27 07:06:43.461483 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
  Jul 27 07:06:43.461653 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
  Jul 27 07:06:43.461816 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
  Jul 27 07:06:43.461989 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/exception_wrapper.py", line 76, in wrapped
  Jul 27 07:06:43.462153 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     function_name, call_dict, binary)
  Jul 27 07:06:43.462324 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  Jul 27 07:06:43.462484 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     self.force_reraise()
  Jul 27 07:06:43.462653 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  Jul 27 07:06:43.462818 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  Jul 27 07:06:43.462988 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/exception_wrapper.py", line 67, in wrapped
  Jul 27 07:06:43.463151 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     return f(self, context, *args, **kw)
  Jul 27 07:06:43.463373 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/compute/manager.py", line 6907, in external_instance_event
  Jul 27 07:06:43.463543 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     self.extend_volume(context, instance, event.tag)
  Jul 27 07:06:43.463800 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/compute/utils.py", line 864, in decorated_function
  Jul 27 07:06:43.463993 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
  Jul 27 07:06:43.464156 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/compute/manager.py", line 211, in decorated_function
  Jul 27 07:06:43.464316 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     kwargs['instance'], e, sys.exc_info())
  Jul 27 07:06:43.464476 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  Jul 27 07:06:43.464633 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     self.force_reraise()
  Jul 27 07:06:43.464793 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  Jul 27 07:06:43.464963 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  Jul 27 07:06:43.465122 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/compute/manager.py", line 199, in decorated_function
  Jul 27 07:06:43.465293 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
  Jul 27 07:06:43.465460 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/compute/manager.py", line 6868, in extend_volume
  Jul 27 07:06:43.465625 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     instance)
  Jul 27 07:06:43.465783 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1412, in extend_volume
  Jul 27 07:06:43.465948 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     new_size = self._extend_volume(connection_info, instance)
  Jul 27 07:06:43.466118 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1173, in _extend_volume
  Jul 27 07:06:43.466278 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     return vol_driver.extend_volume(connection_info, instance)
  Jul 27 07:06:43.467246 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/virt/libvirt/volume/iscsi.py", line 85, in extend_volume
  Jul 27 07:06:43.467453 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     new_size = self.connector.extend_volume(connection_info['data'])
  Jul 27 07:06:43.467708 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/os_brick/utils.py", line 150, in trace_logging_wrapper
  Jul 27 07:06:43.467884 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     result = f(*args, **kwargs)
  Jul 27 07:06:43.468070 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 271, in inner
  Jul 27 07:06:43.468249 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     return f(*args, **kwargs)
  Jul 27 07:06:43.468418 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/os_brick/initiator/connectors/iscsi.py", line 472, in extend_volume
  Jul 27 07:06:43.468581 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server     raise exception.VolumePathsNotFound()
  Jul 27 07:06:43.468741 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server VolumePathsNotFound: Could not find any paths for the volume.
  Jul 27 07:06:43.468910 ubuntu-xenial-citycloud-la1-10111613 nova-compute[18654]: ERROR oslo_messaging.rpc.server

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1707284/+subscriptions
  Thread Previous • Date Previous • • Thread Next