yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1389728] [NEW] cinder do not import image from glance

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Vitalii <vb@xxxxxxxxxxx>
Date: Wed, 05 Nov 2014 14:49:40 -0000
Reply-to: Bug 1389728 <1389728@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

Steps to reproduce:

1. Create raw bare glance image
2. Create LVM volume group
3. Boot instance with nova:

nova --debug boot --flavor m1.small --block-device
source=image,id=<GLANCE IMAGE
ID>,dest=volume,size=10,shutdown=preserve,bootindex=0 --nic net-
id=<NEUTRON NET ID> test

In cinder volume log I can see the following:

2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     incoming.message))
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     return self._do_dispatch(endpoint, method, ctxt, args)
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     result = getattr(endpoint, method)(ctxt, **new_args)
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/osprofiler/profiler.py", line 105, in wrapper
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     return f(*args, **kwargs)
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/cinder/volume/manager.py", line 381, in create_volume
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     _run_flow()
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/cinder/volume/manager.py", line 374, in _run_flow
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     flow_engine.run()
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 99, in run
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     for _state in self.run_iter():
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 156, in run_iter
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     misc.Failure.reraise_if_any(failures.values())
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/taskflow/utils/misc.py", line 733, in reraise_if_any
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     failures[0].reraise()
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/taskflow/utils/misc.py", line 740, in reraise
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     six.reraise(*self._exc_info)
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 35, in _execute_task
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     result = task.execute(**arguments)
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 638, in execute
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     **volume_spec)
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 590, in _create_from_image
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     image_id, image_location, image_service)
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 492, in _copy_image_to_volume
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     raise exception.ImageCopyFailure(reason=ex.stderr)
2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher ImageCopyFailure: Failed to copy image to volume: qemu-img: error writing zeroes at sector 0: Invalid argument

The reason is this code:
cinder/image/image_utils.py:86

Several lines above, in the same function ther's:
   cmd = ('qemu-img', 'convert',
           '-O', out_format, source, dest)

which works. The following one does not work:

        cmd = ('qemu-img', 'convert',
               '-t', 'none',
               '-O', out_format, source, dest)

Example: """
sudo qemu-img convert -t none -O raw /home/vb/var/lib/cinder/conversion/tmpve1XXC /dev/mapper/vg_main-volume--0e201cdb--47b4--4c5d--9621--d6fde3bb901a
qemu-img: error writing zeroes at sector 0: Invalid argument
"""

I have qemu-utils 2.1 from Debian.
I use Juno release.

P.S.
I think it is very dumb  to copy image from glance to temporary file and then copying it to volume.
I read inline comments why it was done, but still, you could have came up with workaround.

** Affects: cinder
     Importance: Undecided
         Status: New

** Project changed: neutron => cinder

** Description changed:

  Steps to reproduce:
  
  1. Create raw bare glance image
  2. Create LVM volume group
- 3. Boot instance with nova: 
+ 3. Boot instance with nova:
  
  nova --debug boot --flavor m1.small --block-device
  source=image,id=<GLANCE IMAGE
  ID>,dest=volume,size=10,shutdown=preserve,bootindex=0 --nic net-
  id=<NEUTRON NET ID> test
  
  In cinder volume log I can see the following:
  
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     incoming.message))
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     return self._do_dispatch(endpoint, method, ctxt, args)
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     result = getattr(endpoint, method)(ctxt, **new_args)
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/osprofiler/profiler.py", line 105, in wrapper
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     return f(*args, **kwargs)
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/cinder/volume/manager.py", line 381, in create_volume
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     _run_flow()
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/cinder/volume/manager.py", line 374, in _run_flow
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     flow_engine.run()
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 99, in run
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     for _state in self.run_iter():
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 156, in run_iter
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     misc.Failure.reraise_if_any(failures.values())
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/taskflow/utils/misc.py", line 733, in reraise_if_any
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     failures[0].reraise()
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/taskflow/utils/misc.py", line 740, in reraise
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     six.reraise(*self._exc_info)
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 35, in _execute_task
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     result = task.execute(**arguments)
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 638, in execute
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     **volume_spec)
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 590, in _create_from_image
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     image_id, image_location, image_service)
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 492, in _copy_image_to_volume
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     raise exception.ImageCopyFailure(reason=ex.stderr)
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher ImageCopyFailure: Failed to copy image to volume: qemu-img: error writing zeroes at sector 0: Invalid argument
  
- 
  The reason is this code:
  cinder/image/image_utils.py:86
  
  Several lines above, in the same function ther's:
-    cmd = ('qemu-img', 'convert',
-            '-O', out_format, source, dest)
+    cmd = ('qemu-img', 'convert',
+            '-O', out_format, source, dest)
  
  which works. The following one does not work:
  
-         cmd = ('qemu-img', 'convert',
-                '-t', 'none',
-                '-O', out_format, source, dest)
+         cmd = ('qemu-img', 'convert',
+                '-t', 'none',
+                '-O', out_format, source, dest)
  
  Example: """
- sudo qemu-img convert -t none -O raw /home/vb/var/lib/cinder/conversion/tmpve1XXC /dev/mapper/vg_main-volume--0e201cdb--47b4--4c5d--9621--d6fde3bb901a                                             
+ sudo qemu-img convert -t none -O raw /home/vb/var/lib/cinder/conversion/tmpve1XXC /dev/mapper/vg_main-volume--0e201cdb--47b4--4c5d--9621--d6fde3bb901a
  qemu-img: error writing zeroes at sector 0: Invalid argument
  """
  
- 
  I have qemu-utils 2.1 from Debian.
+ I use Juno release.
  
  P.S.
  I think it is very dumb  to copy image from glance to temporary file and then copying it to volume.
  I read inline comments why it was done, but still, you could have came up with workaround.

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1389728

Title:
  cinder do not import image from glance

Status in Cinder:
  New

Bug description:
  Steps to reproduce:

  1. Create raw bare glance image
  2. Create LVM volume group
  3. Boot instance with nova:

  nova --debug boot --flavor m1.small --block-device
  source=image,id=<GLANCE IMAGE
  ID>,dest=volume,size=10,shutdown=preserve,bootindex=0 --nic net-
  id=<NEUTRON NET ID> test

  In cinder volume log I can see the following:

  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     incoming.message))
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     return self._do_dispatch(endpoint, method, ctxt, args)
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     result = getattr(endpoint, method)(ctxt, **new_args)
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/osprofiler/profiler.py", line 105, in wrapper
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     return f(*args, **kwargs)
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/cinder/volume/manager.py", line 381, in create_volume
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     _run_flow()
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/cinder/volume/manager.py", line 374, in _run_flow
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     flow_engine.run()
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 99, in run
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     for _state in self.run_iter():
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 156, in run_iter
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     misc.Failure.reraise_if_any(failures.values())
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/taskflow/utils/misc.py", line 733, in reraise_if_any
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     failures[0].reraise()
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/taskflow/utils/misc.py", line 740, in reraise
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     six.reraise(*self._exc_info)
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 35, in _execute_task
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     result = task.execute(**arguments)
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 638, in execute
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     **volume_spec)
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 590, in _create_from_image
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     image_id, image_location, image_service)
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher   File "/home/vb/.virtualenvs/ecs/local/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 492, in _copy_image_to_volume
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher     raise exception.ImageCopyFailure(reason=ex.stderr)
  2014-11-05 14:30:45.271 17750 TRACE oslo.messaging.rpc.dispatcher ImageCopyFailure: Failed to copy image to volume: qemu-img: error writing zeroes at sector 0: Invalid argument

  The reason is this code:
  cinder/image/image_utils.py:86

  Several lines above, in the same function ther's:
     cmd = ('qemu-img', 'convert',
             '-O', out_format, source, dest)

  which works. The following one does not work:

          cmd = ('qemu-img', 'convert',
                 '-t', 'none',
                 '-O', out_format, source, dest)

  Example: """
  sudo qemu-img convert -t none -O raw /home/vb/var/lib/cinder/conversion/tmpve1XXC /dev/mapper/vg_main-volume--0e201cdb--47b4--4c5d--9621--d6fde3bb901a
  qemu-img: error writing zeroes at sector 0: Invalid argument
  """

  I have qemu-utils 2.1 from Debian.
  I use Juno release.

  P.S.
  I think it is very dumb  to copy image from glance to temporary file and then copying it to volume.
  I read inline comments why it was done, but still, you could have came up with workaround.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1389728/+subscriptions
Follow ups

[Bug 1389728] [NEW] cinder do not import image from glance
From: Vitalii, 2014-11-05
References

[Bug 1389728] [NEW] cinder do not import image from glance
From: Vitalii, 2014-11-05