← Back to team overview

openstack team mailing list archive

Re: [Nova] Got 'qemu-img: error while writing sector 634880: Input/output error' in _create_image

 

Thanks for your reminders, I found the reason now, it's my own fault!
I tried to optimize the snapshot_instance process like this blueprint:
https://blueprints.launchpad.net/glance/+spec/hierarchical-images
and I copied the call_if_no_exists & copy_and_extend methods, and I forgot the locks @utils.synchronized!
I think it's really OK now.

What a stupid mistake!

2012-09-26



Wangpan



发件人:Boris Filippov
发送时间:2012-09-26 03:25
主题:Re: Re: [Openstack] [Nova] Got 'qemu-img: error while writing sector 634880: Input/output error' in _create_image
收件人:"Wangpan"<hzwangpan@xxxxxxxxxxxxxxxx>
抄送:"Razique Mahroua"<razique.mahroua@xxxxxxxxx>,"Scott Moser"<smoser@xxxxxxxxxx>,"openstack"<openstack@xxxxxxxxxxxxxxxxxxx>

Actually, this is unlikely because there was locking for image fetching already. This patch only adds it for case when several compute processes share same $instances_dir. 
- disk full really might be the case. Simple check is to extend available space and try to reproduce it again.


2012/9/25 Wangpan <hzwangpan@xxxxxxxxxxxxxxxx>

Thanks for your clue.
yes, the occured condition of my issue is the same as BUG 1014227, but the difference is that my instance directory is NOT shared between compute nodes,
the instances' images are saved in local disk on compute nodes.
I believe the bug Boris found may occur in NOT shared storage circumstance as mine, and my issue may be resloved by this patch which adds a lock to cache image method.
Thanks again.


2012-09-25



Wangpan



发件人:Boris Filippov
发送时间:2012-09-24 19:59
主题:Re: [Openstack] [Nova] Got 'qemu-img: error while writing sector 634880: Input/output error' in _create_image
收件人:"Razique Mahroua"<razique.mahroua@xxxxxxxxx>
抄送:"Wangpan"<hzwangpan@xxxxxxxxxxxxxxxx>,"openstack"<openstack@xxxxxxxxxxxxxxxxxxx>

Your instance directory is shared between compute nodes?
Similar issue was fixed for folsom-rc1 
https://review.openstack.org/#/c/12024/

https://bugs.launchpad.net/nova/+bug/1014227


2012/9/24 Razique Mahroua <razique.mahroua@xxxxxxxxx>

Here is what I found 
http://comments.gmane.org/gmane.comp.emulators.qemu/153254
http://osdir.com/ml/qemu-devel/2012-05/msg04234.html


What do you think ?


Nuage & Co - Razique Mahroua  
razique.mahroua@xxxxxxxxx

 


Le 24 sept. 2012 à 10:43, Wangpan <hzwangpan@xxxxxxxxxxxxxxxx> a écrit :


qemu-img check -f qcow2 test.qcow2
No errors were found on the image.

2012-09-24



Wangpan



发件人:Razique Mahroua
发送时间:2012-09-24 16:30
主题:Re: Re: [Openstack] [Nova] Got 'qemu-img: error while writing sector 634880: Input/output error' in _create_image
收件人:"Wangpan"<hzwangpan@xxxxxxxxxxxxxxxx>
抄送:"openstack"<openstack@xxxxxxxxxxxxxxxxxxx>

hi,  
what about a 
qemu-img check -f qcow2 test.qcow2




Nuage & Co - Razique Mahroua 
razique.mahroua@xxxxxxxxx

<NUAGECO-LOGO-Fblan_petit(1).jpg>


Le 24 sept. 2012 à 10:19, "Wangpan"<hzwangpan@xxxxxxxxxxxxxxxx> a écrit :


Thanks for your attention!

- disk full  ----# I believe there is enough space, because I have tried to boot serveral instances after this error with the same image, they are all booted successfully.
- permission issues    ----# Not this reason, too. explain as above.

I still believe this is a race problem, if we convert to image from qcow2 to raw at a short interval, one of the convertion may fail, 
I have verified this by running two same commands continuously such as:
qemu-img convert -f qcow2 -O raw  test.qcow2 test.part &
and one of them give me the same error:
qemu-img: error while writing sector 1052672: Input/output error



_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to     : openstack@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Follow ups

References