openstack team mailing list archive

[John Paul Walters <jwalters@xxxxxxx>]

Hi Razique,

Thanks for chiming in.  Yes, nova owns the instances directory that it's writing to.  In fact, between the multi-node volume and the single node volume, I gave the same permissions: created a directory "instances" on the gluster volume, and chown nova.nova instances.  The individual instance directories get created whenever I try to launch an instance, and the permissions all seem okay to me:

Here are the permissions on the gluster volume:
[root@openstack-13 instances]# ls -al
total 29
drwxr-xr-x.  4 nova nova   234 Apr 11 14:20 .
drwxr-xr-x.  3 root root  4096 Apr 10 15:52 ..
drwxr-x---. 11 nova nova 24576 Apr 11 14:31 instances  

Inside of instances:
[root@openstack-13 instances]# ls -al
total 33
drwxr-x---. 11 nova nova 24576 Apr 11 14:31 .
drwxr-xr-x.  4 nova nova   234 Apr 11 14:20 ..
drwxr-xr-x.  2 nova nova  8302 Apr 11 14:21 _base
drwxr-xr-x.  2 nova nova   110 Apr 11 14:21 instance-0000023b
drwxr-xr-x.  2 nova nova   110 Apr 11 14:22 instance-0000023c
drwxr-xr-x.  2 nova nova   110 Apr 11 14:22 instance-0000023d
drwxr-xr-x.  2 nova nova   110 Apr 11 14:22 instance-0000023e
drwxr-xr-x.  2 nova nova   110 Apr 11 14:22 instance-0000023f
drwxr-xr-x.  2 nova nova   110 Apr 11 14:22 instance-00000240
drwxr-xr-x.  2 nova nova   110 Apr 11 14:25 instance-00000241
drwxr-xr-x.  2 nova nova   110 Apr 11 14:31 instance-00000242

instance-00000241 is an example of one that's failed, inside of there:
[root@openstack-13 instance-00000241]# ls -al
total 4678
drwxr-xr-x.  2 nova nova     110 Apr 11 14:25 .
drwxr-x---. 11 nova nova   24576 Apr 11 14:31 ..
-rw-rw----.  1 root root       0 Apr 11 14:25 console.log
-rw-r--r--.  1 root root  262144 Apr 11 14:25 disk
-rw-r--r--.  1 root root 4404752 Apr 11 14:25 kernel
-rw-r--r--.  1 nova nova    1277 Apr 11 14:25 libvirt.xml
-rw-r--r--.  1 root root   96629 Apr 11 14:25 ramdisk

To me, it seems reasonable.  I'm happy to be wrong though.
thanks,
JP

On Apr 11, 2013, at 10:49 AM, Razique Mahroua <razique.mahroua@xxxxxxxxx> wrote:

> Hi JP,
> my bet is that this is a writing permissions issue. Does nova has the right to write within the mounted directory?
> 
> Razique Mahroua - Nuage & Co
> razique.mahroua@xxxxxxxxx
> Tel : +33 9 72 37 94 15
> 
> <NUAGECO-LOGO-Fblan_petit.jpg>
> 
> Le 11 avr. 2013 à 16:36, John Paul Walters <jwalters@xxxxxxx> a écrit :
> 
>> Hi,
>> 
>> We've started implementing a Glusterfs-based solution for instance storage in order to provide live migration.  I've run into a strange problem when using a multi-node Gluster setup that I hope someone has a suggestion to resolve.
>> 
>> I have a 12 node distributed/replicated Gluster cluster.  I can mount it to my client machines, and it seems to be working alright.  When I launch instances, the nova-compute log on the client machines are giving me two error messages:
>> 
>> First is a qemu-kvm error: could not open disk image /exports/instances/instances/instance-00000242/disk: Invalid argument
>> (full output at http://pastebin.com/i8vzWegJ)
>> 
>> The second error message comes a short time later ending with nova.openstack.common.rpc.amqp Invalid: Instance has already been created
>> (full output at http://pastebin.com/6Ta4kkBN)
>> 
>> This happens reliably with the multi-Gluster-node setup.  Oddly, after creating a test Gluster volume composed of a single brick and single node, everything works fine.
>> 
>> Does anyone have any suggestions?
>> 
>> thanks,
>> JP
>> 
>> 
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>