openstack team mailing list archive

["Lv, Hui" <hui.lv@xxxxxxxxx>]

Hi openstack list,

I have two problems related to openstack volume service. I really appreciate your kind reply.

1)Recently, we have measured the volume service performance by selecting the "tgt" as the target management software.(We use Diablo Openstack and KVM as the underlying hypervisor)
We created 8 instances, each of which was assigned one iscsi disk from openstack volume service (RAID0 storage as the LVM volumes with up to 800 MB/s local sequential READ I/O badnwidth).
These 8 instances were distributed on 8 physical machines, each of which was assigned 1Gb network bandwidth.
We measured the sequential READ (aio-stress) simultaneously from these 8 instances and identified the total READ I/O bandwidth can only reach up to ~100 MB/s.
We observed the iscsi target was not created in "O_DIRECT" mode, (hard coded in */nova/volume/driver.py).
After changing to use "O_DIRECT" mode, by adding "--bsoflags=direct", the sequential READ I/O bandwidth can reach to ~250 MB/s, greatly improved.

My question is that, why not use "O_DIRECT" mode for "tgt" iscsi target creation? Is there any special consideration? As I know, another target management software "iet" created the iscsi target as "O_DIRECT" mode by default.
Also, we observed that "iet" has better I/O performance than "tgt". We measured this in 8 native system instead of instances, since our openstack (diablo) does not support iet.

We added the "O_DIRECT" mode in the following way:
self._execute('tgtadm', '--op', 'new',
                  '--lld=iscsi', '--mode=logicalunit',
                  "--tid=%s" % iscsi_target,
                  '--lun=1',
                  "--backing-store=%s,Type=fileio" % volume_path,
                  '--bsoflags=direct',
                  run_as_root=True)

2)Does Xenserver support volume services? Is there any document to introduce how to enable volume for Xenserver?
Because, when I attached a volume target to an instance created on Xenserver by using euca tools, no such iscsi target appeared on the instance. Also there were no error messages from log files.

Best regards,

Hui