← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #24409

[Bug 1391443] [NEW] ds_util.py not hundle unicode exceptioin from error_util.FileNotFoundException

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

summary:
exception in functioin “file_exists” and  “get_sub_folders" in module ds_util.py do not handle UnicodeEncodeError .

when error_util.FileNotFoundException raised, some message are translated into local language.
If the local evn is unicode os(like japanese), UnicodeEncodeError encode exception will be raised and not be handled.

code from 2014.2-651-g051e496:

nova/virt/vmwareapi/ds_util.py
392 def file_exists(session, ds_browser, ds_path, file_name):
393     """Check if the file exists on the datastore."""
394     client_factory = session.vim.client.factory
395     search_spec = search_datastore_spec(client_factory, file_name)
396     search_task = session._call_method(session.vim,
397                                              "SearchDatastore_Task",
398                                              ds_browser,
399                                              datastorePath=str(ds_path),
400                                              searchSpec=search_spec)
401     try:
402         task_info = session._wait_for_task(search_task)
403     except vexc.FileNotFoundException:
404         return False
405 
406     file_exists = (getattr(task_info.result, 'file', False) and
407                    task_info.result.file[0].path == file_name)
408     return file_exists
....
424 def get_sub_folders(session, ds_browser, ds_path):
425     """Return a set of subfolders for a path on a datastore.
426 
427     If the path does not exist then an empty set is returned.
428     """
429     search_task = session._call_method(
430             session.vim,
431             "SearchDatastore_Task",
432             ds_browser,
433             datastorePath=str(ds_path))
434     try:
435         task_info = session._wait_for_task(search_task)
436     except vexc.FileNotFoundException:
437         return set()
438     # populate the folder entries
439     if hasattr(task_info.result, 'file'):
440         return set([file.path for file in task_info.result.file])
441     return set()

env:
openstack-nova-common-2014.1.2 or later

reason:


how to reproduce:
1. controler is one unicode os (ja_JP.UTF-8)

2. use one sparce vmdk image to boot one instance
step:
a)create one image in glance
glance image-create --name=rhel64-vmware --disk-format=vmdk  --container-format=bare  --is-public=True  --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hypervisor_type="vmware" < rhel64-vmware.vmdk
b)boot one instance in nova
nova boot --image 0e02eae0-0b03-42a7-ab6b-fd27b26adaa2 --flavor 2 zztest_05

3. we found exception in nova-compute.log

2014-11-10 21:52:24.033 10475 DEBUG nova.openstack.common.lockutils [req-5d1f2bd7-e8a2-4378-95b3-c7d5d7d1b5c4 c5e78c9f8897461f8ba2
4f98fb5f4553 6c4834fa3eeb45468b9164b9429cd062] Released file lock "[24:storage1] 10.10.6.11_base/0e02eae0-0b03-42a7-ab6b-fd27b26ad
aa2" at /var/lib/nova/lock/nova-vmware-ts-[24:storage1] 10.10.6.11_base_0e02eae0-0b03-42a7-ab6b-fd27b26adaa2 lock /usr/lib/python2
.6/site-packages/nova/openstack/common/lockutils.py:210
2014-11-10 21:52:24.033 10475 ERROR nova.compute.manager [req-5d1f2bd7-e8a2-4378-95b3-c7d5d7d1b5c4 c5e78c9f8897461f8ba24f98fb5f455
3 6c4834fa3eeb45468b9164b9429cd062] [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142] Instance failed to spawn
2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142] Traceback (most recent c
all last):
2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]   File "/usr/lib/python2
.6/site-packages/nova/compute/manager.py", line 1765, in _spawn
2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]     block_device_info)
2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]   File "/usr/lib/python2
.6/site-packages/nova/virt/vmwareapi/driver.py", line 854, in spawn
2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]     admin_password, netw
ork_info, block_device_info)
2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]   File "/usr/lib/python2
.6/site-packages/nova/virt/vmwareapi/vmops.py", line 589, in spawn
2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]     data_store_ref, data
_store_name, path)
2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]   File "/usr/lib/python2
.6/site-packages/nova/virt/vmwareapi/imagecache.py", line 85, in timestamp_cleanup
2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]     ts = self._get_times
tamp(ds_browser, ds_path)
2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]   File "/usr/lib/python2
.6/site-packages/nova/virt/vmwareapi/imagecache.py", line 93, in _get_timestamp
2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]     files = ds_util.get_
sub_folders(self._session, ds_browser, ds_path)
2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]   File "/usr/lib/python2
.6/site-packages/nova/virt/vmwareapi/ds_util.py", line 141, in get_sub_folders
2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]     task_info = session.
_wait_for_task(search_task)
2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]   File "/usr/lib/python2
.6/site-packages/nova/virt/vmwareapi/driver.py", line 1220, in _wait_for_task
2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]     ret_val = done.wait(
)
2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]   File "/usr/lib/python2
.6/site-packages/eventlet/event.py", line 116, in wait
2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]     return hubs.get_hub(
).switch()
2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]   File "/usr/lib/python2
.6/site-packages/eventlet/hubs/hub.py", line 187, in switch
2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]     return self.greenlet
.switch()
2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142] UnicodeEncodeError: 'asc
ii' codec can't encode characters in position 0-3: ordinal not in range(128)
2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]
2014-11-10 21:52:24.049 10475 DEBUG nova.compute.claims [req-5d1f2bd7-e8a2-4378-95b3-c7d5d7d1b5c4 c5e78c9f8897461f8ba24f98fb5f4553
6c4834fa3eeb45468b9164b9429cd062] [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142] Aborting claim: [Claim: 2048 MB memory, 20 GB
disk, 1 VCPUS] abort /usr/lib/python2.6/site-packages/nova/compute/claims.py:113

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1391443

Title:
  ds_util.py not hundle unicode  exceptioin from
  error_util.FileNotFoundException

Status in OpenStack Compute (Nova):
  New

Bug description:
  summary:
  exception in functioin “file_exists” and  “get_sub_folders" in module ds_util.py do not handle UnicodeEncodeError .

  when error_util.FileNotFoundException raised, some message are translated into local language.
  If the local evn is unicode os(like japanese), UnicodeEncodeError encode exception will be raised and not be handled.

  code from 2014.2-651-g051e496:

  nova/virt/vmwareapi/ds_util.py
  392 def file_exists(session, ds_browser, ds_path, file_name):
  393     """Check if the file exists on the datastore."""
  394     client_factory = session.vim.client.factory
  395     search_spec = search_datastore_spec(client_factory, file_name)
  396     search_task = session._call_method(session.vim,
  397                                              "SearchDatastore_Task",
  398                                              ds_browser,
  399                                              datastorePath=str(ds_path),
  400                                              searchSpec=search_spec)
  401     try:
  402         task_info = session._wait_for_task(search_task)
  403     except vexc.FileNotFoundException:
  404         return False
  405 
  406     file_exists = (getattr(task_info.result, 'file', False) and
  407                    task_info.result.file[0].path == file_name)
  408     return file_exists
  ....
  424 def get_sub_folders(session, ds_browser, ds_path):
  425     """Return a set of subfolders for a path on a datastore.
  426 
  427     If the path does not exist then an empty set is returned.
  428     """
  429     search_task = session._call_method(
  430             session.vim,
  431             "SearchDatastore_Task",
  432             ds_browser,
  433             datastorePath=str(ds_path))
  434     try:
  435         task_info = session._wait_for_task(search_task)
  436     except vexc.FileNotFoundException:
  437         return set()
  438     # populate the folder entries
  439     if hasattr(task_info.result, 'file'):
  440         return set([file.path for file in task_info.result.file])
  441     return set()

  env:
  openstack-nova-common-2014.1.2 or later

  reason:

  
  how to reproduce:
  1. controler is one unicode os (ja_JP.UTF-8)

  2. use one sparce vmdk image to boot one instance
  step:
  a)create one image in glance
  glance image-create --name=rhel64-vmware --disk-format=vmdk  --container-format=bare  --is-public=True  --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hypervisor_type="vmware" < rhel64-vmware.vmdk
  b)boot one instance in nova
  nova boot --image 0e02eae0-0b03-42a7-ab6b-fd27b26adaa2 --flavor 2 zztest_05

  3. we found exception in nova-compute.log

  2014-11-10 21:52:24.033 10475 DEBUG nova.openstack.common.lockutils [req-5d1f2bd7-e8a2-4378-95b3-c7d5d7d1b5c4 c5e78c9f8897461f8ba2
  4f98fb5f4553 6c4834fa3eeb45468b9164b9429cd062] Released file lock "[24:storage1] 10.10.6.11_base/0e02eae0-0b03-42a7-ab6b-fd27b26ad
  aa2" at /var/lib/nova/lock/nova-vmware-ts-[24:storage1] 10.10.6.11_base_0e02eae0-0b03-42a7-ab6b-fd27b26adaa2 lock /usr/lib/python2
  .6/site-packages/nova/openstack/common/lockutils.py:210
  2014-11-10 21:52:24.033 10475 ERROR nova.compute.manager [req-5d1f2bd7-e8a2-4378-95b3-c7d5d7d1b5c4 c5e78c9f8897461f8ba24f98fb5f455
  3 6c4834fa3eeb45468b9164b9429cd062] [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142] Instance failed to spawn
  2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142] Traceback (most recent c
  all last):
  2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]   File "/usr/lib/python2
  .6/site-packages/nova/compute/manager.py", line 1765, in _spawn
  2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]     block_device_info)
  2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]   File "/usr/lib/python2
  .6/site-packages/nova/virt/vmwareapi/driver.py", line 854, in spawn
  2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]     admin_password, netw
  ork_info, block_device_info)
  2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]   File "/usr/lib/python2
  .6/site-packages/nova/virt/vmwareapi/vmops.py", line 589, in spawn
  2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]     data_store_ref, data
  _store_name, path)
  2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]   File "/usr/lib/python2
  .6/site-packages/nova/virt/vmwareapi/imagecache.py", line 85, in timestamp_cleanup
  2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]     ts = self._get_times
  tamp(ds_browser, ds_path)
  2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]   File "/usr/lib/python2
  .6/site-packages/nova/virt/vmwareapi/imagecache.py", line 93, in _get_timestamp
  2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]     files = ds_util.get_
  sub_folders(self._session, ds_browser, ds_path)
  2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]   File "/usr/lib/python2
  .6/site-packages/nova/virt/vmwareapi/ds_util.py", line 141, in get_sub_folders
  2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]     task_info = session.
  _wait_for_task(search_task)
  2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]   File "/usr/lib/python2
  .6/site-packages/nova/virt/vmwareapi/driver.py", line 1220, in _wait_for_task
  2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]     ret_val = done.wait(
  )
  2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]   File "/usr/lib/python2
  .6/site-packages/eventlet/event.py", line 116, in wait
  2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]     return hubs.get_hub(
  ).switch()
  2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]   File "/usr/lib/python2
  .6/site-packages/eventlet/hubs/hub.py", line 187, in switch
  2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]     return self.greenlet
  .switch()
  2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142] UnicodeEncodeError: 'asc
  ii' codec can't encode characters in position 0-3: ordinal not in range(128)
  2014-11-10 21:52:24.033 10475 TRACE nova.compute.manager [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142]
  2014-11-10 21:52:24.049 10475 DEBUG nova.compute.claims [req-5d1f2bd7-e8a2-4378-95b3-c7d5d7d1b5c4 c5e78c9f8897461f8ba24f98fb5f4553
  6c4834fa3eeb45468b9164b9429cd062] [instance: e0dce9e2-f1e3-43bc-8426-98f7716e0142] Aborting claim: [Claim: 2048 MB memory, 20 GB
  disk, 1 VCPUS] abort /usr/lib/python2.6/site-packages/nova/compute/claims.py:113

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1391443/+subscriptions

Follow ups

References

  Thread Previous • Date Previous • • Thread Next