yahoo-eng-team team mailing list archive

[Tobias Urdin <2076966@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

If fail late in a resize operation the instance base directory has
already been renamed but we don't check if it exists before trying to
rename it and thus crash with FileNotFound

{'code': 500, 'created': '2024-08-14T07:02:49Z', 'message':
'FileNotFoundError', 'details': 'Traceback (most recent call last):\n
File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line
201, in decorated_function\n    return function(self, context, *args,
**kwargs)\n  File "/usr/lib/python3.9/site-
packages/nova/compute/manager.py", line 5857, in resize_instance\n
self._revert_allocation(context, instance, migration)\n  File
"/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 227, in
__exit__\n    self.force_reraise()\n  File "/usr/lib/python3.9/site-
packages/oslo_utils/excutils.py", line 200, in force_reraise\n    raise
self.value\n  File "/usr/lib/python3.9/site-
packages/nova/compute/manager.py", line 5853, in resize_instance\n
self._resize_instance(context, instance, image, migration,\n  File
"/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 5890,
in _resize_instance\n    disk_info =
self.driver.migrate_disk_and_power_off(\n  File
"/usr/lib/python3.9/site-packages/nova/virt/libvirt/driver.py", line
11521, in migrate_disk_and_power_off\n
self._cleanup_remote_migration(dest, inst_base,\n  File
"/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 227, in
__exit__\n    self.force_reraise()\n  File "/usr/lib/python3.9/site-
packages/oslo_utils/excutils.py", line 200, in force_reraise\n    raise
self.value\n  File "/usr/lib/python3.9/site-
packages/nova/virt/libvirt/driver.py", line 11471, in
migrate_disk_and_power_off\n    os.rename(inst_base,
inst_base_resize)\nFileNotFoundError: [Errno 2] No such file or
directory:
\'/var/lib/nova/instances/891d8b95-45ce-4a52-b630-6805648ce67e\' ->
\'/var/lib/nova/instances/891d8b95-45ce-4a52-b630-6805648ce67e_resize\'\n'}

** Affects: nova
     Importance: Undecided
         Status: In Progress

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2076966

Title:
  libvirt driver FileNotFound trying to rename old instance base

Status in OpenStack Compute (nova):
  In Progress

Bug description:
  If fail late in a resize operation the instance base directory has
  already been renamed but we don't check if it exists before trying to
  rename it and thus crash with FileNotFound

  {'code': 500, 'created': '2024-08-14T07:02:49Z', 'message':
  'FileNotFoundError', 'details': 'Traceback (most recent call last):\n
  File "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line
  201, in decorated_function\n    return function(self, context, *args,
  **kwargs)\n  File "/usr/lib/python3.9/site-
  packages/nova/compute/manager.py", line 5857, in resize_instance\n
  self._revert_allocation(context, instance, migration)\n  File
  "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 227,
  in __exit__\n    self.force_reraise()\n  File
  "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 200,
  in force_reraise\n    raise self.value\n  File
  "/usr/lib/python3.9/site-packages/nova/compute/manager.py", line 5853,
  in resize_instance\n    self._resize_instance(context, instance,
  image, migration,\n  File "/usr/lib/python3.9/site-
  packages/nova/compute/manager.py", line 5890, in _resize_instance\n
  disk_info = self.driver.migrate_disk_and_power_off(\n  File
  "/usr/lib/python3.9/site-packages/nova/virt/libvirt/driver.py", line
  11521, in migrate_disk_and_power_off\n
  self._cleanup_remote_migration(dest, inst_base,\n  File
  "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 227,
  in __exit__\n    self.force_reraise()\n  File
  "/usr/lib/python3.9/site-packages/oslo_utils/excutils.py", line 200,
  in force_reraise\n    raise self.value\n  File
  "/usr/lib/python3.9/site-packages/nova/virt/libvirt/driver.py", line
  11471, in migrate_disk_and_power_off\n    os.rename(inst_base,
  inst_base_resize)\nFileNotFoundError: [Errno 2] No such file or
  directory:
  \'/var/lib/nova/instances/891d8b95-45ce-4a52-b630-6805648ce67e\' ->
  \'/var/lib/nova/instances/891d8b95-45ce-4a52-b630-6805648ce67e_resize\'\n'}

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2076966/+subscriptions