yahoo-eng-team team mailing list archive

[Sabari Murugesan <1459015@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

In general, when a glance v2 task runs into an error during  the flow
execution, the task gets stuck in the processing state. We are currently
not marking the task as failed.

I specifically ran into an issue where the import was failing due to a
permission issue on the task.work_dir which was masked by the following
exception :-

  File "/opt/stack/glance/glance/async/__init__.py", line 64, in begin_processing
    self._run(task_id, task.type)
  File "/opt/stack/glance/glance/async/taskflow_executor.py", line 133, in _run
    self.task_repo.save(task)
  File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
    six.reraise(self.type_, self.value, self.tb)
  File "/opt/stack/glance/glance/async/taskflow_executor.py", line 127, in _run
    engine.run()
  File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 96, in run
    for _state in self.run_iter():
  File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 146, in run_iter
    self._change_state(states.FAILURE)
  File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
    six.reraise(self.type_, self.value, self.tb)
  File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 130, in run_iter
    failure.Failure.reraise_if_any(failures)
  File "/usr/local/lib/python2.7/dist-packages/taskflow/types/failure.py", line 244, in reraise_if_any
    failures[0].reraise()
  File "/usr/local/lib/python2.7/dist-packages/taskflow/types/failure.py", line 251, in reraise
    six.reraise(*self._exc_info)
  File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/executor.py", line 86, in _revert_task
    result = task.revert(**arguments)
  File "/opt/stack/glance/glance/async/flows/base_import.py", line 158, in revert
    if os.path.exists(result.split("file://")[-1]):
AttributeError: 'Failure' object has no attribute 'split'

** Affects: glance
     Importance: Undecided
     Assignee: Sabari Murugesan (smurugesan)
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/1459015

Title:
  Tasks stuck in processing state when import to filesystem fails

Status in OpenStack Image Registry and Delivery Service (Glance):
  New

Bug description:
  In general, when a glance v2 task runs into an error during  the flow
  execution, the task gets stuck in the processing state. We are
  currently not marking the task as failed.

  I specifically ran into an issue where the import was failing due to a
  permission issue on the task.work_dir which was masked by the
  following exception :-

    File "/opt/stack/glance/glance/async/__init__.py", line 64, in begin_processing
      self._run(task_id, task.type)
    File "/opt/stack/glance/glance/async/taskflow_executor.py", line 133, in _run
      self.task_repo.save(task)
    File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
      six.reraise(self.type_, self.value, self.tb)
    File "/opt/stack/glance/glance/async/taskflow_executor.py", line 127, in _run
      engine.run()
    File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 96, in run
      for _state in self.run_iter():
    File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 146, in run_iter
      self._change_state(states.FAILURE)
    File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
      six.reraise(self.type_, self.value, self.tb)
    File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 130, in run_iter
      failure.Failure.reraise_if_any(failures)
    File "/usr/local/lib/python2.7/dist-packages/taskflow/types/failure.py", line 244, in reraise_if_any
      failures[0].reraise()
    File "/usr/local/lib/python2.7/dist-packages/taskflow/types/failure.py", line 251, in reraise
      six.reraise(*self._exc_info)
    File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/executor.py", line 86, in _revert_task
      result = task.revert(**arguments)
    File "/opt/stack/glance/glance/async/flows/base_import.py", line 158, in revert
      if os.path.exists(result.split("file://")[-1]):
  AttributeError: 'Failure' object has no attribute 'split'

To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/1459015/+subscriptions