← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #87406

[Bug 1946974] [NEW] TypeError: Can't upgrade a READER transaction to a WRITER mid-transaction

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

Hello,

keystone fails to generate the catalog when there is an entry in project_endpoint table with no corresponding project id in project table and responds with a 400/Bad request status, instead of cleaning the incorrect entry (cf https://opendev.org/openstack/keystone/src/branch/master/keystone/catalog/core.py#L308) Here is the traceback :
Traceback (most recent call last):
  File "/opt/keystone/local/lib/python2.7/site-packages/keystone/common/wsgi.py", line 228, in __call__
    result = method(req, **params)
  File "/opt/keystone/local/lib/python2.7/site-packages/keystone/auth/controllers.py", line 144, in authenticate_for_token
    parent_audit_id=token_audit_id)
  File "/opt/keystone/local/lib/python2.7/site-packages/keystone/common/manager.py", line 123, in wrapped
    __ret_val = __f(*args, **kwargs)
  File "/opt/keystone/local/lib/python2.7/site-packages/keystone/token/provider.py", line 221, in issue_token
    auth_context, trust, include_catalog, parent_audit_id)
  File "/opt/keystone/local/lib/python2.7/site-packages/keystone/token/providers/fernet/core.py", line 55, in issue_token
    *args, **kwargs)
  File "/opt/keystone/local/lib/python2.7/site-packages/keystone/token/providers/common.py", line 481, in issue_token
    audit_info=parent_audit_id)
  File "/opt/keystone/local/lib/python2.7/site-packages/keystone/token/providers/common.py", line 410, in get_token_data
    project_id, trust)
  File "/opt/keystone/local/lib/python2.7/site-packages/keystone/token/providers/common.py", line 354, in _populate_service_catalog
    user_id, project_id, domain_id)
  File "/opt/keystone/local/lib/python2.7/site-packages/keystone/common/manager.py", line 123, in wrapped
    __ret_val = __f(*args, **kwargs)
  File "/opt/keystone/local/lib/python2.7/site-packages/dogpile/cache/region.py", line 1220, in decorate
    should_cache_fn)
  File "/opt/keystone/local/lib/python2.7/site-packages/dogpile/cache/region.py", line 825, in get_or_create
    async_creator) as value:
  File "/opt/keystone/local/lib/python2.7/site-packages/dogpile/lock.py", line 154, in __enter__
    return self._enter()
  File "/opt/keystone/local/lib/python2.7/site-packages/dogpile/lock.py", line 94, in _enter
    generated = self._enter_create(createdtime)
  File "/opt/keystone/local/lib/python2.7/site-packages/dogpile/lock.py", line 145, in _enter_create
    created = self.creator()
  File "/opt/keystone/local/lib/python2.7/site-packages/dogpile/cache/region.py", line 792, in gen_value
    created_value = creator()
  File "/opt/keystone/local/lib/python2.7/site-packages/dogpile/cache/region.py", line 1216, in creator
    return fn(*arg, **kw)
  File "/opt/keystone/local/lib/python2.7/site-packages/keystone/catalog/core.py", line 227, in get_v3_catalog
    return self.driver.get_v3_catalog(user_id, project_id, domain_id)
  File "/opt/keystone/local/lib/python2.7/site-packages/keystone/catalog/backends/sql.py", line 421, in get_v3_catalog
    self.catalog_api.list_endpoints_for_project(project_id))
  File "/opt/keystone/local/lib/python2.7/site-packages/keystone/common/manager.py", line 123, in wrapped
    __ret_val = __f(*args, **kwargs)
  File "/opt/keystone/local/lib/python2.7/site-packages/keystone/catalog/core.py", line 300, in list_endpoints_for_project
    project_id)
  File "/opt/keystone/local/lib/python2.7/site-packages/keystone/common/manager.py", line 123, in wrapped
    __ret_val = __f(*args, **kwargs)
  File "/opt/keystone/local/lib/python2.7/site-packages/keystone/catalog/core.py", line 234, in remove_endpoint_from_project
    self.driver.remove_endpoint_from_project(endpoint_id, project_id)
  File "/opt/keystone/local/lib/python2.7/site-packages/keystone/catalog/backends/sql.py", line 484, in remove_endpoint_from_project
    with sql.session_for_write() as session:
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/opt/keystone/local/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 944, in _transaction_scope
    allow_async=self._allow_async) as resource:
  File "/opt/keystone/local/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 600, in _produce_block
    self._writer()
  File "/opt/keystone/local/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 615, in _writer
    "Can't upgrade a READER transaction "
TypeError: Can't upgrade a READER transaction to a WRITER mid-transaction

This bug raised on our old v11.0.3 but has been reproduced with version
20.0.0.0rc2.dev3 (using devstack) by adding an entry in project_endpoint
table without a corresponding project id in project table.

** Affects: keystone
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Identity (keystone).
https://bugs.launchpad.net/bugs/1946974

Title:
  TypeError: Can't upgrade a READER transaction to a WRITER mid-
  transaction

Status in OpenStack Identity (keystone):
  New

Bug description:
  Hello,

  keystone fails to generate the catalog when there is an entry in project_endpoint table with no corresponding project id in project table and responds with a 400/Bad request status, instead of cleaning the incorrect entry (cf https://opendev.org/openstack/keystone/src/branch/master/keystone/catalog/core.py#L308) Here is the traceback :
  Traceback (most recent call last):
    File "/opt/keystone/local/lib/python2.7/site-packages/keystone/common/wsgi.py", line 228, in __call__
      result = method(req, **params)
    File "/opt/keystone/local/lib/python2.7/site-packages/keystone/auth/controllers.py", line 144, in authenticate_for_token
      parent_audit_id=token_audit_id)
    File "/opt/keystone/local/lib/python2.7/site-packages/keystone/common/manager.py", line 123, in wrapped
      __ret_val = __f(*args, **kwargs)
    File "/opt/keystone/local/lib/python2.7/site-packages/keystone/token/provider.py", line 221, in issue_token
      auth_context, trust, include_catalog, parent_audit_id)
    File "/opt/keystone/local/lib/python2.7/site-packages/keystone/token/providers/fernet/core.py", line 55, in issue_token
      *args, **kwargs)
    File "/opt/keystone/local/lib/python2.7/site-packages/keystone/token/providers/common.py", line 481, in issue_token
      audit_info=parent_audit_id)
    File "/opt/keystone/local/lib/python2.7/site-packages/keystone/token/providers/common.py", line 410, in get_token_data
      project_id, trust)
    File "/opt/keystone/local/lib/python2.7/site-packages/keystone/token/providers/common.py", line 354, in _populate_service_catalog
      user_id, project_id, domain_id)
    File "/opt/keystone/local/lib/python2.7/site-packages/keystone/common/manager.py", line 123, in wrapped
      __ret_val = __f(*args, **kwargs)
    File "/opt/keystone/local/lib/python2.7/site-packages/dogpile/cache/region.py", line 1220, in decorate
      should_cache_fn)
    File "/opt/keystone/local/lib/python2.7/site-packages/dogpile/cache/region.py", line 825, in get_or_create
      async_creator) as value:
    File "/opt/keystone/local/lib/python2.7/site-packages/dogpile/lock.py", line 154, in __enter__
      return self._enter()
    File "/opt/keystone/local/lib/python2.7/site-packages/dogpile/lock.py", line 94, in _enter
      generated = self._enter_create(createdtime)
    File "/opt/keystone/local/lib/python2.7/site-packages/dogpile/lock.py", line 145, in _enter_create
      created = self.creator()
    File "/opt/keystone/local/lib/python2.7/site-packages/dogpile/cache/region.py", line 792, in gen_value
      created_value = creator()
    File "/opt/keystone/local/lib/python2.7/site-packages/dogpile/cache/region.py", line 1216, in creator
      return fn(*arg, **kw)
    File "/opt/keystone/local/lib/python2.7/site-packages/keystone/catalog/core.py", line 227, in get_v3_catalog
      return self.driver.get_v3_catalog(user_id, project_id, domain_id)
    File "/opt/keystone/local/lib/python2.7/site-packages/keystone/catalog/backends/sql.py", line 421, in get_v3_catalog
      self.catalog_api.list_endpoints_for_project(project_id))
    File "/opt/keystone/local/lib/python2.7/site-packages/keystone/common/manager.py", line 123, in wrapped
      __ret_val = __f(*args, **kwargs)
    File "/opt/keystone/local/lib/python2.7/site-packages/keystone/catalog/core.py", line 300, in list_endpoints_for_project
      project_id)
    File "/opt/keystone/local/lib/python2.7/site-packages/keystone/common/manager.py", line 123, in wrapped
      __ret_val = __f(*args, **kwargs)
    File "/opt/keystone/local/lib/python2.7/site-packages/keystone/catalog/core.py", line 234, in remove_endpoint_from_project
      self.driver.remove_endpoint_from_project(endpoint_id, project_id)
    File "/opt/keystone/local/lib/python2.7/site-packages/keystone/catalog/backends/sql.py", line 484, in remove_endpoint_from_project
      with sql.session_for_write() as session:
    File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
      return self.gen.next()
    File "/opt/keystone/local/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 944, in _transaction_scope
      allow_async=self._allow_async) as resource:
    File "/opt/keystone/local/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 600, in _produce_block
      self._writer()
    File "/opt/keystone/local/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 615, in _writer
      "Can't upgrade a READER transaction "
  TypeError: Can't upgrade a READER transaction to a WRITER mid-transaction

  This bug raised on our old v11.0.3 but has been reproduced with
  version 20.0.0.0rc2.dev3 (using devstack) by adding an entry in
  project_endpoint table without a corresponding project id in project
  table.

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1946974/+subscriptions
  Thread Previous • Date Previous • • Thread Next