← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1945538] Re: Database permission configured properly but I have the Access deny

 

This looks to me an unrelated issue for the Nova repository. This rather
looks to me an issue with the Ubuntu packages.

** Changed in: nova
       Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1945538

Title:
  Database permission configured properly but I have the Access deny

Status in OpenStack Compute (nova):
  Invalid

Bug description:
  Description
  ===========
  I installed Openstack using [OpenStack Installation Guide][OpenStack Installation Guide], all command and configuration is on my [Github][My-Github].

  At this point **one controller** with **two compute** node.

  * Nova configuration on controller*
  user@controller001:~$ sudo !!
  sudo grep -v '^\s*$\|^\s*\#' /etc/nova/nova.conf
  [DEFAULT]
  log_dir = /var/log/nova
  lock_path = /var/lock/nova
  state_path = /var/lib/nova
  transport_url = rabbit://openstack:openstack@controller001:5672/
  my_ip = 192.168.56.50
  [api]
  auth_strategy = keystone
  [api_database]
  connection = mysql+pymysql://nova:openstack@controller001/nova_api
  [barbican]
  [cache]
  [cinder]
  [compute]
  [conductor]
  [console]
  [consoleauth]
  [cors]
  [cyborg]
  [database]
  connection = mysql+pymysql://nova:openstack@controller001/nova
  [devices]
  [ephemeral_storage_encryption]
  [filter_scheduler]
  [glance]
  api_servers = http://controller001:9292
  [guestfs]
  [healthcheck]
  [hyperv]
  [image_cache]
  [ironic]
  [key_manager]
  [keystone]
  [keystone_authtoken]
  www_authenticate_uri = http://controller001:5000/
  auth_url = http://controller001:5000/
  memcached_servers = controller001:11211
  auth_type = password
  project_domain_name = Default
  user_domain_name = Default
  project_name = service
  username = nova
  password = openstack
  [libvirt]
  [metrics]
  [mks]
  [neutron]
  [notifications]
  [oslo_concurrency]
  lock_path = /var/lib/nova/tmp
  [oslo_messaging_amqp]
  [oslo_messaging_kafka]
  [oslo_messaging_notifications]
  [oslo_messaging_rabbit]
  [oslo_middleware]
  [oslo_policy]
  [pci]
  [placement]
  region_name = RegionOne
  project_domain_name = Default
  project_name = service
  auth_type = password
  user_domain_name = Default
  auth_url = http://controller001:5000/v3
  username = placement
  password = openstack
  [powervm]
  [privsep]
  [profiler]
  [quota]
  [rdp]
  [remote_debug]
  [scheduler]
  discover_hosts_in_cells_interval = 300
  [serial_console]
  [service_user]
  [spice]
  [upgrade_levels]
  [vault]
  [vendordata_dynamic_auth]
  [vmware]
  [vnc]
  enabled = true
  server_listen = $my_ip
  server_proxyclient_address = $my_ip
  [workarounds]
  [wsgi]
  [zvm]
  [cells]
  enable = False
  [os_region_name]
  openstack = 

  
  * Nova configuration on compute:*
  user@compute001:~$ sudo grep -v '^\s*$\|^\s*\#' /etc/nova/nova.conf
  [DEFAULT]
  log_dir = /var/log/nova
  lock_path = /var/lock/nova
  state_path = /var/lib/nova
  transport_url = rabbit://openstack:openstack@controller001
  my_ip = 172.16.56.51
  [api]
  auth_strategy = keystone
  [api_database]
  connection = sqlite:////var/lib/nova/nova_api.sqlite
  [barbican]
  [cache]
  [cinder]
  [compute]
  [conductor]
  [console]
  [consoleauth]
  [cors]
  [cyborg]
  [database]
  connection = sqlite:////var/lib/nova/nova.sqlite
  [devices]
  [ephemeral_storage_encryption]
  [filter_scheduler]
  [glance]
  api_servers = http://controller001:9292
  [guestfs]
  [healthcheck]
  [hyperv]
  [image_cache]
  [ironic]
  [key_manager]
  [keystone]
  [keystone_authtoken]
  www_authenticate_uri = http://controller001:5000/
  auth_url = http://controller001:5000/
  memcached_servers = controller001:11211
  auth_type = password
  project_domain_name = Default
  user_domain_name = Default
  project_name = service
  username = nova
  password = openstack
  [libvirt]
  [metrics]
  [mks]
  [neutron]
  [notifications]
  [oslo_concurrency]
  lock_path = /var/lib/nova/tmp
  [oslo_messaging_amqp]
  [oslo_messaging_kafka]
  [oslo_messaging_notifications]
  [oslo_messaging_rabbit]
  [oslo_middleware]
  [oslo_policy]
  [pci]
  [placement]
  region_name = RegionOne
  project_domain_name = Default
  project_name = service
  auth_type = password
  user_domain_name = Default
  auth_url = http://controller001:5000/v3
  username = placement
  password = openstack
  [powervm]
  [privsep]
  [profiler]
  [quota]
  [rdp]
  [remote_debug]
  [scheduler]
  discover_hosts_in_cells_interval = 300
  [serial_console]
  [service_user]
  [spice]
  [upgrade_levels]
  [vault]
  [vendordata_dynamic_auth]
  [vmware]
  [vnc]
  enabled = true
  server_listen = 0.0.0.0
  server_proxyclient_address = $my_ip
  novncproxy_base_url = http://controller001:6080/vnc_auto.html
  [workarounds]
  [wsgi]
  [zvm]
  [cells]
  enable = False
  [os_region_name]
  openstack =

  
  In the below section improve database permission configured correctly
  --------------------------------------------------------------------

  *Databases are created properly:*

      user@controller001:~$ sudo mysql
      MariaDB [(none)]> SHOW DATABASES;
      +--------------------+
      | Database           |
      +--------------------+
      | glance             |
      | information_schema |
      | keystone           |
      | mysql              |
      | nova               |
      | nova_api           |
      | nova_cell0         |
      | performance_schema |
      | placement          |
      +--------------------+

  
  *Grant all permissions for those databases:*

      MariaDB [(none)]> SHOW GRANTS FOR nova;
      +-----------------------------------------------------------------------------------------------------+
      | Grants for nova@%                                                                                   |
      +-----------------------------------------------------------------------------------------------------+
      | GRANT USAGE ON *.* TO `nova`@`%` IDENTIFIED BY PASSWORD '*3A4A03AC22526F6B591010973A741D59A71D728E' |
      | GRANT ALL PRIVILEGES ON `nova`.* TO `nova`@`%`                                                      |
      | GRANT ALL PRIVILEGES ON `nova_cell0`.* TO `nova`@`%`                                                |
      | GRANT ALL PRIVILEGES ON `nova_api`.* TO `nova`@`%`                                                  |
      +-----------------------------------------------------------------------------------------------------+

  *Have access to databases remotely from compute node:*

      user@compute001:~$ mysql -unova -popenstack -h controller001
      
      MariaDB [(none)]> show databases;
      +--------------------+
      | Database           |
      +--------------------+
      | information_schema |
      | nova               |
      | nova_api           |
      | nova_cell0         |
      +--------------------+


  
  [OpenStack Installation Guide]:https://docs.openstack.org/install-guide/
  [My-Github]:https://github.com/hojat-gazestani/openstack
  [solution]:https://www.tikalk.com/posts/2012/06/25/solution-mysql-error-1045-access-denied-for-userlocalhost-breaks-openstack/

  *But there is a problem here:*

      user@controller001:~$ mysql
      ERROR 1698 (28000): Access denied for user 'user'@'localhost'

      user@controller001:~$ sudo mysql    
      MariaDB [(none)]>

  and:

      user@controller001:~$ mysqladmin -uroot ping
      mysqladmin: connect to server at 'localhost' failed

      error: 'Access denied for user 'root'@'localhost''
      user@controller001:~$ sudo mysqladmin -uroot ping
      mysqld is alive

  
  *Any try to discover new host make the below error:*

      user@controller001:~$ sudo su -s /bin/sh -c "nova-manage cell_v2 discover_hosts --verbose" nova
      ....
      sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1045, "Access denied for user 'nova'@'controller001' (using password: YES)")
      (Background on this error at: http://sqlalche.me/e/13/e3q8)

  
  Expected result
  ===============
  I am trying to add compute node to controller but it said to report this bug

  
  Actual result
  =============
      user@controller001:~$ sudo su -s /bin/sh -c "nova-manage cell_v2 discover_hosts --verbose" nova
      Found 2 cell mappings.
      Skipping cell0 since it does not contain hosts.
      Getting computes from cell 'cell1': 7466356a-3636-4698-af1c-fd64818ce722
      An error has occurred:
      Traceback (most recent call last):
        File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 2338, in _wrap_pool_connect
          return fn()
        File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 304, in unique_connection
          return _ConnectionFairy._checkout(self)
        File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 778, in _checkout
          fairy = _ConnectionRecord.checkout(pool)
        File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 495, in checkout
          rec = pool._do_get()
        File "/usr/lib/python3/dist-packages/sqlalchemy/pool/impl.py", line 140, in _do_get
          self._dec_overflow()
        File "/usr/lib/python3/dist-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
          compat.raise_(
        File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 182, in raise_
          raise exception
        File "/usr/lib/python3/dist-packages/sqlalchemy/pool/impl.py", line 137, in _do_get
          return self._create_connection()
        File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 309, in _create_connection
          return _ConnectionRecord(self)
        File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 440, in __init__
          self.__connect(first_connect_check=True)
        File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 661, in __connect
          pool.logger.debug("Error on connect(): %s", e)
        File "/usr/lib/python3/dist-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
          compat.raise_(
        File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 182, in raise_
          raise exception
        File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 656, in __connect
          connection = pool._invoke_creator(self)
        File "/usr/lib/python3/dist-packages/sqlalchemy/engine/strategies.py", line 114, in connect
          return dialect.connect(*cargs, **cparams)
        File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 493, in connect
          return self.dbapi.connect(*cargs, **cparams)
        File "/usr/lib/python3/dist-packages/pymysql/__init__.py", line 94, in Connect
          return Connection(*args, **kwargs)
        File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 325, in __init__
          self.connect()
        File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 599, in connect
          self._request_authentication()
        File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 861, in _request_authentication
          auth_packet = self._read_packet()
        File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 684, in _read_packet
          packet.check_error()
        File "/usr/lib/python3/dist-packages/pymysql/protocol.py", line 220, in check_error
          err.raise_mysql_exception(self._data)
        File "/usr/lib/python3/dist-packages/pymysql/err.py", line 109, in raise_mysql_exception
          raise errorclass(errno, errval)
      pymysql.err.OperationalError: (1045, "Access denied for user 'nova'@'controller001' (using password: YES)")
      
      The above exception was the direct cause of the following exception:
      
      Traceback (most recent call last):
        File "/usr/lib/python3/dist-packages/nova/cmd/manage.py", line 2793, in main
          ret = fn(*fn_args, **fn_kwargs)
        File "/usr/lib/python3/dist-packages/nova/cmd/manage.py", line 1138, in discover_hosts
          hosts = host_mapping_obj.discover_hosts(ctxt, cell_uuid, status_fn,
        File "/usr/lib/python3/dist-packages/nova/objects/host_mapping.py", line 272, in discover_hosts
          added_hm = _check_and_create_host_mappings(cctxt, cm, status_fn,
        File "/usr/lib/python3/dist-packages/nova/objects/host_mapping.py", line 234, in _check_and_create_host_mappings
          compute_nodes = objects.ComputeNodeList.get_all_by_not_mapped(
        File "/usr/lib/python3/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper
          result = fn(cls, context, *args, **kwargs)
        File "/usr/lib/python3/dist-packages/nova/objects/compute_node.py", line 425, in get_all_by_not_mapped
          db_computes = db.compute_node_get_all_mapped_less_than(
        File "/usr/lib/python3/dist-packages/nova/db/api.py", line 280, in compute_node_get_all_mapped_less_than
          return IMPL.compute_node_get_all_mapped_less_than(context,
        File "/usr/lib/python3/dist-packages/nova/db/sqlalchemy/api.py", line 237, in wrapped
          with ctxt_mgr.reader.using(context):
        File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
          return next(self.gen)
        File "/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1058, in _transaction_scope
          with current._produce_block(
        File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
          return next(self.gen)
        File "/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 657, in _session
          self.session = self.factory._create_session(
        File "/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 418, in _create_session
          self._start()
        File "/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 508, in _start
          self._setup_for_connection(
        File "/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 533, in _setup_for_connection
          engine = engines.create_engine(
        File "/usr/lib/python3/dist-packages/debtcollector/renames.py", line 43, in decorator
          return wrapped(*args, **kwargs)
        File "/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/engines.py", line 201, in create_engine
          test_conn = _test_connection(engine, max_retries, retry_interval)
        File "/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/engines.py", line 376, in _test_connection
          return engine.connect()
        File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 2265, in connect
          return self._connection_cls(self, **kwargs)
        File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 104, in __init__
          else engine.raw_connection()
        File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 2371, in raw_connection
          return self._wrap_pool_connect(
        File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 2341, in _wrap_pool_connect
          Connection._handle_dbapi_exception_noconnection(
        File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1581, in _handle_dbapi_exception_noconnection
          util.raise_(newraise, with_traceback=exc_info[2], from_=e)
        File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 182, in raise_
          raise exception
        File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 2338, in _wrap_pool_connect
          return fn()
        File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 304, in unique_connection
          return _ConnectionFairy._checkout(self)
        File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 778, in _checkout
          fairy = _ConnectionRecord.checkout(pool)
        File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 495, in checkout
          rec = pool._do_get()
        File "/usr/lib/python3/dist-packages/sqlalchemy/pool/impl.py", line 140, in _do_get
          self._dec_overflow()
        File "/usr/lib/python3/dist-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
          compat.raise_(
        File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 182, in raise_
          raise exception
        File "/usr/lib/python3/dist-packages/sqlalchemy/pool/impl.py", line 137, in _do_get
          return self._create_connection()
        File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 309, in _create_connection
          return _ConnectionRecord(self)
        File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 440, in __init__
          self.__connect(first_connect_check=True)
        File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 661, in __connect
          pool.logger.debug("Error on connect(): %s", e)
        File "/usr/lib/python3/dist-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
          compat.raise_(
        File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 182, in raise_
          raise exception
        File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 656, in __connect
          connection = pool._invoke_creator(self)
        File "/usr/lib/python3/dist-packages/sqlalchemy/engine/strategies.py", line 114, in connect
          return dialect.connect(*cargs, **cparams)
        File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 493, in connect
          return self.dbapi.connect(*cargs, **cparams)
        File "/usr/lib/python3/dist-packages/pymysql/__init__.py", line 94, in Connect
          return Connection(*args, **kwargs)
        File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 325, in __init__
          self.connect()
        File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 599, in connect
          self._request_authentication()
        File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 861, in _request_authentication
          auth_packet = self._read_packet()
        File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 684, in _read_packet
          packet.check_error()
        File "/usr/lib/python3/dist-packages/pymysql/protocol.py", line 220, in check_error
          err.raise_mysql_exception(self._data)
        File "/usr/lib/python3/dist-packages/pymysql/err.py", line 109, in raise_mysql_exception
          raise errorclass(errno, errval)
      sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1045, "Access denied for user 'nova'@'controller001' (using password: YES)")
      (Background on this error at: http://sqlalche.me/e/13/e3q8)

  user@controller001:~$ os hypervisor list
  Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'sqlalchemy.exc.OperationalError'> (HTTP 500) (Request-ID: req-d6edcfa3-ccf2-4eb8-80f8-041cb55e772d)


  
  Environment
  ===========

  1- Openstack victoria on Ubuntu 20.04.1

  2-

      user@controller001:~$ dpkg -l | grep nova
      ii  nova-api                             2:22.2.2-0ubuntu1~cloud1.1                           all          OpenStack Compute - API frontend
      ii  nova-common                          2:22.2.2-0ubuntu1~cloud1.1                           all          OpenStack Compute - common files
      ii  nova-conductor                       2:22.2.2-0ubuntu1~cloud1.1                           all          OpenStack Compute - conductor service
      ii  nova-novncproxy                      2:22.2.2-0ubuntu1~cloud1.1                           all          OpenStack Compute - NoVNC proxy
      ii  nova-scheduler                       2:22.2.2-0ubuntu1~cloud1.1                           all          OpenStack Compute - virtual machine scheduler
      ii  python3-nova                         2:22.2.2-0ubuntu1~cloud1.1                           all          OpenStack Compute Python 3 libraries
      ii  python3-novaclient                   2:17.0.0-0ubuntu1                                    all          client library for OpenStack Compute API - 3.x

  
  3- hypervisor is qemo

  4- Networking is next step and not configred, using management IP

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1945538/+subscriptions



References