openstack team mailing list archive

Thread
Date
Re: Nova Compute Errors

To: Razique Mahroua <razique.mahroua@xxxxxxxxx>
From: Rahul Sharma <rahulsharmaait@xxxxxxxxx>
Date: Thu, 23 May 2013 15:45:47 +0530
Cc: openstack@xxxxxxxxxxxxxxxxxxx
In-reply-to: <8A348203-CED0-4C0A-BA83-FEC518DA8954@gmail.com>
Hi Trinath,

I don't know how you have landed to this situation, mostly would be because
of upgrade. What had happened is that in *Nova* databases's
*instances*table, column
*availability_zones* is not present(was there in earlier releases). You
might be having the latest database and old nova-code.  Due to this,
sqlalchemy is sending commands to fetch values based on availability_zones
column but the table in database don't have that column. Due to this,  you
are hitting the exception.

One solution is to manually add the extra column to the database to support
the old code. But chances are that there will be more tables in database in
which you need to do this procedure again and again.

Second solution is to have latest code for the Nova, that will solve the
problem.

Thanks and Regards
Rahul Sharma


On Thu, May 23, 2013 at 2:58 PM, Razique Mahroua
<razique.mahroua@xxxxxxxxx>wrote:

> Did you run an upgrade?
> try maybe :
> $nova-manage db sync
>
> (make a backup first)
>
> regards,
>
> *Razique Mahroua** - **Nuage & Co*
> razique.mahroua@xxxxxxxxx
> Tel : +33 9 72 37 94 15
>
>
> Le 23 mai 2013 à 11:19, Trinath Somanchi <trinath.somanchi@xxxxxxxxx> a
> écrit :
>
> Hi Stackers-
>
> I have installed nova-compute from the source code, with command, "python
> setup.py install".
>
> Also, configured the configuration files.
>
> When I start the "nova-compute" service, I get the following issues.
>
> 2013-05-23 14:37:27 3124 AUDIT nova.service [-] Starting compute node
> (version 2013.1-LOCALBRANCH:LOCALREVISION)
> 2013-05-23 14:37:27 3124 DEBUG nova.virt.libvirt.driver [-] Connecting to
> libvirt: qemu:///system _get_connection
> /usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/virt/libvirt/driver.py:340
> 2013-05-23 14:37:28 INFO nova.compute.manager
> [req-f5cbe8c7-1fba-430d-965d-e6f72cef1ad2 None None] Updating host status
> 2013-05-23 14:37:28 DEBUG nova.virt.libvirt.driver
> [req-f5cbe8c7-1fba-430d-965d-e6f72cef1ad2 None None] Updating host stats
> update_status
> /usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/virt/libvirt/driver.py:3039
> 2013-05-23 14:37:28 DEBUG nova.virt.libvirt.driver
> [req-f5cbe8c7-1fba-430d-965d-e6f72cef1ad2 None None] Connecting to libvirt:
> qemu:///system _get_connection
> /usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/virt/libvirt/driver.py:340
> 2013-05-23 14:37:28 DEBUG nova.virt.libvirt.driver
> [req-f5cbe8c7-1fba-430d-965d-e6f72cef1ad2 None None] Updating host stats
> update_status
> /usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/virt/libvirt/driver.py:3039
> 2013-05-23 14:37:29 DEBUG nova.manager
> [req-f5cbe8c7-1fba-430d-965d-e6f72cef1ad2 None None] Notifying Schedulers
> of capabilities ... publish_service_capabilities
> /usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/manager.py:256
> 2013-05-23 14:37:29 3124 DEBUG nova.openstack.common.rpc.amqp [-] Making
> asynchronous fanout cast... fanout_cast
> /usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/openstack/common/rpc/amqp.py:385
> 2013-05-23 14:37:29 3124 DEBUG nova.openstack.common.rpc.amqp [-] Pool
> creating new connection create
> /usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/openstack/common/rpc/amqp.py:58
> 2013-05-23 14:37:29 3124 INFO nova.openstack.common.rpc.common [-]
> Connected to AMQP server on 10.232.91.81:5672
> Traceback (most recent call last):
>   File
> "/usr/lib/python2.7/site-packages/eventlet-0.12.1-py2.7.egg/eventlet/hubs/hub.py",
> line 346, in fire_timers
>     timer()
>   File
> "/usr/lib/python2.7/site-packages/eventlet-0.12.1-py2.7.egg/eventlet/hubs/timer.py",
> line 56, in __call__
>     cb(*args, **kw)
>   File
> "/usr/lib/python2.7/site-packages/eventlet-0.12.1-py2.7.egg/eventlet/greenthread.py",
> line 194, in main
>     result = function(*args, **kwargs)
>   File
> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/service.py",
> line 124, in run_server
>     server.start()
>   File
> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/service.py",
> line 399, in start
>     self.binary)
>   File
> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/db/api.py",
> line 158, in service_get_by_args
>     return IMPL.service_get_by_args(context, host, binary)
>   File
> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/db/sqlalchemy/api.py",
> line 107, in wrapper
>     return f(*args, **kwargs)
>   File
> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/db/sqlalchemy/api.py",
> line 411, in service_get_by_args
>     filter_by(binary=binary).\
>   File
> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/orm/query.py",
> line 2156, in first
>     ret = list(self[0:1])
>   File
> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/orm/query.py",
> line 2023, in __getitem__
>     return list(res)
>   File
> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/orm/query.py",
> line 2227, in __iter__
>     return self._execute_and_instances(context)
>   File
> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/orm/query.py",
> line 2242, in _execute_and_instances
>     result = conn.execute(querycontext.statement, self._params)
>   File
> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/engine/base.py",
> line 1449, in execute
>     params)
>   File
> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/engine/base.py",
> line 1584, in _execute_clauseelement
>     compiled_sql, distilled_params
>   File
> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/engine/base.py",
> line 1698, in _execute_context
>     context)
>   File
> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/engine/base.py",
> line 1850, in _handle_dbapi_exception
>     None, sys.exc_info()[2]
> OperationalError: (OperationalError) (1054, "Unknown column
> 'services.availability_zone' in 'field list'") 'SELECT services.created_at
> AS services_created_at, services.updated_at AS services_updated_at,
> services.deleted_at AS services_deleted_at, services.deleted AS
> services_deleted, services.id AS services_id, services.host AS
> services_host, services.`binary` AS services_binary, services.topic AS
> services_topic, services.report_count AS services_report_count,
> services.disabled AS services_disabled, services.availability_zone AS
> services_availability_zone \nFROM services \nWHERE services.deleted = %s
> AND services.host = %s AND services.`binary` = %s \n LIMIT %s' (0,
> 'ComputeNode-1', 'nova-compute', 1)
> 2013-05-23 14:37:29 3124 CRITICAL nova [-] (OperationalError) (1054,
> "Unknown column 'services.availability_zone' in 'field list'") 'SELECT
> services.created_at AS services_created_at, services.updated_at AS
> services_updated_at, services.deleted_at AS services_deleted_at,
> services.deleted AS services_deleted, services.id AS services_id,
> services.host AS services_host, services.`binary` AS services_binary,
> services.topic AS services_topic, services.report_count AS
> services_report_count, services.disabled AS services_disabled,
> services.availability_zone AS services_availability_zone \nFROM services
> \nWHERE services.deleted = %s AND services.host = %s AND services.`binary`
> = %s \n LIMIT %s' (0, 'ComputeNode-1', 'nova-compute', 1)
> 2013-05-23 14:37:29 3124 TRACE nova Traceback (most recent call last):
> 2013-05-23 14:37:29 3124 TRACE nova   File "/usr/bin/nova-compute", line
> 5, in <module>
> 2013-05-23 14:37:29 3124 TRACE nova
> pkg_resources.run_script('nova==2013.1', 'nova-compute')
> 2013-05-23 14:37:29 3124 TRACE nova   File
> "/usr/lib/python2.7/site-packages/pkg_resources.py", line 505, in run_script
> 2013-05-23 14:37:29 3124 TRACE nova
> self.require(requires)[0].run_script(script_name, ns)
> 2013-05-23 14:37:29 3124 TRACE nova   File
> "/usr/lib/python2.7/site-packages/pkg_resources.py", line 1245, in
> run_script
> 2013-05-23 14:37:29 3124 TRACE nova     execfile(script_filename,
> namespace, namespace)
> 2013-05-23 14:37:29 3124 TRACE nova   File
> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/EGG-INFO/scripts/nova-compute",
> line 60, in <module>
> 2013-05-23 14:37:29 3124 TRACE nova     service.wait()
> 2013-05-23 14:37:29 3124 TRACE nova   File
> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/service.py",
> line 667, in wait
> 2013-05-23 14:37:29 3124 TRACE nova     _launcher.wait()
> 2013-05-23 14:37:29 3124 TRACE nova   File
> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/service.py",
> line 186, in wait
> 2013-05-23 14:37:29 3124 TRACE nova     super(ServiceLauncher, self).wait()
> 2013-05-23 14:37:29 3124 TRACE nova   File
> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/service.py",
> line 156, in wait
> 2013-05-23 14:37:29 3124 TRACE nova     service.wait()
>  2013-05-23 14:37:29 3124 TRACE nova   File
> "/usr/lib/python2.7/site-packages/eventlet-0.12.1-py2.7.egg/eventlet/greenthread.py",
> line 168, in wait
> 2013-05-23 14:37:29 3124 TRACE nova     return self._exit_event.wait()
> 2013-05-23 14:37:29 3124 TRACE nova   File
> "/usr/lib/python2.7/site-packages/eventlet-0.12.1-py2.7.egg/eventlet/event.py",
> line 116, in wait
> 2013-05-23 14:37:29 3124 TRACE nova     return hubs.get_hub().switch()
> 2013-05-23 14:37:29 3124 TRACE nova   File
> "/usr/lib/python2.7/site-packages/eventlet-0.12.1-py2.7.egg/eventlet/hubs/hub.py",
> line 187, in switch
> 2013-05-23 14:37:29 3124 TRACE nova     return self.greenlet.switch()
> 2013-05-23 14:37:29 3124 TRACE nova   File
> "/usr/lib/python2.7/site-packages/eventlet-0.12.1-py2.7.egg/eventlet/greenthread.py",
> line 194, in main
> 2013-05-23 14:37:29 3124 TRACE nova     result = function(*args, **kwargs)
> 2013-05-23 14:37:29 3124 TRACE nova   File
> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/service.py",
> line 124, in run_server
> 2013-05-23 14:37:29 3124 TRACE nova     server.start()
> 2013-05-23 14:37:29 3124 TRACE nova   File
> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/service.py",
> line 399, in start
> 2013-05-23 14:37:29 3124 TRACE nova     self.binary)
> 2013-05-23 14:37:29 3124 TRACE nova   File
> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/db/api.py",
> line 158, in service_get_by_args
> 2013-05-23 14:37:29 3124 TRACE nova     return
> IMPL.service_get_by_args(context, host, binary)
> 2013-05-23 14:37:29 3124 TRACE nova   File
> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/db/sqlalchemy/api.py",
> line 107, in wrapper
> 2013-05-23 14:37:29 3124 TRACE nova     return f(*args, **kwargs)
> 2013-05-23 14:37:29 3124 TRACE nova   File
> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/db/sqlalchemy/api.py",
> line 411, in service_get_by_args
> 2013-05-23 14:37:29 3124 TRACE nova     filter_by(binary=binary).\
> 2013-05-23 14:37:29 3124 TRACE nova   File
> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/orm/query.py",
> line 2156, in first
> 2013-05-23 14:37:29 3124 TRACE nova     ret = list(self[0:1])
> 2013-05-23 14:37:29 3124 TRACE nova   File
> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/orm/query.py",
> line 2023, in __getitem__
> 2013-05-23 14:37:29 3124 TRACE nova     return list(res)
> 2013-05-23 14:37:29 3124 TRACE nova   File
> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/orm/query.py",
> line 2227, in __iter__
> 2013-05-23 14:37:29 3124 TRACE nova     return
> self._execute_and_instances(context)
> 2013-05-23 14:37:29 3124 TRACE nova   File
> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/orm/query.py",
> line 2242, in _execute_and_instances
> 2013-05-23 14:37:29 3124 TRACE nova     result =
> conn.execute(querycontext.statement, self._params)
> 2013-05-23 14:37:29 3124 TRACE nova   File
> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/engine/base.py",
> line 1449, in execute
> 2013-05-23 14:37:29 3124 TRACE nova     params)
> 2013-05-23 14:37:29 3124 TRACE nova   File
> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/engine/base.py",
> line 1584, in _execute_clauseelement
> 2013-05-23 14:37:29 3124 TRACE nova     compiled_sql, distilled_params
> 2013-05-23 14:37:29 3124 TRACE nova   File
> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/engine/base.py",
> line 1698, in _execute_context
> 2013-05-23 14:37:29 3124 TRACE nova     context)
> 2013-05-23 14:37:29 3124 TRACE nova   File
> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/engine/base.py",
> line 1850, in _handle_dbapi_exception
> 2013-05-23 14:37:29 3124 TRACE nova     None, sys.exc_info()[2]
> 2013-05-23 14:37:29 3124 TRACE nova OperationalError: (OperationalError)
> (1054, "Unknown column 'services.availability_zone' in 'field list'")
> 'SELECT services.created_at AS services_created_at, services.updated_at AS
> services_updated_at, services.deleted_at AS services_deleted_at,
> services.deleted AS services_deleted, services.id AS services_id,
> services.host AS services_host, services.`binary` AS services_binary,
> services.topic AS services_topic, services.report_count AS
> services_report_count, services.disabled AS services_disabled,
> services.availability_zone AS services_availability_zone \nFROM services
> \nWHERE services.deleted = %s AND services.host = %s AND services.`binary`
> = %s \n LIMIT %s' (0, 'ComputeNode-1', 'nova-compute', 1)
> 2013-05-23 14:37:29 3124 TRACE nova
>
> With respect to the above issue, I feel there exists some access to an
> unknown column in the DB tables.
>
> The SERVICES table of the nova DB is as below
>
> mysql> desc services;
> +--------------+--------------+------+-----+---------+----------------+
> | Field        | Type         | Null | Key | Default | Extra          |
> +--------------+--------------+------+-----+---------+----------------+
> | created_at   | datetime     | YES  |     | NULL    |                |
> | updated_at   | datetime     | YES  |     | NULL    |                |
> | deleted_at   | datetime     | YES  |     | NULL    |                |
> | id           | int(11)      | NO   | PRI | NULL    | auto_increment |
> | host         | varchar(255) | YES  |     | NULL    |                |
> | binary       | varchar(255) | YES  |     | NULL    |                |
> | topic        | varchar(255) | YES  |     | NULL    |                |
> | report_count | int(11)      | NO   |     | NULL    |                |
> | disabled     | tinyint(1)   | YES  |     | NULL    |                |
> | deleted      | int(11)      | YES  |     | NULL    |                |
> +--------------+--------------+------+-----+---------+----------------+
>
> Kindly help me resolve the issue.
>
>
>
> --
> Regards,
> ----------------------------------------------
> Trinath Somanchi,
> +91 9866 235 130
>  _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
Follow ups

Re: Nova Compute Errors
From: Rahul Sharma, 2013-05-23
References

Nova Compute Errors
From: Trinath Somanchi, 2013-05-23
Re: Nova Compute Errors
From: Razique Mahroua, 2013-05-23