openstack team mailing list archive

Thread
Date
Re: Nova Compute Errors

To: Razique Mahroua <razique.mahroua@xxxxxxxxx>
From: Rahul Sharma <rahulsharmaait@xxxxxxxxx>
Date: Thu, 23 May 2013 16:27:20 +0530
Cc: openstack@xxxxxxxxxxxxxxxxxxx
In-reply-to: <CAAw0FvTyEqAtQaGscji4sxKfTdXgEDwWnS+aYqe-Ht_qFAWZMA@mail.gmail.com>
Correction: the table_name is services not instances.

-Regards
Rahul


On Thu, May 23, 2013 at 3:45 PM, Rahul Sharma <rahulsharmaait@xxxxxxxxx>wrote:

> Hi Trinath,
>
> I don't know how you have landed to this situation, mostly would be
> because of upgrade. What had happened is that in *Nova* databases's *
> instances* table, column  *availability_zones* is not present(was there
> in earlier releases). You might be having the latest database and old
> nova-code.  Due to this, sqlalchemy is sending commands to fetch values
> based on availability_zones column but the table in database don't have
> that column. Due to this,  you are hitting the exception.
>
> One solution is to manually add the extra column to the database to
> support the old code. But chances are that there will be more tables in
> database in which you need to do this procedure again and again.
>
> Second solution is to have latest code for the Nova, that will solve the
> problem.
>
> Thanks and Regards
> Rahul Sharma
>
>
> On Thu, May 23, 2013 at 2:58 PM, Razique Mahroua <
> razique.mahroua@xxxxxxxxx> wrote:
>
>> Did you run an upgrade?
>> try maybe :
>> $nova-manage db sync
>>
>> (make a backup first)
>>
>> regards,
>>
>> *Razique Mahroua** - **Nuage & Co*
>> razique.mahroua@xxxxxxxxx
>> Tel : +33 9 72 37 94 15
>>
>>
>> Le 23 mai 2013 à 11:19, Trinath Somanchi <trinath.somanchi@xxxxxxxxx> a
>> écrit :
>>
>> Hi Stackers-
>>
>> I have installed nova-compute from the source code, with command, "python
>> setup.py install".
>>
>> Also, configured the configuration files.
>>
>> When I start the "nova-compute" service, I get the following issues.
>>
>> 2013-05-23 14:37:27 3124 AUDIT nova.service [-] Starting compute node
>> (version 2013.1-LOCALBRANCH:LOCALREVISION)
>> 2013-05-23 14:37:27 3124 DEBUG nova.virt.libvirt.driver [-] Connecting to
>> libvirt: qemu:///system _get_connection
>> /usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/virt/libvirt/driver.py:340
>> 2013-05-23 14:37:28 INFO nova.compute.manager
>> [req-f5cbe8c7-1fba-430d-965d-e6f72cef1ad2 None None] Updating host status
>> 2013-05-23 14:37:28 DEBUG nova.virt.libvirt.driver
>> [req-f5cbe8c7-1fba-430d-965d-e6f72cef1ad2 None None] Updating host stats
>> update_status
>> /usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/virt/libvirt/driver.py:3039
>> 2013-05-23 14:37:28 DEBUG nova.virt.libvirt.driver
>> [req-f5cbe8c7-1fba-430d-965d-e6f72cef1ad2 None None] Connecting to libvirt:
>> qemu:///system _get_connection
>> /usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/virt/libvirt/driver.py:340
>> 2013-05-23 14:37:28 DEBUG nova.virt.libvirt.driver
>> [req-f5cbe8c7-1fba-430d-965d-e6f72cef1ad2 None None] Updating host stats
>> update_status
>> /usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/virt/libvirt/driver.py:3039
>> 2013-05-23 14:37:29 DEBUG nova.manager
>> [req-f5cbe8c7-1fba-430d-965d-e6f72cef1ad2 None None] Notifying Schedulers
>> of capabilities ... publish_service_capabilities
>> /usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/manager.py:256
>> 2013-05-23 14:37:29 3124 DEBUG nova.openstack.common.rpc.amqp [-] Making
>> asynchronous fanout cast... fanout_cast
>> /usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/openstack/common/rpc/amqp.py:385
>> 2013-05-23 14:37:29 3124 DEBUG nova.openstack.common.rpc.amqp [-] Pool
>> creating new connection create
>> /usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/openstack/common/rpc/amqp.py:58
>> 2013-05-23 14:37:29 3124 INFO nova.openstack.common.rpc.common [-]
>> Connected to AMQP server on 10.232.91.81:5672
>> Traceback (most recent call last):
>>   File
>> "/usr/lib/python2.7/site-packages/eventlet-0.12.1-py2.7.egg/eventlet/hubs/hub.py",
>> line 346, in fire_timers
>>     timer()
>>   File
>> "/usr/lib/python2.7/site-packages/eventlet-0.12.1-py2.7.egg/eventlet/hubs/timer.py",
>> line 56, in __call__
>>     cb(*args, **kw)
>>   File
>> "/usr/lib/python2.7/site-packages/eventlet-0.12.1-py2.7.egg/eventlet/greenthread.py",
>> line 194, in main
>>     result = function(*args, **kwargs)
>>   File
>> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/service.py",
>> line 124, in run_server
>>     server.start()
>>   File
>> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/service.py",
>> line 399, in start
>>     self.binary)
>>   File
>> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/db/api.py",
>> line 158, in service_get_by_args
>>     return IMPL.service_get_by_args(context, host, binary)
>>   File
>> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/db/sqlalchemy/api.py",
>> line 107, in wrapper
>>     return f(*args, **kwargs)
>>   File
>> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/db/sqlalchemy/api.py",
>> line 411, in service_get_by_args
>>     filter_by(binary=binary).\
>>   File
>> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/orm/query.py",
>> line 2156, in first
>>     ret = list(self[0:1])
>>   File
>> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/orm/query.py",
>> line 2023, in __getitem__
>>     return list(res)
>>   File
>> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/orm/query.py",
>> line 2227, in __iter__
>>     return self._execute_and_instances(context)
>>   File
>> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/orm/query.py",
>> line 2242, in _execute_and_instances
>>     result = conn.execute(querycontext.statement, self._params)
>>   File
>> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/engine/base.py",
>> line 1449, in execute
>>     params)
>>   File
>> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/engine/base.py",
>> line 1584, in _execute_clauseelement
>>     compiled_sql, distilled_params
>>   File
>> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/engine/base.py",
>> line 1698, in _execute_context
>>     context)
>>   File
>> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/engine/base.py",
>> line 1850, in _handle_dbapi_exception
>>     None, sys.exc_info()[2]
>> OperationalError: (OperationalError) (1054, "Unknown column
>> 'services.availability_zone' in 'field list'") 'SELECT services.created_at
>> AS services_created_at, services.updated_at AS services_updated_at,
>> services.deleted_at AS services_deleted_at, services.deleted AS
>> services_deleted, services.id AS services_id, services.host AS
>> services_host, services.`binary` AS services_binary, services.topic AS
>> services_topic, services.report_count AS services_report_count,
>> services.disabled AS services_disabled, services.availability_zone AS
>> services_availability_zone \nFROM services \nWHERE services.deleted = %s
>> AND services.host = %s AND services.`binary` = %s \n LIMIT %s' (0,
>> 'ComputeNode-1', 'nova-compute', 1)
>> 2013-05-23 14:37:29 3124 CRITICAL nova [-] (OperationalError) (1054,
>> "Unknown column 'services.availability_zone' in 'field list'") 'SELECT
>> services.created_at AS services_created_at, services.updated_at AS
>> services_updated_at, services.deleted_at AS services_deleted_at,
>> services.deleted AS services_deleted, services.id AS services_id,
>> services.host AS services_host, services.`binary` AS services_binary,
>> services.topic AS services_topic, services.report_count AS
>> services_report_count, services.disabled AS services_disabled,
>> services.availability_zone AS services_availability_zone \nFROM services
>> \nWHERE services.deleted = %s AND services.host = %s AND services.`binary`
>> = %s \n LIMIT %s' (0, 'ComputeNode-1', 'nova-compute', 1)
>> 2013-05-23 14:37:29 3124 TRACE nova Traceback (most recent call last):
>> 2013-05-23 14:37:29 3124 TRACE nova   File "/usr/bin/nova-compute", line
>> 5, in <module>
>> 2013-05-23 14:37:29 3124 TRACE nova
>> pkg_resources.run_script('nova==2013.1', 'nova-compute')
>> 2013-05-23 14:37:29 3124 TRACE nova   File
>> "/usr/lib/python2.7/site-packages/pkg_resources.py", line 505, in run_script
>> 2013-05-23 14:37:29 3124 TRACE nova
>> self.require(requires)[0].run_script(script_name, ns)
>> 2013-05-23 14:37:29 3124 TRACE nova   File
>> "/usr/lib/python2.7/site-packages/pkg_resources.py", line 1245, in
>> run_script
>> 2013-05-23 14:37:29 3124 TRACE nova     execfile(script_filename,
>> namespace, namespace)
>> 2013-05-23 14:37:29 3124 TRACE nova   File
>> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/EGG-INFO/scripts/nova-compute",
>> line 60, in <module>
>> 2013-05-23 14:37:29 3124 TRACE nova     service.wait()
>> 2013-05-23 14:37:29 3124 TRACE nova   File
>> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/service.py",
>> line 667, in wait
>> 2013-05-23 14:37:29 3124 TRACE nova     _launcher.wait()
>> 2013-05-23 14:37:29 3124 TRACE nova   File
>> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/service.py",
>> line 186, in wait
>> 2013-05-23 14:37:29 3124 TRACE nova     super(ServiceLauncher,
>> self).wait()
>> 2013-05-23 14:37:29 3124 TRACE nova   File
>> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/service.py",
>> line 156, in wait
>> 2013-05-23 14:37:29 3124 TRACE nova     service.wait()
>>  2013-05-23 14:37:29 3124 TRACE nova   File
>> "/usr/lib/python2.7/site-packages/eventlet-0.12.1-py2.7.egg/eventlet/greenthread.py",
>> line 168, in wait
>> 2013-05-23 14:37:29 3124 TRACE nova     return self._exit_event.wait()
>> 2013-05-23 14:37:29 3124 TRACE nova   File
>> "/usr/lib/python2.7/site-packages/eventlet-0.12.1-py2.7.egg/eventlet/event.py",
>> line 116, in wait
>> 2013-05-23 14:37:29 3124 TRACE nova     return hubs.get_hub().switch()
>> 2013-05-23 14:37:29 3124 TRACE nova   File
>> "/usr/lib/python2.7/site-packages/eventlet-0.12.1-py2.7.egg/eventlet/hubs/hub.py",
>> line 187, in switch
>> 2013-05-23 14:37:29 3124 TRACE nova     return self.greenlet.switch()
>> 2013-05-23 14:37:29 3124 TRACE nova   File
>> "/usr/lib/python2.7/site-packages/eventlet-0.12.1-py2.7.egg/eventlet/greenthread.py",
>> line 194, in main
>> 2013-05-23 14:37:29 3124 TRACE nova     result = function(*args, **kwargs)
>> 2013-05-23 14:37:29 3124 TRACE nova   File
>> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/service.py",
>> line 124, in run_server
>> 2013-05-23 14:37:29 3124 TRACE nova     server.start()
>> 2013-05-23 14:37:29 3124 TRACE nova   File
>> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/service.py",
>> line 399, in start
>> 2013-05-23 14:37:29 3124 TRACE nova     self.binary)
>> 2013-05-23 14:37:29 3124 TRACE nova   File
>> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/db/api.py",
>> line 158, in service_get_by_args
>> 2013-05-23 14:37:29 3124 TRACE nova     return
>> IMPL.service_get_by_args(context, host, binary)
>> 2013-05-23 14:37:29 3124 TRACE nova   File
>> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/db/sqlalchemy/api.py",
>> line 107, in wrapper
>> 2013-05-23 14:37:29 3124 TRACE nova     return f(*args, **kwargs)
>> 2013-05-23 14:37:29 3124 TRACE nova   File
>> "/usr/lib/python2.7/site-packages/nova-2013.1-py2.7.egg/nova/db/sqlalchemy/api.py",
>> line 411, in service_get_by_args
>> 2013-05-23 14:37:29 3124 TRACE nova     filter_by(binary=binary).\
>> 2013-05-23 14:37:29 3124 TRACE nova   File
>> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/orm/query.py",
>> line 2156, in first
>> 2013-05-23 14:37:29 3124 TRACE nova     ret = list(self[0:1])
>> 2013-05-23 14:37:29 3124 TRACE nova   File
>> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/orm/query.py",
>> line 2023, in __getitem__
>> 2013-05-23 14:37:29 3124 TRACE nova     return list(res)
>> 2013-05-23 14:37:29 3124 TRACE nova   File
>> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/orm/query.py",
>> line 2227, in __iter__
>> 2013-05-23 14:37:29 3124 TRACE nova     return
>> self._execute_and_instances(context)
>> 2013-05-23 14:37:29 3124 TRACE nova   File
>> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/orm/query.py",
>> line 2242, in _execute_and_instances
>> 2013-05-23 14:37:29 3124 TRACE nova     result =
>> conn.execute(querycontext.statement, self._params)
>> 2013-05-23 14:37:29 3124 TRACE nova   File
>> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/engine/base.py",
>> line 1449, in execute
>> 2013-05-23 14:37:29 3124 TRACE nova     params)
>> 2013-05-23 14:37:29 3124 TRACE nova   File
>> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/engine/base.py",
>> line 1584, in _execute_clauseelement
>> 2013-05-23 14:37:29 3124 TRACE nova     compiled_sql, distilled_params
>> 2013-05-23 14:37:29 3124 TRACE nova   File
>> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/engine/base.py",
>> line 1698, in _execute_context
>> 2013-05-23 14:37:29 3124 TRACE nova     context)
>> 2013-05-23 14:37:29 3124 TRACE nova   File
>> "/usr/lib/python2.7/site-packages/SQLAlchemy-0.7.9-py2.7-linux-ppc.egg/sqlalchemy/engine/base.py",
>> line 1850, in _handle_dbapi_exception
>> 2013-05-23 14:37:29 3124 TRACE nova     None, sys.exc_info()[2]
>> 2013-05-23 14:37:29 3124 TRACE nova OperationalError: (OperationalError)
>> (1054, "Unknown column 'services.availability_zone' in 'field list'")
>> 'SELECT services.created_at AS services_created_at, services.updated_at AS
>> services_updated_at, services.deleted_at AS services_deleted_at,
>> services.deleted AS services_deleted, services.id AS services_id,
>> services.host AS services_host, services.`binary` AS services_binary,
>> services.topic AS services_topic, services.report_count AS
>> services_report_count, services.disabled AS services_disabled,
>> services.availability_zone AS services_availability_zone \nFROM services
>> \nWHERE services.deleted = %s AND services.host = %s AND services.`binary`
>> = %s \n LIMIT %s' (0, 'ComputeNode-1', 'nova-compute', 1)
>> 2013-05-23 14:37:29 3124 TRACE nova
>>
>> With respect to the above issue, I feel there exists some access to an
>> unknown column in the DB tables.
>>
>> The SERVICES table of the nova DB is as below
>>
>> mysql> desc services;
>> +--------------+--------------+------+-----+---------+----------------+
>> | Field        | Type         | Null | Key | Default | Extra          |
>> +--------------+--------------+------+-----+---------+----------------+
>> | created_at   | datetime     | YES  |     | NULL    |                |
>> | updated_at   | datetime     | YES  |     | NULL    |                |
>> | deleted_at   | datetime     | YES  |     | NULL    |                |
>> | id           | int(11)      | NO   | PRI | NULL    | auto_increment |
>> | host         | varchar(255) | YES  |     | NULL    |                |
>> | binary       | varchar(255) | YES  |     | NULL    |                |
>> | topic        | varchar(255) | YES  |     | NULL    |                |
>> | report_count | int(11)      | NO   |     | NULL    |                |
>> | disabled     | tinyint(1)   | YES  |     | NULL    |                |
>> | deleted      | int(11)      | YES  |     | NULL    |                |
>> +--------------+--------------+------+-----+---------+----------------+
>>
>> Kindly help me resolve the issue.
>>
>>
>>
>> --
>> Regards,
>> ----------------------------------------------
>> Trinath Somanchi,
>> +91 9866 235 130
>>  _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
References

Nova Compute Errors
From: Trinath Somanchi, 2013-05-23
Re: Nova Compute Errors
From: Razique Mahroua, 2013-05-23
Re: Nova Compute Errors
From: Rahul Sharma, 2013-05-23