yahoo-eng-team team mailing list archive

[Richard Jones <r1chardj0n3s@xxxxxxxxx>]

Public bug reported:

While writing the patch https://review.openstack.org/#/c/224431/ I
noticed that a separate test failed if I did not str() the filter
values. The failing test is
nova.tests.unit.compute.test_compute.ComputeAPITestCase.test_get_all_by_state
and I'll attach the output, but it is also at this test output URL:

http://logs.openstack.org/31/224431/1/check/gate-nova-python27/41a4c00/console.html#_2015-09-17_05_47_20_844
(also at http://paste.openstack.org/show/467079/)

The code is in nova/db/sqlalchemy/api.py in the _regex_instance_filter
function.  It would make sense to move non-string columns off to a
separate filter, rather than applying a REGEX to them.

** Affects: horizon
     Importance: Undecided
         Status: New

** Description changed:

  While writing the patch https://review.openstack.org/#/c/224431/ I
  noticed that a separate test failed if I did not str() the filter
  values. The failing test is
  nova.tests.unit.compute.test_compute.ComputeAPITestCase.test_get_all_by_state
- and the output of it is below, but also at this test output URL:
+ and I'll attach the output, but it is also at this test output URL:
  
  http://logs.openstack.org/31/224431/1/check/gate-nova-python27/41a4c00/console.html#_2015-09-17_05_47_20_844
  (also at http://paste.openstack.org/show/467079/)
  
  The code is in nova/db/sqlalchemy/api.py in the _regex_instance_filter
  function.  It would make sense to move non-string columns off to a
  separate filter, rather than applying a REGEX to them.
- 
- 
- 2015-09-17 05:47:20.844 | nova.tests.unit.compute.test_compute.ComputeAPITestCase.test_get_all_by_state
- 2015-09-17 05:47:20.844 | -----------------------------------------------------------------------------
- 2015-09-17 05:47:20.844 | 
- 2015-09-17 05:47:20.844 | Captured pythonlogging:
- 2015-09-17 05:47:20.844 | ~~~~~~~~~~~~~~~~~~~~~~~
- 2015-09-17 05:47:20.844 |     2015-09-17 05:45:10,858 INFO [nova.virt.driver] Loading compute driver 'nova.virt.fake.SmallFakeDriver'
- 2015-09-17 05:47:20.844 |     2015-09-17 05:45:10,858 WARNING [nova.compute.monitors] Excluding nova.compute.monitors.cpu monitor virt_driver. Not in the list of enabled monitors (CONF.compute_monitors).
- 2015-09-17 05:47:20.844 |     2015-09-17 05:45:10,859 INFO [nova.compute.resource_tracker] Auditing locally available compute resources for node fakenode1
- 2015-09-17 05:47:20.844 |     2015-09-17 05:45:10,866 WARNING [nova.compute.resource_tracker] No compute node record for fake-mini:fakenode1
- 2015-09-17 05:47:20.844 |     2015-09-17 05:45:10,869 INFO [nova.compute.resource_tracker] Compute_service record created for fake-mini:fakenode1
- 2015-09-17 05:47:20.845 |     2015-09-17 05:45:10,906 INFO [nova.compute.resource_tracker] Total usable vcpus: 1, total allocated vcpus: 0
- 2015-09-17 05:47:20.845 |     2015-09-17 05:45:10,906 INFO [nova.compute.resource_tracker] Final resource view: name=fakenode1 phys_ram=8192MB used_ram=512MB phys_disk=1028GB used_disk=0GB total_vcpus=1 used_vcpus=0 pci_stats=PciDevicePoolList(objects=[])
- 2015-09-17 05:47:20.845 |     2015-09-17 05:45:10,907 INFO [nova.compute.resource_tracker] Compute_service record updated for fake-mini:fakenode1
- 2015-09-17 05:47:20.845 |     2015-09-17 05:45:10,907 INFO [nova.compute.manager] Deleting orphan compute node 2
- 2015-09-17 05:47:20.845 |     
- 2015-09-17 05:47:20.845 | 
- 2015-09-17 05:47:20.845 | Captured traceback:
- 2015-09-17 05:47:20.845 | ~~~~~~~~~~~~~~~~~~~
- 2015-09-17 05:47:20.845 |     Traceback (most recent call last):
- 2015-09-17 05:47:20.845 |       File "nova/tests/unit/compute/test_compute.py", line 8248, in test_get_all_by_state
- 2015-09-17 05:47:20.845 |         search_opts={'power_state': power_state.SUSPENDED})
- 2015-09-17 05:47:20.846 |       File "nova/compute/api.py", line 2115, in get_all
- 2015-09-17 05:47:20.846 |         sort_keys=sort_keys, sort_dirs=sort_dirs)
- 2015-09-17 05:47:20.846 |       File "nova/compute/api.py", line 2165, in _get_instances_by_filters
- 2015-09-17 05:47:20.846 |         expected_attrs=fields, sort_keys=sort_keys, sort_dirs=sort_dirs)
- 2015-09-17 05:47:20.846 |       File "/home/jenkins/workspace/gate-nova-python27/.tox/py27/local/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 171, in wrapper
- 2015-09-17 05:47:20.846 |         result = fn(cls, context, *args, **kwargs)
- 2015-09-17 05:47:20.846 |       File "nova/objects/instance.py", line 1117, in get_by_filters
- 2015-09-17 05:47:20.846 |         use_slave=use_slave)
- 2015-09-17 05:47:20.846 |       File "nova/db/api.py", line 671, in instance_get_all_by_filters
- 2015-09-17 05:47:20.846 |         use_slave=use_slave)
- 2015-09-17 05:47:20.846 |       File "nova/db/sqlalchemy/api.py", line 216, in wrapper
- 2015-09-17 05:47:20.846 |         return f(*args, **kwargs)
- 2015-09-17 05:47:20.847 |       File "nova/db/sqlalchemy/api.py", line 1875, in instance_get_all_by_filters
- 2015-09-17 05:47:20.847 |         sort_dirs=[sort_dir])
- 2015-09-17 05:47:20.847 |       File "nova/db/sqlalchemy/api.py", line 216, in wrapper
- 2015-09-17 05:47:20.847 |         return f(*args, **kwargs)
- 2015-09-17 05:47:20.847 |       File "nova/db/sqlalchemy/api.py", line 2065, in instance_get_all_by_filters_sort
- 2015-09-17 05:47:20.847 |         return _instances_fill_metadata(context, query_prefix.all(), manual_joins)
- 2015-09-17 05:47:20.847 |       File "/home/jenkins/workspace/gate-nova-python27/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2399, in all
- 2015-09-17 05:47:20.847 |         return list(self)
- 2015-09-17 05:47:20.847 |       File "/home/jenkins/workspace/gate-nova-python27/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2516, in __iter__
- 2015-09-17 05:47:20.847 |         return self._execute_and_instances(context)
- 2015-09-17 05:47:20.847 |       File "/home/jenkins/workspace/gate-nova-python27/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2531, in _execute_and_instances
- 2015-09-17 05:47:20.848 |         result = conn.execute(querycontext.statement, self._params)
- 2015-09-17 05:47:20.848 |       File "/home/jenkins/workspace/gate-nova-python27/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 914, in execute
- 2015-09-17 05:47:20.848 |         return meth(self, multiparams, params)
- 2015-09-17 05:47:20.848 |       File "/home/jenkins/workspace/gate-nova-python27/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
- 2015-09-17 05:47:20.848 |         return connection._execute_clauseelement(self, multiparams, params)
- 2015-09-17 05:47:20.848 |       File "/home/jenkins/workspace/gate-nova-python27/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
- 2015-09-17 05:47:20.848 |         compiled_sql, distilled_params
- 2015-09-17 05:47:20.848 |       File "/home/jenkins/workspace/gate-nova-python27/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
- 2015-09-17 05:47:20.848 |         context)
- 2015-09-17 05:47:20.849 |       File "/home/jenkins/workspace/gate-nova-python27/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1337, in _handle_dbapi_exception
- 2015-09-17 05:47:20.849 |         util.raise_from_cause(newraise, exc_info)
- 2015-09-17 05:47:20.849 |       File "/home/jenkins/workspace/gate-nova-python27/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
- 2015-09-17 05:47:20.849 |         reraise(type(exception), exception, tb=exc_tb)
- 2015-09-17 05:47:20.849 |       File "/home/jenkins/workspace/gate-nova-python27/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
- 2015-09-17 05:47:20.849 |         context)
- 2015-09-17 05:47:20.849 |       File "/home/jenkins/workspace/gate-nova-python27/.tox/py27/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
- 2015-09-17 05:47:20.849 |         cursor.execute(statement, parameters)
- 2015-09-17 05:47:20.850 |     sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) user-defined function raised exception [SQL: u'SELECT instances.created_at AS instances_created_at, instances.updated_at AS instances_updated_at, instances.deleted_at AS instances_deleted_at, instances.deleted AS instances_deleted, instances.id AS instances_id, instances.user_id AS instances_user_id, instances.project_id AS instances_project_id, instances.image_ref AS instances_image_ref, instances.kernel_id AS instances_kernel_id, instances.ramdisk_id AS instances_ramdisk_id, instances.hostname AS instances_hostname, instances.launch_index AS instances_launch_index, instances.key_name AS instances_key_name, instances.key_data AS instances_key_data, instances.power_state AS instances_power_state, instances.vm_state AS instances_vm_state, instances.task_state AS instances_task_state, instances.memory_mb AS instances_memory_mb, instances.vcpus AS instances_vcpus, instances.root_gb AS instances_root_gb, instances.ephemeral_gb AS instances_ephemeral_gb, instances.ephemeral_key_uuid AS instances_ephemeral_key_uuid, instances.host AS instances_host, instances.node AS instances_node, instances.instance_type_id AS instances_instance_type_id, instances.user_data AS instances_user_data, instances.reservation_id AS instances_reservation_id, instances.launched_at AS instances_launched_at, instances.terminated_at AS instances_terminated_at, instances.availability_zone AS instances_availability_zone, instances.display_name AS instances_display_name, instances.display_description AS instances_display_description, instances.launched_on AS instances_launched_on, instances.locked AS instances_locked, instances.locked_by AS instances_locked_by, instances.os_type AS instances_os_type, instances.architecture AS instances_architecture, instances.vm_mode AS instances_vm_mode, instances.uuid AS instances_uuid, instances.root_device_name AS instances_root_device_name, instances.default_ephemeral_device AS instances_default_ephemeral_device, instances.default_swap_device AS instances_default_swap_device, instances.config_drive AS instances_config_drive, instances.access_ip_v4 AS instances_access_ip_v4, instances.access_ip_v6 AS instances_access_ip_v6, instances.auto_disk_config AS instances_auto_disk_config, instances.progress AS instances_progress, instances.shutdown_terminate AS instances_shutdown_terminate, instances.disable_terminate AS instances_disable_terminate, instances.cell_name AS instances_cell_name, instances.internal_id AS instances_internal_id, instances.cleaned AS instances_cleaned, instance_info_caches_1.created_at AS instance_info_caches_1_created_at, instance_info_caches_1.updated_at AS instance_info_caches_1_updated_at, instance_info_caches_1.deleted_at AS instance_info_caches_1_deleted_at, instance_info_caches_1.deleted AS instance_info_caches_1_deleted, instance_info_caches_1.id AS instance_info_caches_1_id, instance_info_caches_1.network_info AS instance_info_caches_1_network_info, instance_info_caches_1.instance_uuid AS instance_info_caches_1_instance_uuid, anon_1.security_groups_1_created_at AS security_groups_1_created_at, anon_1.security_groups_1_updated_at AS security_groups_1_updated_at, anon_1.security_groups_1_deleted_at AS security_groups_1_deleted_at, anon_1.security_groups_1_deleted AS security_groups_1_deleted, anon_1.security_groups_1_id AS security_groups_1_id, anon_1.security_groups_1_name AS security_groups_1_name, anon_1.security_groups_1_description AS security_groups_1_description, anon_1.security_groups_1_user_id AS security_groups_1_user_id, anon_1.security_groups_1_project_id AS security_groups_1_project_id \nFROM instances LEFT OUTER JOIN instance_info_caches AS instance_info_caches_1 ON instance_info_caches_1.instance_uuid = instances.uuid LEFT OUTER JOIN (SELECT security_group_instance_association_1.created_at AS security_group_instance_association_1_created_at, security_group_instance_association_1.updated_at AS security_group_instance_association_1_updated_at, security_group_instance_association_1.deleted_at AS security_group_instance_association_1_deleted_at, security_group_instance_association_1.deleted AS security_group_instance_association_1_deleted, security_group_instance_association_1.id AS security_group_instance_association_1_id, security_group_instance_association_1.security_group_id AS security_group_instance_association_1_security_group_id, security_group_instance_association_1.instance_uuid AS security_group_instance_association_1_instance_uuid, security_groups_1.created_at AS security_groups_1_created_at, security_groups_1.updated_at AS security_groups_1_updated_at, security_groups_1.deleted_at AS security_groups_1_deleted_at, security_groups_1.deleted AS security_groups_1_deleted, security_groups_1.id AS security_groups_1_id, security_groups_1.name AS security_groups_1_name, security_groups_1.description AS security_groups_1_description, security_groups_1.user_id AS security_groups_1_user_id, security_groups_1.project_id AS security_groups_1_project_id \nFROM security_group_instance_association AS security_group_instance_association_1 JOIN security_groups AS security_groups_1 ON security_groups_1.id = security_group_instance_association_1.security_group_id AND security_group_instance_association_1.deleted = ? AND security_groups_1.deleted = ?) AS anon_1 ON anon_1.security_group_instance_association_1_instance_uuid = instances.uuid AND instances.deleted = ? \nWHERE instances.power_state REGEXP ? ORDER BY instances.created_at DESC, instances.id DESC'] [parameters: (0, 0, 0, 7)]
- 2015-09-17 05:47:20.850 |

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Dashboard (Horizon).
https://bugs.launchpad.net/bugs/1497057

Title:
  DB filtering applies REGEX to non-string columns

Status in OpenStack Dashboard (Horizon):
  New

Bug description:
  While writing the patch https://review.openstack.org/#/c/224431/ I
  noticed that a separate test failed if I did not str() the filter
  values. The failing test is
  nova.tests.unit.compute.test_compute.ComputeAPITestCase.test_get_all_by_state
  and I'll attach the output, but it is also at this test output URL:

  http://logs.openstack.org/31/224431/1/check/gate-nova-python27/41a4c00/console.html#_2015-09-17_05_47_20_844
  (also at http://paste.openstack.org/show/467079/)

  The code is in nova/db/sqlalchemy/api.py in the _regex_instance_filter
  function.  It would make sense to move non-string columns off to a
  separate filter, rather than applying a REGEX to them.

To manage notifications about this bug go to:
https://bugs.launchpad.net/horizon/+bug/1497057/+subscriptions