yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1733917] [NEW] novaclient list servers by update attribute throws TypeError instead of CommandError

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Theodoros Tsioutsias <1733917@xxxxxxxxxxxxxxxxxx>
Date: Wed, 22 Nov 2017 17:16:57 -0000
Reply-to: Bug 1733917 <1733917@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

Description
===========

nova list command fails with TypeError instead of CommandError when an
existing but not valid attribute of the object is given as field.

$ /usr/bin/nova list --all --status ERROR --fields update
ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()

Steps to reproduce
==================

At least one server has to exist so that the list is not empty.
In this case update was given as a field. 
The Server object has an update method so the (not hasattr) check is False:

python-novaclient/novaclient/v2/shell.py:

1634 def _get_list_table_columns_and_formatters(fields, objs, exclude_fields=(),
1635                                            filters=None):
[...]
1680     for field in fields.split(','):
1681         if not hasattr(obj, field):
1682             non_existent_fields.append(field)
1683             continue
1684         if field in exclude_fields:
1685             continue
1686         field_title, formatter = utils.make_field_formatter(field,
1687                                                             filters)
1688         columns.append(field_title)
1689         formatters[field_title] = formatter
1690         exclude_fields.add(field)

As a result of this check, all of the attributes of the server object can be used as fields.
Most of them cause a TypeError to be raised.

e.g:
[devstack-006 ~]$ /usr/bin/nova list --fields __dict__
ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
[devstack-006 ~]$ /usr/bin/nova list --fields __getattr__
ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
[devstack-006 ~]$ /usr/bin/nova list --fields __getattribute__
ERROR (TypeError): object.__new__(method-wrapper) is not safe, use method-wrapper.__new__()
[devstack-006 ~]$ /usr/bin/nova list --fields __hash__
ERROR (TypeError): object.__new__(method-wrapper) is not safe, use method-wrapper.__new__()
[devstack-006 ~]$ /usr/bin/nova list --fields __init__
ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
[devstack-006 v2]$ /usr/bin/nova list --field unshelve
ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
[devstack-006 ~]$ /usr/bin/nova list --fields to_dict
ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
[devstack-006 ~]$ /usr/bin/nova list --fields revert_resize
ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
[devstack-006 ~]$ /usr/bin/nova list --fields _add_details
ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
[...]

Expected result
===============

A CommandError should be raised.

Actual result
=============

A TypeError is raised.

Environment
===========

$ rpm -qf /usr/bin/nova 
python2-novaclient-9.1.1-1.el7.noarch

Reproduced in devstack:

[devstack-006 ~]$ /usr/bin/nova list --field update
ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
[devstack-006 python-novaclient]$ git log -1
commit c9e7a64ca83302bdeab2044c09f9063646cc59a3
Merge: dd520c7 bef6765
Author: Zuul <zuul@xxxxxxxxxxxxxxxxxxxx>
Date:   Tue Nov 21 19:51:32 2017 +0000

    Merge "Microversion 2.54 - Enable reset keypair while rebuild"

Logs & Configs
==============

Attaching the debug output of the command.

Comments
========

Attributes like __module__ or __class__, are not raising a TypeError.
Should they be allowed?

[devstack-006 ~]$ /usr/bin/nova list --field __class__
+--------------------------------------+----------------------------------------+
| ID                                   |   Class                                |
+--------------------------------------+----------------------------------------+
| 353e6118-919e-4684-9e73-2441bbd8f0bd | <class 'novaclient.v2.servers.Server'> |
| b7f1f1f2-b195-4bd3-a5a1-5ad855526e4a | <class 'novaclient.v2.servers.Server'> |
| d7d99415-3bc5-4633-9c57-2b6f730a9bb1 | <class 'novaclient.v2.servers.Server'> |
+--------------------------------------+----------------------------------------+
[devstack-006 ~]$ /usr/bin/nova list --field __module__
+--------------------------------------+-----------------------+
| ID                                   |   Module              |
+--------------------------------------+-----------------------+
| 353e6118-919e-4684-9e73-2441bbd8f0bd | novaclient.v2.servers |
| b7f1f1f2-b195-4bd3-a5a1-5ad855526e4a | novaclient.v2.servers |
| d7d99415-3bc5-4633-9c57-2b6f730a9bb1 | novaclient.v2.servers |
+--------------------------------------+-----------------------+

** Affects: nova
     Importance: Undecided
     Assignee: Theodoros Tsioutsias (ttsiouts)
         Status: New

** Attachment added: "Debug output of the command"
   https://bugs.launchpad.net/bugs/1733917/+attachment/5012990/+files/debug_output.log

** Changed in: nova
     Assignee: (unassigned) => Theodoros Tsioutsias (ttsiouts)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1733917

Title:
  novaclient list servers by update attribute throws TypeError instead
  of CommandError

Status in OpenStack Compute (nova):
  New

Bug description:
  Description
  ===========

  nova list command fails with TypeError instead of CommandError when an
  existing but not valid attribute of the object is given as field.

  $ /usr/bin/nova list --all --status ERROR --fields update
  ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()

  Steps to reproduce
  ==================

  At least one server has to exist so that the list is not empty.
  In this case update was given as a field. 
  The Server object has an update method so the (not hasattr) check is False:

  python-novaclient/novaclient/v2/shell.py:

  1634 def _get_list_table_columns_and_formatters(fields, objs, exclude_fields=(),
  1635                                            filters=None):
  [...]
  1680     for field in fields.split(','):
  1681         if not hasattr(obj, field):
  1682             non_existent_fields.append(field)
  1683             continue
  1684         if field in exclude_fields:
  1685             continue
  1686         field_title, formatter = utils.make_field_formatter(field,
  1687                                                             filters)
  1688         columns.append(field_title)
  1689         formatters[field_title] = formatter
  1690         exclude_fields.add(field)

  As a result of this check, all of the attributes of the server object can be used as fields.
  Most of them cause a TypeError to be raised.

  e.g:
  [devstack-006 ~]$ /usr/bin/nova list --fields __dict__
  ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
  [devstack-006 ~]$ /usr/bin/nova list --fields __getattr__
  ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
  [devstack-006 ~]$ /usr/bin/nova list --fields __getattribute__
  ERROR (TypeError): object.__new__(method-wrapper) is not safe, use method-wrapper.__new__()
  [devstack-006 ~]$ /usr/bin/nova list --fields __hash__
  ERROR (TypeError): object.__new__(method-wrapper) is not safe, use method-wrapper.__new__()
  [devstack-006 ~]$ /usr/bin/nova list --fields __init__
  ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
  [devstack-006 v2]$ /usr/bin/nova list --field unshelve
  ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
  [devstack-006 ~]$ /usr/bin/nova list --fields to_dict
  ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
  [devstack-006 ~]$ /usr/bin/nova list --fields revert_resize
  ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
  [devstack-006 ~]$ /usr/bin/nova list --fields _add_details
  ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
  [...]

  Expected result
  ===============

  A CommandError should be raised.

  Actual result
  =============

  A TypeError is raised.

  Environment
  ===========

  $ rpm -qf /usr/bin/nova 
  python2-novaclient-9.1.1-1.el7.noarch

  Reproduced in devstack:

  [devstack-006 ~]$ /usr/bin/nova list --field update
  ERROR (TypeError): object.__new__(thread.lock) is not safe, use thread.lock.__new__()
  [devstack-006 python-novaclient]$ git log -1
  commit c9e7a64ca83302bdeab2044c09f9063646cc59a3
  Merge: dd520c7 bef6765
  Author: Zuul <zuul@xxxxxxxxxxxxxxxxxxxx>
  Date:   Tue Nov 21 19:51:32 2017 +0000

      Merge "Microversion 2.54 - Enable reset keypair while rebuild"

  Logs & Configs
  ==============

  Attaching the debug output of the command.

  Comments
  ========

  Attributes like __module__ or __class__, are not raising a TypeError.
  Should they be allowed?

  [devstack-006 ~]$ /usr/bin/nova list --field __class__
  +--------------------------------------+----------------------------------------+
  | ID                                   |   Class                                |
  +--------------------------------------+----------------------------------------+
  | 353e6118-919e-4684-9e73-2441bbd8f0bd | <class 'novaclient.v2.servers.Server'> |
  | b7f1f1f2-b195-4bd3-a5a1-5ad855526e4a | <class 'novaclient.v2.servers.Server'> |
  | d7d99415-3bc5-4633-9c57-2b6f730a9bb1 | <class 'novaclient.v2.servers.Server'> |
  +--------------------------------------+----------------------------------------+
  [devstack-006 ~]$ /usr/bin/nova list --field __module__
  +--------------------------------------+-----------------------+
  | ID                                   |   Module              |
  +--------------------------------------+-----------------------+
  | 353e6118-919e-4684-9e73-2441bbd8f0bd | novaclient.v2.servers |
  | b7f1f1f2-b195-4bd3-a5a1-5ad855526e4a | novaclient.v2.servers |
  | d7d99415-3bc5-4633-9c57-2b6f730a9bb1 | novaclient.v2.servers |
  +--------------------------------------+-----------------------+

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1733917/+subscriptions
Follow ups

[Bug 1733917] Re: novaclient list servers by update attribute throws TypeError instead of CommandError
From: Theodoros Tsioutsias, 2017-11-27