← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1618413] Re: VMs are going into error state when nova host aggregation is used to boot VMs

 

[Expired for OpenStack Compute (nova) because there has been no activity
for 60 days.]

** Changed in: nova
       Status: Incomplete => Expired

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1618413

Title:
  VMs are going into error state when nova host aggregation is used to
  boot VMs

Status in OpenStack Compute (nova):
  Expired

Bug description:
  VMs are going into error state when nova host aggregation is used to
  boot VMs.i have added host aggregation and added compute hosts to
  it.The meta data key-value is added in ext_specs of flavor.

  Setup used: Devstack-master branch

  In nova.conf, have the below config for filters

  scheduler_default_filters = RetryFilter,AvailabilityZoneFilter,RamFilter,DiskFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,SameHostFilter,DifferentHostFilter,AggregateInstanceExtraSpecsFilter
  scheduler_driver = filter_scheduler

  Steps:

  1. Create an aggregate

  stack@devstack:~/devstack$ nova aggregate-create test
  +----+------+-------------------+-------+----------+
  | Id | Name | Availability Zone | Hosts | Metadata |
  +----+------+-------------------+-------+----------+
  | 4  | test | -                 |       |          |
  +----+------+-------------------+-------+----------+

  2.Set a meta-data “family=intel” to the created aggregate

  stack@devstack:~/devstack$ nova aggregate-set-metadata 4 family=intel
  Metadata has been successfully updated for aggregate 4.
  +----+------+-------------------+-------+----------------+
  | Id | Name | Availability Zone | Hosts | Metadata       |
  +----+------+-------------------+-------+----------------+
  | 4  | test | -                 |       | 'family=intel' |
  +----+------+-------------------+-------+----------------+

  3. Add a host(devstack) to this aggregate

  stack@devstack:~/devstack$ nova aggregate-add-host 4 devstack
  Host devstack has been successfully added for aggregate 4
  +----+------+-------------------+------------+----------------+
  | Id | Name | Availability Zone | Hosts      | Metadata       |
  +----+------+-------------------+------------+----------------+
  | 4  | test | -                 | 'devstack' | 'family=intel' |
  +----+------+-------------------+------------+----------------+

  4. Create a new flavor

  stack@devstack:~/devstack$ nova flavor-create testflavor1 auto 4096 10 2 --is-public true
  +--------------------------------------+-------------+-----------+------+-----------+------+-------+-------------+-----------+
  | ID                                   | Name        | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor | Is_Public |
  +--------------------------------------+-------------+-----------+------+-----------+------+-------+-------------+-----------+
  | 75a142d8-3124-4fc9-97b3-9b1db1469d76 | testflavor1 | 4096      | 10   | 0         |      | 2     | 1.0         | True      |
  +--------------------------------------+-------------+-----------+------+-----------+------+-------+-------------+-----------+

  5.Create flavor key-value to match meta-data of the aggregate

  stack@devstack:~/devstack$ nova flavor-key testflavor1 set
  family=intel

  stack@devstack:~/devstack$  nova flavor-list
  +--------------------------------------+-------------+-----------+------+-----------+------+-------+-------------+-----------+
  | ID                                   | Name        | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor | Is_Public |
  +--------------------------------------+-------------+-----------+------+-----------+------+-------+-------------+-----------+
  | 1                                    | m1.tiny     | 512       | 1    | 0         |      | 1     | 1.0         | True      |
  | 2                                    | m1.small    | 2048      | 20   | 0         |      | 1     | 1.0         | True      |
  | 3                                    | m1.medium   | 4096      | 40   | 0         |      | 2     | 1.0         | True      |
  | 4                                    | m1.large    | 8192      | 80   | 0         |      | 4     | 1.0         | True      |
  | 42                                   | m1.nano     | 64        | 0    | 0         |      | 1     | 1.0         | True      |
  | 5                                    | m1.xlarge   | 16384     | 160  | 0         |      | 8     | 1.0         | True      |
  | 75a142d8-3124-4fc9-97b3-9b1db1469d76 | testflavor1 | 4096      | 10   | 0         |      | 2     | 1.0         | True      |
  | 84                                   | m1.micro    | 128       | 0    | 0         |      | 1     | 1.0         | True      |
  | c1                                   | cirros256   | 256       | 0    | 0         |      | 1     | 1.0         | True      |
  | d1                                   | ds512M      | 512       | 5    | 0         |      | 1     | 1.0         | True      |
  | d2                                   | ds1G        | 1024      | 10   | 0         |      | 1     | 1.0         | True      |
  | d3                                   | ds2G        | 2048      | 10   | 0         |      | 2     | 1.0         | True      |
  | d4                                   | ds4G        | 4096      | 20   | 0         |      | 4     | 1.0         | True      |
  +--------------------------------------+-------------+-----------+------+-----------+------+-------+-------------+-----------+

  6.From the nova flavor-show, we can see the key-value is in
  “extra_specs” already

  stack@devstack:~/devstack$ nova flavor-show testflavor1
  +----------------------------+--------------------------------------+
  | Property                   | Value                                |
  +----------------------------+--------------------------------------+
  | OS-FLV-DISABLED:disabled   | False                                |
  | OS-FLV-EXT-DATA:ephemeral  | 0                                    |
  | disk                       | 10                                   |
  | extra_specs                | {"family": "intel"}                  |
  | id                         | 75a142d8-3124-4fc9-97b3-9b1db1469d76 |
  | name                       | testflavor1                          |
  | os-flavor-access:is_public | True                                 |
  | ram                        | 4096                                 |
  | rxtx_factor                | 1.0                                  |
  | swap                       |                                      |
  | vcpus                      | 2                                    |
  +----------------------------+--------------------------------------+
  stack@devstack:~/devstack$

  7. Now boot a VM with the flavor

  stack@devstack:~/devstack$ nova boot --image cirros-0.3.4-x86_64-uec --flavor testflavor1 --nic net-name=private vm-test      +--------------------------------------+----------------------------------------------------------------+
  | Property                             | Value                                                          |
  +--------------------------------------+----------------------------------------------------------------+
  | OS-DCF:diskConfig                    | MANUAL                                                         |
  | OS-EXT-AZ:availability_zone          |                                                                |
  | OS-EXT-SRV-ATTR:host                 | -                                                              |
  | OS-EXT-SRV-ATTR:hostname             | vm-test                                                        |
  | OS-EXT-SRV-ATTR:hypervisor_hostname  | -                                                              |
  | OS-EXT-SRV-ATTR:instance_name        | instance-00000016                                              |
  | OS-EXT-SRV-ATTR:kernel_id            | 6c74a0f6-b54a-4604-9d6f-9e3886a811ee                           |
  | OS-EXT-SRV-ATTR:launch_index         | 0                                                              |
  | OS-EXT-SRV-ATTR:ramdisk_id           | 2728b402-2af0-4b01-91ae-0472ecb34a01                           |
  | OS-EXT-SRV-ATTR:reservation_id       | r-6n0igzwj                                                     |
  | OS-EXT-SRV-ATTR:root_device_name     | -                                                              |
  | OS-EXT-SRV-ATTR:user_data            | -                                                              |
  | OS-EXT-STS:power_state               | 0                                                              |
  | OS-EXT-STS:task_state                | scheduling                                                     |
  | OS-EXT-STS:vm_state                  | building                                                       |
  | OS-SRV-USG:launched_at               | -                                                              |
  | OS-SRV-USG:terminated_at             | -                                                              |
  | accessIPv4                           |                                                                |
  | accessIPv6                           |                                                                |
  | adminPass                            | T8PaX8G9F6rF                                                   |
  | config_drive                         |                                                                |
  | created                              | 2016-08-30T10:24:03Z                                           |
  | description                          | -                                                              |
  | flavor                               | testflavor1 (75a142d8-3124-4fc9-97b3-9b1db1469d76)             |
  | hostId                               |                                                                |
  | host_status                          |                                                                |
  | id                                   | 238b7482-8c94-427c-b073-dd1c922c572c                           |
  | image                                | cirros-0.3.4-x86_64-uec (bc8b111e-e312-4124-a3ba-64e7a2376612) |
  | key_name                             | -                                                              |
  | locked                               | False                                                          |
  | metadata                             | {}                                                             |
  | name                                 | vm-test                                                        |
  | os-extended-volumes:volumes_attached | []                                                             |
  | progress                             | 0                                                              |
  | security_groups                      | default                                                        |
  | status                               | BUILD                                                          |
  | tags                                 | []                                                             |
  | tenant_id                            | b26794c2d89748e9996d767a4ccfc5d0                               |
  | updated                              | 2016-08-30T10:24:04Z                                           |
  | user_id                              | 2398670e99844bd5ba30d6b7d3cdeaa5                               |
  +--------------------------------------+----------------------------------------------------------------+
  stack@devstack:~/devstack$ nova show vm-test
  +--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
  | Property                             | Value                                                                                                                                                                                     |
  +--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
  | OS-DCF:diskConfig                    | MANUAL                                                                                                                                                                                    |
  | OS-EXT-AZ:availability_zone          |                                                                                                                                                                                           |
  | OS-EXT-SRV-ATTR:host                 | -                                                                                                                                                                                         |
  | OS-EXT-SRV-ATTR:hostname             | vm-test                                                                                                                                                                                   |
  | OS-EXT-SRV-ATTR:hypervisor_hostname  | -                                                                                                                                                                                         |
  | OS-EXT-SRV-ATTR:instance_name        | instance-00000016                                                                                                                                                                         |
  | OS-EXT-SRV-ATTR:kernel_id            | 6c74a0f6-b54a-4604-9d6f-9e3886a811ee                                                                                                                                                      |
  | OS-EXT-SRV-ATTR:launch_index         | 0                                                                                                                                                                                         |
  | OS-EXT-SRV-ATTR:ramdisk_id           | 2728b402-2af0-4b01-91ae-0472ecb34a01                                                                                                                                                      |
  | OS-EXT-SRV-ATTR:reservation_id       | r-6n0igzwj                                                                                                                                                                                |
  | OS-EXT-SRV-ATTR:root_device_name     | -                                                                                                                                                                                         |
  | OS-EXT-SRV-ATTR:user_data            | -                                                                                                                                                                                         |
  | OS-EXT-STS:power_state               | 0                                                                                                                                                                                         |
  | OS-EXT-STS:task_state                | -                                                                                                                                                                                         |
  | OS-EXT-STS:vm_state                  | error                                                                                                                                                                                     |
  | OS-SRV-USG:launched_at               | -                                                                                                                                                                                         |
  | OS-SRV-USG:terminated_at             | -                                                                                                                                                                                         |
  | accessIPv4                           |                                                                                                                                                                                           |
  | accessIPv6                           |                                                                                                                                                                                           |
  | config_drive                         |                                                                                                                                                                                           |
  | created                              | 2016-08-30T10:24:03Z                                                                                                                                                                      |
  | description                          | -                                                                                                                                                                                         |
  | fault                                | {"message": "No valid host was found. There are not enough hosts available.", "code": 500, "details": "  File \"/opt/stack/nova/nova/conductor/manager.py\", line 484, in build_instances |
  |                                      |     context, request_spec, filter_properties)                                                                                                                                             |
  |                                      |   File \"/opt/stack/nova/nova/conductor/manager.py\", line 541, in _schedule_instances                                                                                                    |
  |                                      |     hosts = self.scheduler_client.select_destinations(context, spec_obj)                                                                                                                  |
  |                                      |   File \"/opt/stack/nova/nova/scheduler/utils.py\", line 370, in wrapped                                                                                                                  |
  |                                      |     return func(*args, **kwargs)                                                                                                                                                          |
  |                                      |   File \"/opt/stack/nova/nova/scheduler/client/__init__.py\", line 51, in select_destinations                                                                                             |
  |                                      |     return self.queryclient.select_destinations(context, spec_obj)                                                                                                                        |
  |                                      |   File \"/opt/stack/nova/nova/scheduler/client/__init__.py\", line 37, in __run_method                                                                                                    |
  |                                      |     return getattr(self.instance, __name)(*args, **kwargs)                                                                                                                                |
  |                                      |   File \"/opt/stack/nova/nova/scheduler/client/query.py\", line 32, in select_destinations                                                                                                |
  |                                      |     return self.scheduler_rpcapi.select_destinations(context, spec_obj)                                                                                                                   |
  |                                      |   File \"/opt/stack/nova/nova/scheduler/rpcapi.py\", line 126, in select_destinations                                                                                                     |
  |                                      |     return cctxt.call(ctxt, 'select_destinations', **msg_args)                                                                                                                            |
  |                                      |   File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py\", line 169, in call                                                                                         |
  |                                      |     retry=self.retry)                                                                                                                                                                     |
  |                                      |   File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/transport.py\", line 97, in _send                                                                                          |
  |                                      |     timeout=timeout, retry=retry)                                                                                                                                                         |
  |                                      |   File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py\", line 464, in send                                                                                |
  |                                      |     retry=retry)                                                                                                                                                                          |
  |                                      |   File \"/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py\", line 455, in _send                                                                               |
  |                                      |     raise result                                                                                                                                                                          |
  |                                      | ", "created": "2016-08-30T10:24:04Z"}                                                                                                                                                     |
  | flavor                               | testflavor1 (75a142d8-3124-4fc9-97b3-9b1db1469d76)                                                                                                                                        |
  | hostId                               |                                                                                                                                                                                           |
  | host_status                          |                                                                                                                                                                                           |
  | id                                   | 238b7482-8c94-427c-b073-dd1c922c572c                                                                                                                                                      |
  | image                                | cirros-0.3.4-x86_64-uec (bc8b111e-e312-4124-a3ba-64e7a2376612)                                                                                                                            |
  | key_name                             | -                                                                                                                                                                                         |
  | locked                               | False                                                                                                                                                                                     |
  | metadata                             | {}                                                                                                                                                                                        |
  | name                                 | vm-test                                                                                                                                                                                   |
  | os-extended-volumes:volumes_attached | []                                                                                                                                                                                        |
  | status                               | ERROR                                                                                                                                                                                     |
  | tags                                 | []                                                                                                                                                                                        |
  | tenant_id                            | b26794c2d89748e9996d767a4ccfc5d0                                                                                                                                                          |
  | updated                              | 2016-08-30T10:24:04Z                                                                                                                                                                      |
  | user_id                              | 2398670e99844bd5ba30d6b7d3cdeaa5                                                                                                                                                          |
  +--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
  stack@devstack:~/devstack$


  Attached devstack logs

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1618413/+subscriptions


References