← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1849657] [NEW] allocation key is missing from the binding:profile of the neutron qos port when the server is created by a non-admin user

 

Public bug reported:

Description
===========
When a server is create by a non-admin tenant with a qos neutron port Nova does not add the allocation key to the binding:profile of the port.

Steps to reproduce
==================
1) Set up a devstack with bandwidth inventory
   * sudo ovs-vsctl add-br br-test
   * devstack local conf:
[[post-config|/etc/neutron/neutron.conf]]
[DEFAULT]
service_plugins = router, placement, qos

[[post-config|/etc/neutron/plugins/ml2/ml2_conf.ini]]
[ml2]
extension_drivers = port_security,qos
mechanism_drivers = openvswitch
tenant_network_types = vxlan

[ml2_type_vlan]
network_vlan_ranges = physnet0:1000:2000
[ovs]
bridge_mappings = public:br-ex,physnet0:br-test
resource_provider_bandwidths = br-test:5000:5000
[ovs_driver]
vnic_type_blacklist = direct

   * stack.sh

2) As admin user set up a network and a qos policy:

   * openstack network create net-demo --provider-network-type vlan  --provider-physical-network physnet0 --provider-segment 101 --share
   * openstack subnet create subnet-demo --network net-demo --subnet-range 10.0.4.0/24
   * openstack network qos policy create qp-demo --share
   * openstack network qos rule create qp-demo --type minimum-bandwidth --min-kbps 1000 --egress
   * openstack network qos rule create qp-demo --type minimum-bandwidth --min-kbps 1000 --ingress

3) As a normal user (demo in devstack) create a port with the qos policy
and create a server with the port

   * openstack port create port-normal-qos-demo --network net-demo --vnic-type normal --qos-policy qp-demo
   * openstack --os-compute-api-version 2.72 server create --image cirros-0.4.0-x86_64-disk --flavor c1 --nic port-id=port-normal-qos-demo vm-demo --wait


Expected result
===============

1) Server is reaching ACTIVE state
2) Bandwidth allocation is created in placement according to the qp-demo policy
3) The allocation key of the binding:profile of the port-normal-qos-demo port contains the UUID of the placement resource provider from where the bandwidth resource is allocated from.


Actual result
=============
1) and 2) are as expected but the binding:porfile of the neutron port does not have an allocation key.

Note that if the server is booted as admin user then both 1) 2) 3) are
as expected.

Environment
===========
Devstack from master:

stack@aio:/opt/stack/nova$ git log --oneline | head -1
d3403e5294 Merge "Fix unit of hw_rng:rate_period"

stack@aio:/opt/stack/neutron$ git log --oneline | head -1
2ffaa40b43 Merge "ovsdb monitor: handle modified ports"

Triage
======
Looking at the port-normal-qos-demo port from the demo user. The resource_request filed of the port is None. While looking at the port from the admin user the resource_request field is properly filled according to the qos policy of the port.

As demo:

stack@aio:~$ openstack port show port-normal-qos-demo
+-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field                   | Value                                                                                                                                                           |
+-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up          | UP                                                                                                                                                              |
| allowed_address_pairs   |                                                                                                                                                                 |
| binding_host_id         | None                                                                                                                                                            |
| binding_profile         | None                                                                                                                                                            |
| binding_vif_details     | None                                                                                                                                                            |
| binding_vif_type        | None                                                                                                                                                            |
| binding_vnic_type       | normal                                                                                                                                                          |
| created_at              | 2019-10-24T11:05:27Z                                                                                                                                            |
| data_plane_status       | None                                                                                                                                                            |
| description             |                                                                                                                                                                 |
| device_id               |                                                                                                                                                                 |
| device_owner            |                                                                                                                                                                 |
| dns_assignment          | None                                                                                                                                                            |
| dns_domain              | None                                                                                                                                                            |
| dns_name                | None                                                                                                                                                            |
| extra_dhcp_opts         |                                                                                                                                                                 |
| fixed_ips               | ip_address='10.0.4.213', subnet_id='736636a1-114f-4d9e-9e8f-82568593061d'                                                                                       |
| id                      | b1593c18-b088-4d5c-b3c6-bdd5348f3b52                                                                                                                            |
| location                | cloud='', project.domain_id='default', project.domain_name=, project.id='05c189206e0d4e3d8ea95f1a4067b420', project.name='demo', region_name='RegionOne', zone= |
| mac_address             | fa:16:3e:8a:56:8a                                                                                                                                               |
| name                    | port-normal-qos-demo                                                                                                                                            |
| network_id              | f32506ae-4cf4-414a-8349-09fc744f024c                                                                                                                            |
| port_security_enabled   | True                                                                                                                                                            |
| project_id              | 05c189206e0d4e3d8ea95f1a4067b420                                                                                                                                |
| propagate_uplink_status | None                                                                                                                                                            |
| qos_policy_id           | f774c8b2-d302-427b-bdef-bd0614a0fbaa                                                                                                                            |
| resource_request        | None                                                                                                                                                            |
| revision_number         | 1                                                                                                                                                               |
| security_group_ids      | fb5f417f-727b-48c9-881a-21571450ae06                                                                                                                            |
| status                  | DOWN                                                                                                                                                            |
| tags                    |                                                                                                                                                                 |
| trunk_details           | None                                                                                                                                                            |
| updated_at              | 2019-10-24T11:05:27Z                                                                                                                                            |
+-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+

As admin:

stack@aio:~$ openstack port show port-normal-qos-demo
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field                   | Value                                                                                                                                                             |
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up          | UP                                                                                                                                                                |
| allowed_address_pairs   |                                                                                                                                                                   |
| binding_host_id         | aio                                                                                                                                                               |
| binding_profile         |                                                                                                                                                                   |
| binding_vif_details     | bridge_name='br-int', connectivity='l2', datapath_type='system', ovs_hybrid_plug='False', port_filter='True'                                                      |
| binding_vif_type        | ovs                                                                                                                                                               |
| binding_vnic_type       | normal                                                                                                                                                            |
| created_at              | 2019-10-24T11:05:27Z                                                                                                                                              |
| data_plane_status       | None                                                                                                                                                              |
| description             |                                                                                                                                                                   |
| device_id               | f60856cc-9817-4fd9-a89f-c87c93fc729b                                                                                                                              |
| device_owner            | compute:nova                                                                                                                                                      |
| dns_assignment          | None                                                                                                                                                              |
| dns_domain              | None                                                                                                                                                              |
| dns_name                | None                                                                                                                                                              |
| extra_dhcp_opts         |                                                                                                                                                                   |
| fixed_ips               | ip_address='10.0.4.213', subnet_id='736636a1-114f-4d9e-9e8f-82568593061d'                                                                                         |
| id                      | b1593c18-b088-4d5c-b3c6-bdd5348f3b52                                                                                                                              |
| location                | cloud='', project.domain_id=, project.domain_name=, project.id='05c189206e0d4e3d8ea95f1a4067b420', project.name=, region_name='RegionOne', zone=                  |
| mac_address             | fa:16:3e:8a:56:8a                                                                                                                                                 |
| name                    | port-normal-qos-demo                                                                                                                                              |
| network_id              | f32506ae-4cf4-414a-8349-09fc744f024c                                                                                                                              |
| port_security_enabled   | True                                                                                                                                                              |
| project_id              | 05c189206e0d4e3d8ea95f1a4067b420                                                                                                                                  |
| propagate_uplink_status | None                                                                                                                                                              |
| qos_policy_id           | f774c8b2-d302-427b-bdef-bd0614a0fbaa                                                                                                                              |
| resource_request        | {u'required': [u'CUSTOM_PHYSNET_PHYSNET0', u'CUSTOM_VNIC_TYPE_NORMAL'], u'resources': {u'NET_BW_EGR_KILOBIT_PER_SEC': 1000, u'NET_BW_IGR_KILOBIT_PER_SEC': 1000}} |
| revision_number         | 4                                                                                                                                                                 |
| security_group_ids      | fb5f417f-727b-48c9-881a-21571450ae06                                                                                                                              |
| status                  | ACTIVE                                                                                                                                                            |
| tags                    |                                                                                                                                                                   |
| trunk_details           | None                                                                                                                                                              |
| updated_at              | 2019-10-24T11:14:46Z                                                                                                                                              |
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+

The placement allocation is correct:

stack@aio:~$ openstack --os-placement-api-version 1.22 resource provider allocation show f60856cc-9817-4fd9-a89f-c87c93fc729b
+--------------------------------------+------------+----------------------------------------------------------------------------+----------------------------------+----------------------------------+
| resource_provider                    | generation | resources                                                                  | project_id                       | user_id                          |
+--------------------------------------+------------+----------------------------------------------------------------------------+----------------------------------+----------------------------------+
| 1110cf59-cabf-526c-bacc-08baabbac692 |          9 | {u'NET_BW_EGR_KILOBIT_PER_SEC': 1000, u'NET_BW_IGR_KILOBIT_PER_SEC': 1000} | 05c189206e0d4e3d8ea95f1a4067b420 | 2d0cf2d4e46348fda2c6d47d0e619544 |
| 40f2860b-f0b4-4325-af91-011d374c8aba |         15 | {u'VCPU': 1, u'MEMORY_MB': 256, u'DISK_GB': 1}                             | 05c189206e0d4e3d8ea95f1a4067b420 | 2d0cf2d4e46348fda2c6d47d0e619544 |
+--------------------------------------+------------+----------------------------------------------------------------------------+----------------------------------+----------------------------------+

So Nova was able to gather the resource_request field from the Neutron
port with admin (service) credentials before the scheduling. But Nova
failed to include the allocation key to the same port. I assume that
this happens because Nova only adds allocation key to the port if the
port has resource_request [1] but Nova checks the port at [1] with the
user credentials not with the service credentials.

[1]
https://github.com/openstack/nova/blob/1bfa4626d13d0a73e63745cc4a864ae86d490daf/nova/network/neutronv2/api.py#L998

** Affects: nova
     Importance: Medium
     Assignee: Balazs Gibizer (balazs-gibizer)
         Status: New


** Tags: neutron

** Changed in: nova
     Assignee: (unassigned) => Balazs Gibizer (balazs-gibizer)

** Changed in: nova
   Importance: Undecided => Medium

** Tags added: neutron

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1849657

Title:
   allocation key is missing from the binding:profile of the neutron qos
  port when the server is created by a non-admin user

Status in OpenStack Compute (nova):
  New

Bug description:
  Description
  ===========
  When a server is create by a non-admin tenant with a qos neutron port Nova does not add the allocation key to the binding:profile of the port.

  Steps to reproduce
  ==================
  1) Set up a devstack with bandwidth inventory
     * sudo ovs-vsctl add-br br-test
     * devstack local conf:
  [[post-config|/etc/neutron/neutron.conf]]
  [DEFAULT]
  service_plugins = router, placement, qos

  [[post-config|/etc/neutron/plugins/ml2/ml2_conf.ini]]
  [ml2]
  extension_drivers = port_security,qos
  mechanism_drivers = openvswitch
  tenant_network_types = vxlan

  [ml2_type_vlan]
  network_vlan_ranges = physnet0:1000:2000
  [ovs]
  bridge_mappings = public:br-ex,physnet0:br-test
  resource_provider_bandwidths = br-test:5000:5000
  [ovs_driver]
  vnic_type_blacklist = direct

     * stack.sh

  2) As admin user set up a network and a qos policy:

     * openstack network create net-demo --provider-network-type vlan  --provider-physical-network physnet0 --provider-segment 101 --share
     * openstack subnet create subnet-demo --network net-demo --subnet-range 10.0.4.0/24
     * openstack network qos policy create qp-demo --share
     * openstack network qos rule create qp-demo --type minimum-bandwidth --min-kbps 1000 --egress
     * openstack network qos rule create qp-demo --type minimum-bandwidth --min-kbps 1000 --ingress

  3) As a normal user (demo in devstack) create a port with the qos
  policy and create a server with the port

     * openstack port create port-normal-qos-demo --network net-demo --vnic-type normal --qos-policy qp-demo
     * openstack --os-compute-api-version 2.72 server create --image cirros-0.4.0-x86_64-disk --flavor c1 --nic port-id=port-normal-qos-demo vm-demo --wait

  
  Expected result
  ===============

  1) Server is reaching ACTIVE state
  2) Bandwidth allocation is created in placement according to the qp-demo policy
  3) The allocation key of the binding:profile of the port-normal-qos-demo port contains the UUID of the placement resource provider from where the bandwidth resource is allocated from.

  
  Actual result
  =============
  1) and 2) are as expected but the binding:porfile of the neutron port does not have an allocation key.

  Note that if the server is booted as admin user then both 1) 2) 3) are
  as expected.

  Environment
  ===========
  Devstack from master:

  stack@aio:/opt/stack/nova$ git log --oneline | head -1
  d3403e5294 Merge "Fix unit of hw_rng:rate_period"

  stack@aio:/opt/stack/neutron$ git log --oneline | head -1
  2ffaa40b43 Merge "ovsdb monitor: handle modified ports"

  Triage
  ======
  Looking at the port-normal-qos-demo port from the demo user. The resource_request filed of the port is None. While looking at the port from the admin user the resource_request field is properly filled according to the qos policy of the port.

  As demo:

  stack@aio:~$ openstack port show port-normal-qos-demo
  +-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+
  | Field                   | Value                                                                                                                                                           |
  +-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+
  | admin_state_up          | UP                                                                                                                                                              |
  | allowed_address_pairs   |                                                                                                                                                                 |
  | binding_host_id         | None                                                                                                                                                            |
  | binding_profile         | None                                                                                                                                                            |
  | binding_vif_details     | None                                                                                                                                                            |
  | binding_vif_type        | None                                                                                                                                                            |
  | binding_vnic_type       | normal                                                                                                                                                          |
  | created_at              | 2019-10-24T11:05:27Z                                                                                                                                            |
  | data_plane_status       | None                                                                                                                                                            |
  | description             |                                                                                                                                                                 |
  | device_id               |                                                                                                                                                                 |
  | device_owner            |                                                                                                                                                                 |
  | dns_assignment          | None                                                                                                                                                            |
  | dns_domain              | None                                                                                                                                                            |
  | dns_name                | None                                                                                                                                                            |
  | extra_dhcp_opts         |                                                                                                                                                                 |
  | fixed_ips               | ip_address='10.0.4.213', subnet_id='736636a1-114f-4d9e-9e8f-82568593061d'                                                                                       |
  | id                      | b1593c18-b088-4d5c-b3c6-bdd5348f3b52                                                                                                                            |
  | location                | cloud='', project.domain_id='default', project.domain_name=, project.id='05c189206e0d4e3d8ea95f1a4067b420', project.name='demo', region_name='RegionOne', zone= |
  | mac_address             | fa:16:3e:8a:56:8a                                                                                                                                               |
  | name                    | port-normal-qos-demo                                                                                                                                            |
  | network_id              | f32506ae-4cf4-414a-8349-09fc744f024c                                                                                                                            |
  | port_security_enabled   | True                                                                                                                                                            |
  | project_id              | 05c189206e0d4e3d8ea95f1a4067b420                                                                                                                                |
  | propagate_uplink_status | None                                                                                                                                                            |
  | qos_policy_id           | f774c8b2-d302-427b-bdef-bd0614a0fbaa                                                                                                                            |
  | resource_request        | None                                                                                                                                                            |
  | revision_number         | 1                                                                                                                                                               |
  | security_group_ids      | fb5f417f-727b-48c9-881a-21571450ae06                                                                                                                            |
  | status                  | DOWN                                                                                                                                                            |
  | tags                    |                                                                                                                                                                 |
  | trunk_details           | None                                                                                                                                                            |
  | updated_at              | 2019-10-24T11:05:27Z                                                                                                                                            |
  +-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+

  As admin:

  stack@aio:~$ openstack port show port-normal-qos-demo
  +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
  | Field                   | Value                                                                                                                                                             |
  +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
  | admin_state_up          | UP                                                                                                                                                                |
  | allowed_address_pairs   |                                                                                                                                                                   |
  | binding_host_id         | aio                                                                                                                                                               |
  | binding_profile         |                                                                                                                                                                   |
  | binding_vif_details     | bridge_name='br-int', connectivity='l2', datapath_type='system', ovs_hybrid_plug='False', port_filter='True'                                                      |
  | binding_vif_type        | ovs                                                                                                                                                               |
  | binding_vnic_type       | normal                                                                                                                                                            |
  | created_at              | 2019-10-24T11:05:27Z                                                                                                                                              |
  | data_plane_status       | None                                                                                                                                                              |
  | description             |                                                                                                                                                                   |
  | device_id               | f60856cc-9817-4fd9-a89f-c87c93fc729b                                                                                                                              |
  | device_owner            | compute:nova                                                                                                                                                      |
  | dns_assignment          | None                                                                                                                                                              |
  | dns_domain              | None                                                                                                                                                              |
  | dns_name                | None                                                                                                                                                              |
  | extra_dhcp_opts         |                                                                                                                                                                   |
  | fixed_ips               | ip_address='10.0.4.213', subnet_id='736636a1-114f-4d9e-9e8f-82568593061d'                                                                                         |
  | id                      | b1593c18-b088-4d5c-b3c6-bdd5348f3b52                                                                                                                              |
  | location                | cloud='', project.domain_id=, project.domain_name=, project.id='05c189206e0d4e3d8ea95f1a4067b420', project.name=, region_name='RegionOne', zone=                  |
  | mac_address             | fa:16:3e:8a:56:8a                                                                                                                                                 |
  | name                    | port-normal-qos-demo                                                                                                                                              |
  | network_id              | f32506ae-4cf4-414a-8349-09fc744f024c                                                                                                                              |
  | port_security_enabled   | True                                                                                                                                                              |
  | project_id              | 05c189206e0d4e3d8ea95f1a4067b420                                                                                                                                  |
  | propagate_uplink_status | None                                                                                                                                                              |
  | qos_policy_id           | f774c8b2-d302-427b-bdef-bd0614a0fbaa                                                                                                                              |
  | resource_request        | {u'required': [u'CUSTOM_PHYSNET_PHYSNET0', u'CUSTOM_VNIC_TYPE_NORMAL'], u'resources': {u'NET_BW_EGR_KILOBIT_PER_SEC': 1000, u'NET_BW_IGR_KILOBIT_PER_SEC': 1000}} |
  | revision_number         | 4                                                                                                                                                                 |
  | security_group_ids      | fb5f417f-727b-48c9-881a-21571450ae06                                                                                                                              |
  | status                  | ACTIVE                                                                                                                                                            |
  | tags                    |                                                                                                                                                                   |
  | trunk_details           | None                                                                                                                                                              |
  | updated_at              | 2019-10-24T11:14:46Z                                                                                                                                              |
  +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+

  The placement allocation is correct:

  stack@aio:~$ openstack --os-placement-api-version 1.22 resource provider allocation show f60856cc-9817-4fd9-a89f-c87c93fc729b
  +--------------------------------------+------------+----------------------------------------------------------------------------+----------------------------------+----------------------------------+
  | resource_provider                    | generation | resources                                                                  | project_id                       | user_id                          |
  +--------------------------------------+------------+----------------------------------------------------------------------------+----------------------------------+----------------------------------+
  | 1110cf59-cabf-526c-bacc-08baabbac692 |          9 | {u'NET_BW_EGR_KILOBIT_PER_SEC': 1000, u'NET_BW_IGR_KILOBIT_PER_SEC': 1000} | 05c189206e0d4e3d8ea95f1a4067b420 | 2d0cf2d4e46348fda2c6d47d0e619544 |
  | 40f2860b-f0b4-4325-af91-011d374c8aba |         15 | {u'VCPU': 1, u'MEMORY_MB': 256, u'DISK_GB': 1}                             | 05c189206e0d4e3d8ea95f1a4067b420 | 2d0cf2d4e46348fda2c6d47d0e619544 |
  +--------------------------------------+------------+----------------------------------------------------------------------------+----------------------------------+----------------------------------+

  So Nova was able to gather the resource_request field from the Neutron
  port with admin (service) credentials before the scheduling. But Nova
  failed to include the allocation key to the same port. I assume that
  this happens because Nova only adds allocation key to the port if the
  port has resource_request [1] but Nova checks the port at [1] with the
  user credentials not with the service credentials.

  [1]
  https://github.com/openstack/nova/blob/1bfa4626d13d0a73e63745cc4a864ae86d490daf/nova/network/neutronv2/api.py#L998

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1849657/+subscriptions


Follow ups