yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #55617
[Bug 1617429] [NEW] pci: Can't get DHCP address after resize with port type 'direct-physical'
Public bug reported:
Description:
============
Booting a guest with a neutron port of type 'direct-physical' will cause
nova to allocate a PCI passthrough device for the port. The MAC address
of the PCI passthrough device in the guest is not a virtual MAC address
(fa:16:...) but the MAC address of the physical device since the full
device is allocated to the guest (compared to SR-IOV where a virtual MAC
address is arbitrarily chosen for the port).
When resizing the guest (to another flavor), nova will allocate a new
PCI device for the guest. After the resize, the guest will be bound to
another PCI device which has a different MAC address. However the MAC
address on the neutron port is not updated, causing DHCP to not work
because the MAC address is unknown.
The same issue can be observed when migrating a guest to another host.
Steps to reproduce:
===================
1- Configure a compute with two NICs PCI passthrough devices
2- Create a neutron port of type 'direct-physical':
PORTID=`neutron port-create $NETID --binding:vnic_type direct-physical | grep "\ id\ " | awk '{ print $4 }'`
3- Boot a guest with the port-id:
nova boot guest --image=ubuntu --nic port-id=$PORTID --flavor=m1.medium
4- Note the MAC address of the neutron port:
[centos@IronPass-2 devstack]$ neutron port-show 657990ce-3446-40a2-bc33-8040a32cb72b
+-----------------------+-----------------------------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+-----------------------------------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:vnic_type | direct-physical |
| created_at | 2016-08-26T14:57:07 |
| description | |
| device_id | f1005ec2-1875-4345-af01-58afaee7e68d |
| device_owner | compute:None |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "8e6aaf8e-78e8-42d7-9b59-f96789728abc", "ip_address": "10.0.0.12"} |
| | {"subnet_id": "4210fe1c-b156-4ecb-a896-922c87f85c3f", "ip_address": "fdab:2588:f4:0:f816:3eff:fe37:acba"} |
| id | 657990ce-3446-40a2-bc33-8040a32cb72b |
| mac_address | 90:e2:ba:48:27:ed |
| name | sriov_port |
| network_id | dfdff739-a27b-4160-9e40-c4824e8a351d |
| port_security_enabled | True |
| revision | 57 |
| security_groups | 8c7b8173-f050-480d-968a-33e0b75332fc |
| status | ACTIVE |
| tenant_id | a11cb2aece1943c2a86ee0a55e1bd8f7 |
| updated_at | 2016-08-26T17:33:12 |
+-----------------------+-----------------------------------------------------------------------------------------------------------+
5- Log in the guest and get a DHCP address for the interface.
ip link add link eth1 name eth1.451 type vlan id 451
ip link set eth1.451 up
dhclient eth1.451
Note the MAC address of the interface:
4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 90:e2:ba:48:27:ed brd ff:ff:ff:ff:ff:ff
inet6 fe80::92e2:baff:fe48:27ed/64 scope link
valid_lft forever preferred_lft forever
5: eth1.451@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 90:e2:ba:48:27:ed brd ff:ff:ff:ff:ff:ff
inet 10.0.0.12/24 brd 10.0.0.255 scope global eth1.451
valid_lft forever preferred_lft forever
inet6 fdab:2588:f4:0:92e2:baff:fe48:27ed/64 scope global dynamic
valid_lft 86396sec preferred_lft 14396sec
inet6 fe80::92e2:baff:fe48:27ed/64 scope link
valid_lft forever preferred_lft forever
6- Resize the guest
7- Note the MAC address of the neutron port (after resize):
[centos@IronPass-2 devstack]$ neutron port-show 657990ce-3446-40a2-bc33-8040a32cb72b
+-----------------------+-----------------------------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+-----------------------------------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:vnic_type | direct-physical |
| created_at | 2016-08-26T14:57:07 |
| description | |
| device_id | f1005ec2-1875-4345-af01-58afaee7e68d |
| device_owner | compute:None |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "8e6aaf8e-78e8-42d7-9b59-f96789728abc", "ip_address": "10.0.0.12"} |
| | {"subnet_id": "4210fe1c-b156-4ecb-a896-922c87f85c3f", "ip_address": "fdab:2588:f4:0:f816:3eff:fe37:acba"} |
| id | 657990ce-3446-40a2-bc33-8040a32cb72b |
| mac_address | 90:e2:ba:48:27:ed |
| name | sriov_port |
| network_id | dfdff739-a27b-4160-9e40-c4824e8a351d |
| port_security_enabled | True |
| revision | 59 |
| security_groups | 8c7b8173-f050-480d-968a-33e0b75332fc |
| status | ACTIVE |
| tenant_id | a11cb2aece1943c2a86ee0a55e1bd8f7 |
| updated_at | 2016-08-26T17:38:30 |
+-----------------------+-----------------------------------------------------------------------------------------------------------+
Notice that it's still hasn't changed: 90:e2:ba:48:27:ed
8- Log in the guest and get a DHCP address for the interface.
ip link add link eth1 name eth1.451 type vlan id 451
ip link set eth1.451 up
dhclient eth1.451
Note the MAC address of the interface has changed:
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 00:1e:67:51:36:71 brd ff:ff:ff:ff:ff:ff
inet6 fe80::21e:67ff:fe51:3671/64 scope link
valid_lft forever preferred_lft forever
5: eth2.451@eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 00:1e:67:51:36:71 brd ff:ff:ff:ff:ff:ff
inet6 fe80::21e:67ff:fe51:3671/64 scope link
valid_lft forever preferred_lft forever
Also note that the DHCP client wasn't successful in getting a DHCP
address.
9- From the dnsmasq host file, we can also see that it hasn't been
updated:
cat /opt/stack/data/neutron/dhcp/dfdff739-a27b-4160-9e40-c4824e8a351d/host
fa:16:3e:e5:e0:c9,host-10-0-0-2.openstacklocal,10.0.0.2
90:e2:ba:48:27:ed,host-10-0-0-12.openstacklocal,10.0.0.12
fa:16:3e:99:c8:b9,host-10-0-0-5.openstacklocal,10.0.0.5
Environment:
============
Latest master
commit 7500bef94f526a82392400415f07f744700324a9
Merge: 8d5aff7 39fb302
Author: Jenkins <jenkins@xxxxxxxxxxxxxxxxxxxx>
Date: Fri Aug 26 05:03:48 2016 +0000
Merge "Revert "Optional separate database for placement API""
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1617429
Title:
pci: Can't get DHCP address after resize with port type 'direct-
physical'
Status in OpenStack Compute (nova):
New
Bug description:
Description:
============
Booting a guest with a neutron port of type 'direct-physical' will
cause nova to allocate a PCI passthrough device for the port. The MAC
address of the PCI passthrough device in the guest is not a virtual
MAC address (fa:16:...) but the MAC address of the physical device
since the full device is allocated to the guest (compared to SR-IOV
where a virtual MAC address is arbitrarily chosen for the port).
When resizing the guest (to another flavor), nova will allocate a new
PCI device for the guest. After the resize, the guest will be bound
to another PCI device which has a different MAC address. However the
MAC address on the neutron port is not updated, causing DHCP to not
work because the MAC address is unknown.
The same issue can be observed when migrating a guest to another host.
Steps to reproduce:
===================
1- Configure a compute with two NICs PCI passthrough devices
2- Create a neutron port of type 'direct-physical':
PORTID=`neutron port-create $NETID --binding:vnic_type direct-physical | grep "\ id\ " | awk '{ print $4 }'`
3- Boot a guest with the port-id:
nova boot guest --image=ubuntu --nic port-id=$PORTID --flavor=m1.medium
4- Note the MAC address of the neutron port:
[centos@IronPass-2 devstack]$ neutron port-show 657990ce-3446-40a2-bc33-8040a32cb72b
+-----------------------+-----------------------------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+-----------------------------------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:vnic_type | direct-physical |
| created_at | 2016-08-26T14:57:07 |
| description | |
| device_id | f1005ec2-1875-4345-af01-58afaee7e68d |
| device_owner | compute:None |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "8e6aaf8e-78e8-42d7-9b59-f96789728abc", "ip_address": "10.0.0.12"} |
| | {"subnet_id": "4210fe1c-b156-4ecb-a896-922c87f85c3f", "ip_address": "fdab:2588:f4:0:f816:3eff:fe37:acba"} |
| id | 657990ce-3446-40a2-bc33-8040a32cb72b |
| mac_address | 90:e2:ba:48:27:ed |
| name | sriov_port |
| network_id | dfdff739-a27b-4160-9e40-c4824e8a351d |
| port_security_enabled | True |
| revision | 57 |
| security_groups | 8c7b8173-f050-480d-968a-33e0b75332fc |
| status | ACTIVE |
| tenant_id | a11cb2aece1943c2a86ee0a55e1bd8f7 |
| updated_at | 2016-08-26T17:33:12 |
+-----------------------+-----------------------------------------------------------------------------------------------------------+
5- Log in the guest and get a DHCP address for the interface.
ip link add link eth1 name eth1.451 type vlan id 451
ip link set eth1.451 up
dhclient eth1.451
Note the MAC address of the interface:
4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 90:e2:ba:48:27:ed brd ff:ff:ff:ff:ff:ff
inet6 fe80::92e2:baff:fe48:27ed/64 scope link
valid_lft forever preferred_lft forever
5: eth1.451@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 90:e2:ba:48:27:ed brd ff:ff:ff:ff:ff:ff
inet 10.0.0.12/24 brd 10.0.0.255 scope global eth1.451
valid_lft forever preferred_lft forever
inet6 fdab:2588:f4:0:92e2:baff:fe48:27ed/64 scope global dynamic
valid_lft 86396sec preferred_lft 14396sec
inet6 fe80::92e2:baff:fe48:27ed/64 scope link
valid_lft forever preferred_lft forever
6- Resize the guest
7- Note the MAC address of the neutron port (after resize):
[centos@IronPass-2 devstack]$ neutron port-show 657990ce-3446-40a2-bc33-8040a32cb72b
+-----------------------+-----------------------------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+-----------------------------------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:vnic_type | direct-physical |
| created_at | 2016-08-26T14:57:07 |
| description | |
| device_id | f1005ec2-1875-4345-af01-58afaee7e68d |
| device_owner | compute:None |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "8e6aaf8e-78e8-42d7-9b59-f96789728abc", "ip_address": "10.0.0.12"} |
| | {"subnet_id": "4210fe1c-b156-4ecb-a896-922c87f85c3f", "ip_address": "fdab:2588:f4:0:f816:3eff:fe37:acba"} |
| id | 657990ce-3446-40a2-bc33-8040a32cb72b |
| mac_address | 90:e2:ba:48:27:ed |
| name | sriov_port |
| network_id | dfdff739-a27b-4160-9e40-c4824e8a351d |
| port_security_enabled | True |
| revision | 59 |
| security_groups | 8c7b8173-f050-480d-968a-33e0b75332fc |
| status | ACTIVE |
| tenant_id | a11cb2aece1943c2a86ee0a55e1bd8f7 |
| updated_at | 2016-08-26T17:38:30 |
+-----------------------+-----------------------------------------------------------------------------------------------------------+
Notice that it's still hasn't changed: 90:e2:ba:48:27:ed
8- Log in the guest and get a DHCP address for the interface.
ip link add link eth1 name eth1.451 type vlan id 451
ip link set eth1.451 up
dhclient eth1.451
Note the MAC address of the interface has changed:
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 00:1e:67:51:36:71 brd ff:ff:ff:ff:ff:ff
inet6 fe80::21e:67ff:fe51:3671/64 scope link
valid_lft forever preferred_lft forever
5: eth2.451@eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 00:1e:67:51:36:71 brd ff:ff:ff:ff:ff:ff
inet6 fe80::21e:67ff:fe51:3671/64 scope link
valid_lft forever preferred_lft forever
Also note that the DHCP client wasn't successful in getting a DHCP
address.
9- From the dnsmasq host file, we can also see that it hasn't been
updated:
cat /opt/stack/data/neutron/dhcp/dfdff739-a27b-4160-9e40-c4824e8a351d/host
fa:16:3e:e5:e0:c9,host-10-0-0-2.openstacklocal,10.0.0.2
90:e2:ba:48:27:ed,host-10-0-0-12.openstacklocal,10.0.0.12
fa:16:3e:99:c8:b9,host-10-0-0-5.openstacklocal,10.0.0.5
Environment:
============
Latest master
commit 7500bef94f526a82392400415f07f744700324a9
Merge: 8d5aff7 39fb302
Author: Jenkins <jenkins@xxxxxxxxxxxxxxxxxxxx>
Date: Fri Aug 26 05:03:48 2016 +0000
Merge "Revert "Optional separate database for placement API""
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1617429/+subscriptions
