yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #61857
[Bug 1658802] Re: Issue booting instance with normal port and macvtap agent
Reviewed: https://review.openstack.org/435144
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=3dddfa56db69b504408c6894a814a89ea63b05f0
Submitter: Jenkins
Branch: master
commit 3dddfa56db69b504408c6894a814a89ea63b05f0
Author: Kevin Benton <kevin@xxxxxxxxxx>
Date: Thu Feb 16 14:53:31 2017 -0800
Macvtap: Check for no original port in is_live_migration
Change 6865f4d9f22d5daa2f07ff9651c2280aed489c8c mistakenly
assumed that None would not be present for the 'original'
property on a PortContext. However, this is the default value
for the original field in PortContext, which is what is used
in the construction as part of the _create_port_db process
in ML2.
This resulted in binding failures for the macvtap mech drivers
due to an attribute error in cases like brand new ports.
This patch simply checks for None before trying to determine
if it's a live migration (which it isn't in the case of port creation).
Part of the issue is likely that the FakePortContext in the unit tests
was defaulting to an empty dict which is the not the same behavior as
the real PortContext.
Change-Id: I6659235a70aa4528fd21911c04e651194591e449
Closes-Bug: #1658802
** Changed in: neutron
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1658802
Title:
Issue booting instance with normal port and macvtap agent
Status in neutron:
Fix Released
Bug description:
OpenStack Release: Newton
Operating System: Ubuntu 16.04 LTS 4.4.0-45-generic
OpenStack Distro: OpenStack-Ansible 14.0.2
While working to test/implement macvtap functionality, I found it was
not possible to boot an instance when using the macvtap mech driver
and macvtap agent. My goal is/was to boot an instance with a 'normal'
(non-PCI) port, and have a macvtap interface created on the compute
node. What I found was that that booting the instance resulted in an
ERROR state, with errors reported in neutron-server.log[1] and nova-
compute.log[2]:
[1] Neutron Server: http://paste.openstack.org/show/596128/
[2] Nova Compute: http://paste.openstack.org/show/596129/
The Nova boot syntax can be seen here:
nova boot --image 'Cirros Test Image' --flavor 'm1.test' --nic port-
id=5f08dcec-6689-4c1c-9f95-ffb69548c606 --availability-zone=ZONE-B
TEST-1
An availability zone consisting of a single node was specified.
The interesting thing was observed (multiple times) in neutron-
server.log during the boot process:
2017-01-23 15:25:57.464 17265 ERROR neutron.plugins.ml2.managers [req-f531783a-8f29-423e-9764-026f4cd4693f 3e202337454b4813b0ce6ecff74d43d6 b009912cbdcd45a0b4ef34fb3d22e7e1 - - -] Mechanism driver macvtap failed in bind_port
2017-01-23 15:25:57.464 17265 ERROR neutron.plugins.ml2.managers Traceback (most recent call last):
2017-01-23 15:25:57.464 17265 ERROR neutron.plugins.ml2.managers File "/openstack/venvs/neutron-14.0.2/lib/python2.7/site-packages/neutron/plugins/ml2/managers.py", line 787, in _bind_port_level
2017-01-23 15:25:57.464 17265 ERROR neutron.plugins.ml2.managers driver.obj.bind_port(context)
2017-01-23 15:25:57.464 17265 ERROR neutron.plugins.ml2.managers File "/openstack/venvs/neutron-14.0.2/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/mech_agent.py", line 109, in bind_port
2017-01-23 15:25:57.464 17265 ERROR neutron.plugins.ml2.managers agent):
2017-01-23 15:25:57.464 17265 ERROR neutron.plugins.ml2.managers File "/openstack/venvs/neutron-14.0.2/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/macvtap/mech_driver/mech_macvtap.py", line 106, in try_to_bind_segment_for_agent
2017-01-23 15:25:57.464 17265 ERROR neutron.plugins.ml2.managers if self._is_live_migration(context):
2017-01-23 15:25:57.464 17265 ERROR neutron.plugins.ml2.managers File "/openstack/venvs/neutron-14.0.2/lib/python2.7/site-packages/neutron/plugins/ml2/drivers/macvtap/mech_driver/mech_macvtap.py", line 68, in _is_live_migration
2017-01-23 15:25:57.464 17265 ERROR neutron.plugins.ml2.managers port_profile = context.original.get(portbindings.PROFILE)
2017-01-23 15:25:57.464 17265 ERROR neutron.plugins.ml2.managers AttributeError: 'NoneType' object has no attribute 'get'
2017-01-23 15:25:57.464 17265 ERROR neutron.plugins.ml2.managers
The _is_live_migration function was implemented in [3]:
[3]
https://review.openstack.org/#/c/361301/5/neutron/plugins/ml2/drivers/macvtap/mech_driver/mech_macvtap.py
context.original is set to None, which is resulting in the Traceback
observed here. I have logged the object data at the time it hits the
function in [4]:
[4] Object: http://paste.openstack.org/show/596130/
To work around this issue for the sake of booting the instance, I set
'port_profile' to None in mech_macvtap.py and restarted Neutron
server. I was then able to boot the instance, and the agent created
the VLAN and macvtap interfaces accordingly:
+--------------------------------------+------------------------------------+--------+------------+-------------+------------------------------------------------------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------------------------------------+--------+------------+-------------+------------------------------------------------------------------------------+
| 0022df55-6500-44c0-8ff5-4433c37157c1 | TEST-1 | ERROR | - | NOSTATE | |
| 69eda7de-00ad-4927-955f-9e5e1d9d6909 | TEST-1 | ACTIVE | - | Running | TEST=192.168.7.8 |
113: em50.62@em50: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 5c:b9:01:88:fd:a5 brd ff:ff:ff:ff:ff:ff
inet6 fe80::5eb9:1ff:fe88:fda5/64 scope link
valid_lft forever preferred_lft forever
115: macvtap0@em50.62: <BROADCAST,MULTICAST,ALLMULTI,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 500
link/ether fa:16:3e:67:da:b8 brd ff:ff:ff:ff:ff:ff
inet6 fe80::f816:3eff:fe67:dab8/64 scope link
valid_lft forever preferred_lft forever
root@817769-i812846:~# tcpdump -i macvtap0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on macvtap0, link-type EN10MB (Ethernet), capture size 262144 bytes
15:30:52.524485 IP6 :: > ff02::1:ff67:dab8: ICMP6, neighbor solicitation, who has fe80::f816:3eff:fe67:dab8, length 24
15:30:52.994199 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:67:da:b8 (oui Unknown), length 290
15:30:52.998185 IP6 :: > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28
15:30:53.390218 IP6 :: > ff02::1:ff67:dab8: ICMP6, neighbor solicitation, who has fe80::f816:3eff:fe67:dab8, length 24
15:30:53.524385 IP6 fe80::f816:3eff:fe67:dab8 > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28
15:30:53.524420 IP6 fe80::f816:3eff:fe67:dab8 > ip6-allrouters: ICMP6, router solicitation, length 16
15:30:54.008276 IP6 fe80::f816:3eff:fe67:dab8 > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28
15:30:54.390184 IP6 fe80::f816:3eff:fe67:dab8 > ip6-allrouters: ICMP6, router solicitation, length 16
15:30:57.528300 IP6 fe80::f816:3eff:fe67:dab8 > ip6-allrouters: ICMP6, router solicitation, length 16
15:30:58.394100 IP6 fe80::f816:3eff:fe67:dab8 > ip6-allrouters: ICMP6, router solicitation, length 16
15:31:01.536286 IP6 fe80::f816:3eff:fe67:dab8 > ip6-allrouters: ICMP6, router solicitation, length 16
15:31:02.042017 IP6 fe80::f816:3eff:fe67:dab8 > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28
15:31:02.402014 IP6 fe80::f816:3eff:fe67:dab8 > ip6-allrouters: ICMP6, router solicitation, length 16
Port details can be seen here:
+-----------------------+-------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+-------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:host_id | 817769-i812846.NewtonTest.com |
| binding:profile | {} |
| binding:vif_details | {"port_filter": false, "macvtap_mode": "bridge", "vlan": 62, "physical_interface": "em50", "macvtap_source": "em50.62"} |
| binding:vif_type | macvtap |
| binding:vnic_type | normal |
| created_at | 2017-01-23T20:18:35Z |
| description | |
| device_id | 69eda7de-00ad-4927-955f-9e5e1d9d6909 |
| device_owner | compute:ZONE-B |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "898171fa-903e-4549-bb82-fbb5f16c9bb1", "ip_address": "192.168.7.8"} |
| id | 5f08dcec-6689-4c1c-9f95-ffb69548c606 |
| mac_address | fa:16:3e:67:da:b8 |
| name | TEST |
| network_id | 508e4e86-0137-4774-8df5-bd6919080bea |
| port_security_enabled | True |
| project_id | a2bc030466504dea81191913f1071a10 |
| revision_number | 23 |
| security_groups | |
| status | ACTIVE |
| tenant_id | a2bc030466504dea81191913f1071a10 |
| updated_at | 2017-01-23T20:29:07Z |
+-----------------------+-------------------------------------------------------------------------------------------------------------------------+
Please let me know what additional information you need, including
further troubleshooting or debug output.
James
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1658802/+subscriptions
References
