← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1367999] [NEW] live-migration causes VM network disconnected forever

 

Public bug reported:

OS: RHEL 6.5
OpenStack: RDO icehouse and master
Neutron: Linuxbridge + VxLAN + L2pop
Testbed: 1 controller node + 2 compute nodes + 1 network node

Reproduction procedure:

1. Start to ping VM from qrouter namespace using fixed IP
    Start to ping VM from outside using floating IP

2. Live-migrate one VM from compute1 to computer2

3. VM Network disconnects after several seconds

4. Even if Nova reports that the migration is finished,
Ping is still not working.

Debug Info on network node:

Command: ['sudo', 'bridge', 'fdb', 'add', 'fa:16:3e:b3:fd:27', 'dev', 'vxlan-1', 'dst', '192.168.2.103']
Exit code: 2
Stdout: ''
Stderr: 'RTNETLINK answers: File exists\n'

Cause:
Before migration, the original fdb entry is there. After migration, l2pop will updates the fdb entry of the VM.
It adds the new entry that causes ERROR.

The right operation should be 'replace' not 'add'.

By the way, 'replace' will safely add the new entry if old entry is not
existed.

I think this bug can be marked as High.

** Affects: neutron
     Importance: Undecided
     Assignee: Li Ma (nick-ma-z)
         Status: New

** Changed in: neutron
     Assignee: (unassigned) => Li Ma (nick-ma-z)

** Description changed:

  OS: RHEL 6.5
  OpenStack: RDO icehouse and master
  Neutron: Linuxbridge + VxLAN + L2pop
  Testbed: 1 controller node + 2 compute nodes + 1 network node
  
  Reproduction procedure:
  
  1. Start to ping VM from qrouter namespace using fixed IP
-     Start to ping VM from outside using floating IP
+     Start to ping VM from outside using floating IP
  
  2. Live-migrate one VM from compute1 to computer2
  
  3. VM Network disconnects after several seconds
  
- 4. Even if Nova reports that the migration is finished, 
+ 4. Even if Nova reports that the migration is finished,
  Ping is still not working.
- 
  
  Debug Info on network node:
  
  Command: ['sudo', 'bridge', 'fdb', 'add', 'fa:16:3e:b3:fd:27', 'dev', 'vxlan-1', 'dst', '192.168.2.103']
  Exit code: 2
  Stdout: ''
  Stderr: 'RTNETLINK answers: File exists\n'
  
- 
  Cause:
  Before migration, the original fdb entry is there. After migration, l2pop will updates the fdb entry of the VM.
  It adds the new entry that causes ERROR.
  
  The right operation should be 'replace' not 'add'.
  
+ By the way, 'replace' will safely add the new entry if old entry is not
+ existed.
  
  I think this bug can be marked as High.

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1367999

Title:
  live-migration causes VM network disconnected forever

Status in OpenStack Neutron (virtual network service):
  New

Bug description:
  OS: RHEL 6.5
  OpenStack: RDO icehouse and master
  Neutron: Linuxbridge + VxLAN + L2pop
  Testbed: 1 controller node + 2 compute nodes + 1 network node

  Reproduction procedure:

  1. Start to ping VM from qrouter namespace using fixed IP
      Start to ping VM from outside using floating IP

  2. Live-migrate one VM from compute1 to computer2

  3. VM Network disconnects after several seconds

  4. Even if Nova reports that the migration is finished,
  Ping is still not working.

  Debug Info on network node:

  Command: ['sudo', 'bridge', 'fdb', 'add', 'fa:16:3e:b3:fd:27', 'dev', 'vxlan-1', 'dst', '192.168.2.103']
  Exit code: 2
  Stdout: ''
  Stderr: 'RTNETLINK answers: File exists\n'

  Cause:
  Before migration, the original fdb entry is there. After migration, l2pop will updates the fdb entry of the VM.
  It adds the new entry that causes ERROR.

  The right operation should be 'replace' not 'add'.

  By the way, 'replace' will safely add the new entry if old entry is
  not existed.

  I think this bug can be marked as High.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1367999/+subscriptions


Follow ups

References