yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #23717
[Bug 1380934] [NEW] failure rebuilding instance after resize-revert
Public bug reported:
Short version:
Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException.
Long version:
Reproduce:
master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c)
1. create instanse
2. do resize(succcess and active in destination host/status is VERIFY_RESIZE)
3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN)
4. do rebuild
After timeout(value of vif_plugging_timeout), vm became ERROR cause of
VirtualInterfaceCreateException.
If you set vif_plugging_is_fatal to True, vm is removed.
Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE.
Cause:
When I do resize-revert, instance come back to source host, but host information is set to destination node.
(host information is 'host' of ml2_port_bindings table of neutron DB)
This lead to mismatch of actual neutron port and DB information of port.
So this port become to DOWN.
Do rebuild in this situation, port don't become to ACTIVE, so results in timuout.
Solution:
nova/compute/manager.py
def finish_revert_resize(self, context, instance, reservations, migration):
instance.system_metadata = sys_meta
instance.memory_mb = instance_type['memory_mb']
instance.vcpus = instance_type['vcpus']
instance.root_gb = instance_type['root_gb']
instance.ephemeral_gb = instance_type['ephemeral_gb']
instance.instance_type_id = instance_type['id']
instance.host = migration['source_compute']
instance.node = migration['source_node']
instance.save()
+migration.dest_compute = migration['source_compute']
+migration.save()
self.network_api.setup_networks_on_host(context, instance,
migration['source_compute'])
block_device_info = self._get_instance_block_device_info(
context, instance, refresh_conn_info=True)
** Affects: nova
Importance: Undecided
Status: New
** Description changed:
Short version:
- Do rebuild instance after resize, vm become ERROR cause of VirtualInterfaceCreateException.
-
+ Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException.
Long version:
Reproduce:
master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c)
1. create instanse
2. do resize(succcess and active in destination host/status is VERIFY_RESIZE)
3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN)
4. do rebuild
After timeout(value of vif_plugging_timeout), vm became ERROR cause of
VirtualInterfaceCreateException.
If you set vif_plugging_is_fatal to True, vm is removed.
Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE.
Cause:
When I do resize-revert, instance come back to source host, but host information is set to destination node.
(host information is 'host' of ml2_port_bindings table of neutron DB)
This lead to mismatch of actual neutron port and DB information of port.
So this port become to DOWN.
Do rebuild in this situation, port don't become to ACTIVE, so results in timuout.
-
Solution:
nova/compute/manager.py
- def finish_revert_resize(self, context, instance, reservations, migration):
- instance.system_metadata = sys_meta
- instance.memory_mb = instance_type['memory_mb']
- instance.vcpus = instance_type['vcpus']
- instance.root_gb = instance_type['root_gb']
- instance.ephemeral_gb = instance_type['ephemeral_gb']
- instance.instance_type_id = instance_type['id']
- instance.host = migration['source_compute']
- instance.node = migration['source_node']
- instance.save()
+ def finish_revert_resize(self, context, instance, reservations, migration):
+ instance.system_metadata = sys_meta
+ instance.memory_mb = instance_type['memory_mb']
+ instance.vcpus = instance_type['vcpus']
+ instance.root_gb = instance_type['root_gb']
+ instance.ephemeral_gb = instance_type['ephemeral_gb']
+ instance.instance_type_id = instance_type['id']
+ instance.host = migration['source_compute']
+ instance.node = migration['source_node']
+ instance.save()
+ migration.dest_compute = migration['source_compute']
+ migration.save()
+
- self.network_api.setup_networks_on_host(context, instance,
- migration['source_compute'])
+ self.network_api.setup_networks_on_host(context, instance,
+ migration['source_compute'])
- block_device_info = self._get_instance_block_device_info(
- context, instance, refresh_conn_info=True)
+ block_device_info = self._get_instance_block_device_info(
+ context, instance, refresh_conn_info=True)
** Description changed:
Short version:
Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException.
Long version:
Reproduce:
master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c)
1. create instanse
2. do resize(succcess and active in destination host/status is VERIFY_RESIZE)
3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN)
4. do rebuild
After timeout(value of vif_plugging_timeout), vm became ERROR cause of
VirtualInterfaceCreateException.
If you set vif_plugging_is_fatal to True, vm is removed.
Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE.
Cause:
When I do resize-revert, instance come back to source host, but host information is set to destination node.
(host information is 'host' of ml2_port_bindings table of neutron DB)
This lead to mismatch of actual neutron port and DB information of port.
So this port become to DOWN.
Do rebuild in this situation, port don't become to ACTIVE, so results in timuout.
Solution:
nova/compute/manager.py
def finish_revert_resize(self, context, instance, reservations, migration):
instance.system_metadata = sys_meta
instance.memory_mb = instance_type['memory_mb']
instance.vcpus = instance_type['vcpus']
instance.root_gb = instance_type['root_gb']
instance.ephemeral_gb = instance_type['ephemeral_gb']
instance.instance_type_id = instance_type['id']
instance.host = migration['source_compute']
instance.node = migration['source_node']
instance.save()
- + migration.dest_compute = migration['source_compute']
- + migration.save()
- +
+ + migration.dest_compute = migration['source_compute']
+ + migration.save()
+ +
self.network_api.setup_networks_on_host(context, instance,
migration['source_compute'])
block_device_info = self._get_instance_block_device_info(
context, instance, refresh_conn_info=True)
** Description changed:
Short version:
Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException.
Long version:
Reproduce:
master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c)
1. create instanse
2. do resize(succcess and active in destination host/status is VERIFY_RESIZE)
3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN)
4. do rebuild
After timeout(value of vif_plugging_timeout), vm became ERROR cause of
VirtualInterfaceCreateException.
If you set vif_plugging_is_fatal to True, vm is removed.
Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE.
Cause:
When I do resize-revert, instance come back to source host, but host information is set to destination node.
(host information is 'host' of ml2_port_bindings table of neutron DB)
This lead to mismatch of actual neutron port and DB information of port.
So this port become to DOWN.
Do rebuild in this situation, port don't become to ACTIVE, so results in timuout.
Solution:
nova/compute/manager.py
def finish_revert_resize(self, context, instance, reservations, migration):
instance.system_metadata = sys_meta
instance.memory_mb = instance_type['memory_mb']
instance.vcpus = instance_type['vcpus']
instance.root_gb = instance_type['root_gb']
instance.ephemeral_gb = instance_type['ephemeral_gb']
instance.instance_type_id = instance_type['id']
instance.host = migration['source_compute']
instance.node = migration['source_node']
instance.save()
- + migration.dest_compute = migration['source_compute']
- + migration.save()
- +
+ + migration.dest_compute = migration['source_compute']
+ + migration.save()
+ +
self.network_api.setup_networks_on_host(context, instance,
migration['source_compute'])
block_device_info = self._get_instance_block_device_info(
context, instance, refresh_conn_info=True)
** Description changed:
Short version:
Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException.
Long version:
Reproduce:
master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c)
1. create instanse
2. do resize(succcess and active in destination host/status is VERIFY_RESIZE)
3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN)
4. do rebuild
After timeout(value of vif_plugging_timeout), vm became ERROR cause of
VirtualInterfaceCreateException.
If you set vif_plugging_is_fatal to True, vm is removed.
Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE.
Cause:
When I do resize-revert, instance come back to source host, but host information is set to destination node.
(host information is 'host' of ml2_port_bindings table of neutron DB)
This lead to mismatch of actual neutron port and DB information of port.
So this port become to DOWN.
Do rebuild in this situation, port don't become to ACTIVE, so results in timuout.
Solution:
nova/compute/manager.py
+ def finish_revert_resize(self, context, instance, reservations, migration):
+ instance.system_metadata = sys_meta
+ instance.memory_mb = instance_type['memory_mb']
+ instance.vcpus = instance_type['vcpus']
+ instance.root_gb = instance_type['root_gb']
+ instance.ephemeral_gb = instance_type['ephemeral_gb']
+ instance.instance_type_id = instance_type['id']
+ instance.host = migration['source_compute']
+ instance.node = migration['source_node']
+ instance.save()
- def finish_revert_resize(self, context, instance, reservations, migration):
- instance.system_metadata = sys_meta
- instance.memory_mb = instance_type['memory_mb']
- instance.vcpus = instance_type['vcpus']
- instance.root_gb = instance_type['root_gb']
- instance.ephemeral_gb = instance_type['ephemeral_gb']
- instance.instance_type_id = instance_type['id']
- instance.host = migration['source_compute']
- instance.node = migration['source_node']
- instance.save()
+ + migration.dest_compute = migration['source_compute']
+ + migration.save()
- + migration.dest_compute = migration['source_compute']
- + migration.save()
- +
- self.network_api.setup_networks_on_host(context, instance,
- migration['source_compute'])
+ self.network_api.setup_networks_on_host(context, instance,
+ migration['source_compute'])
- block_device_info = self._get_instance_block_device_info(
- context, instance, refresh_conn_info=True)
+ block_device_info = self._get_instance_block_device_info(
+ context, instance, refresh_conn_info=True)
** Description changed:
Short version:
Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException.
Long version:
Reproduce:
master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c)
1. create instanse
2. do resize(succcess and active in destination host/status is VERIFY_RESIZE)
3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN)
4. do rebuild
After timeout(value of vif_plugging_timeout), vm became ERROR cause of
VirtualInterfaceCreateException.
If you set vif_plugging_is_fatal to True, vm is removed.
Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE.
Cause:
When I do resize-revert, instance come back to source host, but host information is set to destination node.
(host information is 'host' of ml2_port_bindings table of neutron DB)
This lead to mismatch of actual neutron port and DB information of port.
So this port become to DOWN.
Do rebuild in this situation, port don't become to ACTIVE, so results in timuout.
Solution:
nova/compute/manager.py
def finish_revert_resize(self, context, instance, reservations, migration):
- instance.system_metadata = sys_meta
- instance.memory_mb = instance_type['memory_mb']
- instance.vcpus = instance_type['vcpus']
- instance.root_gb = instance_type['root_gb']
- instance.ephemeral_gb = instance_type['ephemeral_gb']
- instance.instance_type_id = instance_type['id']
- instance.host = migration['source_compute']
- instance.node = migration['source_node']
- instance.save()
+ instance.system_metadata = sys_meta
+ instance.memory_mb = instance_type['memory_mb']
+ instance.vcpus = instance_type['vcpus']
+ instance.root_gb = instance_type['root_gb']
+ instance.ephemeral_gb = instance_type['ephemeral_gb']
+ instance.instance_type_id = instance_type['id']
+ instance.host = migration['source_compute']
+ instance.node = migration['source_node']
+ instance.save()
- + migration.dest_compute = migration['source_compute']
- + migration.save()
+ + migration.dest_compute = migration['source_compute']
+ + migration.save()
- self.network_api.setup_networks_on_host(context, instance,
- migration['source_compute'])
+ self.network_api.setup_networks_on_host(context, instance,
+ migration['source_compute'])
- block_device_info = self._get_instance_block_device_info(
- context, instance, refresh_conn_info=True)
+ block_device_info = self._get_instance_block_device_info(
+ context, instance, refresh_conn_info=True)
** Description changed:
Short version:
Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException.
Long version:
Reproduce:
master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c)
1. create instanse
2. do resize(succcess and active in destination host/status is VERIFY_RESIZE)
3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN)
4. do rebuild
After timeout(value of vif_plugging_timeout), vm became ERROR cause of
VirtualInterfaceCreateException.
If you set vif_plugging_is_fatal to True, vm is removed.
Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE.
Cause:
When I do resize-revert, instance come back to source host, but host information is set to destination node.
(host information is 'host' of ml2_port_bindings table of neutron DB)
This lead to mismatch of actual neutron port and DB information of port.
So this port become to DOWN.
Do rebuild in this situation, port don't become to ACTIVE, so results in timuout.
Solution:
nova/compute/manager.py
def finish_revert_resize(self, context, instance, reservations, migration):
instance.system_metadata = sys_meta
instance.memory_mb = instance_type['memory_mb']
instance.vcpus = instance_type['vcpus']
instance.root_gb = instance_type['root_gb']
instance.ephemeral_gb = instance_type['ephemeral_gb']
instance.instance_type_id = instance_type['id']
instance.host = migration['source_compute']
instance.node = migration['source_node']
instance.save()
- + migration.dest_compute = migration['source_compute']
- + migration.save()
+ + migration.dest_compute = migration['source_compute']
+ + migration.save()
self.network_api.setup_networks_on_host(context, instance,
migration['source_compute'])
block_device_info = self._get_instance_block_device_info(
context, instance, refresh_conn_info=True)
** Description changed:
Short version:
Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException.
Long version:
Reproduce:
master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c)
1. create instanse
2. do resize(succcess and active in destination host/status is VERIFY_RESIZE)
3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN)
4. do rebuild
After timeout(value of vif_plugging_timeout), vm became ERROR cause of
VirtualInterfaceCreateException.
If you set vif_plugging_is_fatal to True, vm is removed.
Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE.
Cause:
When I do resize-revert, instance come back to source host, but host information is set to destination node.
(host information is 'host' of ml2_port_bindings table of neutron DB)
This lead to mismatch of actual neutron port and DB information of port.
So this port become to DOWN.
Do rebuild in this situation, port don't become to ACTIVE, so results in timuout.
Solution:
nova/compute/manager.py
def finish_revert_resize(self, context, instance, reservations, migration):
instance.system_metadata = sys_meta
instance.memory_mb = instance_type['memory_mb']
instance.vcpus = instance_type['vcpus']
instance.root_gb = instance_type['root_gb']
instance.ephemeral_gb = instance_type['ephemeral_gb']
instance.instance_type_id = instance_type['id']
instance.host = migration['source_compute']
instance.node = migration['source_node']
instance.save()
- + migration.dest_compute = migration['source_compute']
- + migration.save()
+ + migration.dest_compute = migration['source_compute']
+ + migration.save()
self.network_api.setup_networks_on_host(context, instance,
migration['source_compute'])
block_device_info = self._get_instance_block_device_info(
context, instance, refresh_conn_info=True)
** Description changed:
Short version:
Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException.
Long version:
Reproduce:
master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c)
1. create instanse
2. do resize(succcess and active in destination host/status is VERIFY_RESIZE)
3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN)
4. do rebuild
After timeout(value of vif_plugging_timeout), vm became ERROR cause of
VirtualInterfaceCreateException.
If you set vif_plugging_is_fatal to True, vm is removed.
Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE.
Cause:
When I do resize-revert, instance come back to source host, but host information is set to destination node.
(host information is 'host' of ml2_port_bindings table of neutron DB)
This lead to mismatch of actual neutron port and DB information of port.
So this port become to DOWN.
Do rebuild in this situation, port don't become to ACTIVE, so results in timuout.
Solution:
nova/compute/manager.py
def finish_revert_resize(self, context, instance, reservations, migration):
instance.system_metadata = sys_meta
instance.memory_mb = instance_type['memory_mb']
instance.vcpus = instance_type['vcpus']
instance.root_gb = instance_type['root_gb']
instance.ephemeral_gb = instance_type['ephemeral_gb']
instance.instance_type_id = instance_type['id']
instance.host = migration['source_compute']
instance.node = migration['source_node']
instance.save()
- + migration.dest_compute = migration['source_compute']
+ + migration.dest_compute = migration['source_compute']
+ migration.save()
self.network_api.setup_networks_on_host(context, instance,
migration['source_compute'])
block_device_info = self._get_instance_block_device_info(
context, instance, refresh_conn_info=True)
** Description changed:
Short version:
Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException.
Long version:
Reproduce:
master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c)
1. create instanse
2. do resize(succcess and active in destination host/status is VERIFY_RESIZE)
3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN)
4. do rebuild
After timeout(value of vif_plugging_timeout), vm became ERROR cause of
VirtualInterfaceCreateException.
If you set vif_plugging_is_fatal to True, vm is removed.
Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE.
Cause:
When I do resize-revert, instance come back to source host, but host information is set to destination node.
(host information is 'host' of ml2_port_bindings table of neutron DB)
This lead to mismatch of actual neutron port and DB information of port.
So this port become to DOWN.
Do rebuild in this situation, port don't become to ACTIVE, so results in timuout.
Solution:
nova/compute/manager.py
def finish_revert_resize(self, context, instance, reservations, migration):
instance.system_metadata = sys_meta
instance.memory_mb = instance_type['memory_mb']
instance.vcpus = instance_type['vcpus']
instance.root_gb = instance_type['root_gb']
instance.ephemeral_gb = instance_type['ephemeral_gb']
instance.instance_type_id = instance_type['id']
instance.host = migration['source_compute']
instance.node = migration['source_node']
instance.save()
- + migration.dest_compute = migration['source_compute']
- + migration.save()
+ migration.dest_compute = migration['source_compute']
+ migration.save()
self.network_api.setup_networks_on_host(context, instance,
migration['source_compute'])
block_device_info = self._get_instance_block_device_info(
context, instance, refresh_conn_info=True)
** Description changed:
Short version:
Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException.
Long version:
Reproduce:
master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c)
1. create instanse
2. do resize(succcess and active in destination host/status is VERIFY_RESIZE)
3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN)
4. do rebuild
After timeout(value of vif_plugging_timeout), vm became ERROR cause of
VirtualInterfaceCreateException.
If you set vif_plugging_is_fatal to True, vm is removed.
Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE.
Cause:
When I do resize-revert, instance come back to source host, but host information is set to destination node.
(host information is 'host' of ml2_port_bindings table of neutron DB)
This lead to mismatch of actual neutron port and DB information of port.
So this port become to DOWN.
Do rebuild in this situation, port don't become to ACTIVE, so results in timuout.
Solution:
nova/compute/manager.py
def finish_revert_resize(self, context, instance, reservations, migration):
- instance.system_metadata = sys_meta
- instance.memory_mb = instance_type['memory_mb']
- instance.vcpus = instance_type['vcpus']
- instance.root_gb = instance_type['root_gb']
- instance.ephemeral_gb = instance_type['ephemeral_gb']
- instance.instance_type_id = instance_type['id']
- instance.host = migration['source_compute']
- instance.node = migration['source_node']
- instance.save()
+ instance.system_metadata = sys_meta
+ instance.memory_mb = instance_type['memory_mb']
+ instance.vcpus = instance_type['vcpus']
+ instance.root_gb = instance_type['root_gb']
+ instance.ephemeral_gb = instance_type['ephemeral_gb']
+ instance.instance_type_id = instance_type['id']
+ instance.host = migration['source_compute']
+ instance.node = migration['source_node']
+ instance.save()
- migration.dest_compute = migration['source_compute']
- migration.save()
+ +migration.dest_compute = migration['source_compute']
+ +migration.save()
- self.network_api.setup_networks_on_host(context, instance,
- migration['source_compute'])
+ self.network_api.setup_networks_on_host(context, instance,
+ migration['source_compute'])
- block_device_info = self._get_instance_block_device_info(
- context, instance, refresh_conn_info=True)
+ block_device_info = self._get_instance_block_device_info(
+ context, instance, refresh_conn_info=True)
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1380934
Title:
failure rebuilding instance after resize-revert
Status in OpenStack Compute (Nova):
New
Bug description:
Short version:
Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException.
Long version:
Reproduce:
master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c)
1. create instanse
2. do resize(succcess and active in destination host/status is VERIFY_RESIZE)
3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN)
4. do rebuild
After timeout(value of vif_plugging_timeout), vm became ERROR cause of
VirtualInterfaceCreateException.
If you set vif_plugging_is_fatal to True, vm is removed.
Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE.
Cause:
When I do resize-revert, instance come back to source host, but host information is set to destination node.
(host information is 'host' of ml2_port_bindings table of neutron DB)
This lead to mismatch of actual neutron port and DB information of port.
So this port become to DOWN.
Do rebuild in this situation, port don't become to ACTIVE, so results in timuout.
Solution:
nova/compute/manager.py
def finish_revert_resize(self, context, instance, reservations, migration):
instance.system_metadata = sys_meta
instance.memory_mb = instance_type['memory_mb']
instance.vcpus = instance_type['vcpus']
instance.root_gb = instance_type['root_gb']
instance.ephemeral_gb = instance_type['ephemeral_gb']
instance.instance_type_id = instance_type['id']
instance.host = migration['source_compute']
instance.node = migration['source_node']
instance.save()
+migration.dest_compute = migration['source_compute']
+migration.save()
self.network_api.setup_networks_on_host(context, instance,
migration['source_compute'])
block_device_info = self._get_instance_block_device_info(
context, instance, refresh_conn_info=True)
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1380934/+subscriptions
Follow ups
References