← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #49845

[Bug 1573165] [NEW] Cannot open network namespace when booting up instances

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

# neutron --version
3.1.0

When booting up instance using nova boot command, I am getting ERROR status on the instance
# nova list
+--------------------------------------+-------+--------+------------+-------------+--------------------+
| ID                                   | Name  | Status | Task State | Power State | Networks           |
+--------------------------------------+-------+--------+------------+-------------+--------------------+
| 98f341b8-19e4-4f14-930f-8c27bc0dc6f9 | test1 | ERROR  | -          | NOSTATE     | public=10.40.18.12 |
+--------------------------------------+-------+--------+------------+-------------+--------------------+
# neutron  net-list
+--------------------------------------+--------+----------------------------------------------------+
| id                                   | name   | subnets                                            |
+--------------------------------------+--------+----------------------------------------------------+
| 5e8aabe0-9561-4f78-82b9-1ca0a350e533 | public | f020c35c-0a8b-495a-bd7d-e9d8de4188de 10.40.16.0/22 |
+--------------------------------------+--------+----------------------------------------------------+
# neutron net-show 5e8aabe0-9561-4f78-82b9-1ca0a350e533
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | True                                 |
| id                        | 5e8aabe0-9561-4f78-82b9-1ca0a350e533 |
| mtu                       | 0                                    |
| name                      | public                               |
| port_security_enabled     | True                                 |
| provider:network_type     | flat                                 |
| provider:physical_network | public                               |
| provider:segmentation_id  |                                      |
| router:external           | False                                |
| shared                    | True                                 |
| status                    | ACTIVE                               |
| subnets                   | f020c35c-0a8b-495a-bd7d-e9d8de4188de |
| tenant_id                 | d78b9cabab66472b937d364ebfce3986     |
+---------------------------+--------------------------------------+

in the nova-conductor.log file on controller
2016-04-20 16:58:22.132 28114 ERROR nova.scheduler.utils [req-93298eea-2308-44ed-bb75-68f83c55dc10 8da5babc58424fffaf8a23bbc0276739 d78b9cabab66472b937d364ebfce3986 - - -] [instance: 98f341b8-19e4-4f14-930f-8c27bc0dc6f9] Error from last host: <compute-host> (node<compute-host>: [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1905, in _do_build_and_run_instance\n    filter_properties)\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2057, in _build_and_run_instance\n    instance_uuid=instance.uuid, reason=six.text_type(e))\n', u'RescheduledException: Build of instance 98f341b8-19e4-4f14-930f-8c27bc0dc6f9 was re-scheduled: Cannot setup network: Unexpected error while running command.\nCommand: sudo nova-rootwrap /etc/nova/rootwrap.conf ip netns exec 70856f7f66e6f3354be5b1ce212c0c299e0b652ea0a7100e1595b1bbac5ea91c ip link set lo up\nExit code: 1\nStdout: u\'\'\nStderr: u\'Cannot open network namespace "70856f7f66e6f3354be5b1ce212c0c299e0b652ea0a7100e1595b1bbac5ea91c": Permission denied\\n\'\n']


on Compute host:
# ip netns
d8b5b2e5f869a35357eb79487f9c3e2e55c2378af2484f218e6152a43517dc39
70856f7f66e6f3354be5b1ce212c0c299e0b652ea0a7100e1595b1bbac5ea91c
f66c01f7d829162c137dd9803c035e635cfc6cdc079da01ae94e856170340d4b
# ls -l
total 0
lrwxrwxrwx. 1 root root 18 Apr 20 17:58 70856f7f66e6f3354be5b1ce212c0c299e0b652ea0a7100e1595b1bbac5ea91c -> /proc/33051/ns/net
lrwxrwxrwx. 1 root root 18 Apr 20 17:58 d8b5b2e5f869a35357eb79487f9c3e2e55c2378af2484f218e6152a43517dc39 -> /proc/33130/ns/net
lrwxrwxrwx. 1 root root 18 Apr 20 17:58 f66c01f7d829162c137dd9803c035e635cfc6cdc079da01ae94e856170340d4b -> /proc/32852/ns/net

# ls -l /proc/33051/ns/net
ls: cannot access /proc/33051/ns/net: No such file or directory

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1573165

Title:
  Cannot open network namespace when booting up  instances

Status in neutron:
  New

Bug description:
  # neutron --version
  3.1.0

  When booting up instance using nova boot command, I am getting ERROR status on the instance
  # nova list
  +--------------------------------------+-------+--------+------------+-------------+--------------------+
  | ID                                   | Name  | Status | Task State | Power State | Networks           |
  +--------------------------------------+-------+--------+------------+-------------+--------------------+
  | 98f341b8-19e4-4f14-930f-8c27bc0dc6f9 | test1 | ERROR  | -          | NOSTATE     | public=10.40.18.12 |
  +--------------------------------------+-------+--------+------------+-------------+--------------------+
  # neutron  net-list
  +--------------------------------------+--------+----------------------------------------------------+
  | id                                   | name   | subnets                                            |
  +--------------------------------------+--------+----------------------------------------------------+
  | 5e8aabe0-9561-4f78-82b9-1ca0a350e533 | public | f020c35c-0a8b-495a-bd7d-e9d8de4188de 10.40.16.0/22 |
  +--------------------------------------+--------+----------------------------------------------------+
  # neutron net-show 5e8aabe0-9561-4f78-82b9-1ca0a350e533
  +---------------------------+--------------------------------------+
  | Field                     | Value                                |
  +---------------------------+--------------------------------------+
  | admin_state_up            | True                                 |
  | id                        | 5e8aabe0-9561-4f78-82b9-1ca0a350e533 |
  | mtu                       | 0                                    |
  | name                      | public                               |
  | port_security_enabled     | True                                 |
  | provider:network_type     | flat                                 |
  | provider:physical_network | public                               |
  | provider:segmentation_id  |                                      |
  | router:external           | False                                |
  | shared                    | True                                 |
  | status                    | ACTIVE                               |
  | subnets                   | f020c35c-0a8b-495a-bd7d-e9d8de4188de |
  | tenant_id                 | d78b9cabab66472b937d364ebfce3986     |
  +---------------------------+--------------------------------------+

  in the nova-conductor.log file on controller
  2016-04-20 16:58:22.132 28114 ERROR nova.scheduler.utils [req-93298eea-2308-44ed-bb75-68f83c55dc10 8da5babc58424fffaf8a23bbc0276739 d78b9cabab66472b937d364ebfce3986 - - -] [instance: 98f341b8-19e4-4f14-930f-8c27bc0dc6f9] Error from last host: <compute-host> (node<compute-host>: [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1905, in _do_build_and_run_instance\n    filter_properties)\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2057, in _build_and_run_instance\n    instance_uuid=instance.uuid, reason=six.text_type(e))\n', u'RescheduledException: Build of instance 98f341b8-19e4-4f14-930f-8c27bc0dc6f9 was re-scheduled: Cannot setup network: Unexpected error while running command.\nCommand: sudo nova-rootwrap /etc/nova/rootwrap.conf ip netns exec 70856f7f66e6f3354be5b1ce212c0c299e0b652ea0a7100e1595b1bbac5ea91c ip link set lo up\nExit code: 1\nStdout: u\'\'\nStderr: u\'Cannot open network namespace "70856f7f66e6f3354be5b1ce212c0c299e0b652ea0a7100e1595b1bbac5ea91c": Permission denied\\n\'\n']

  
  on Compute host:
  # ip netns
  d8b5b2e5f869a35357eb79487f9c3e2e55c2378af2484f218e6152a43517dc39
  70856f7f66e6f3354be5b1ce212c0c299e0b652ea0a7100e1595b1bbac5ea91c
  f66c01f7d829162c137dd9803c035e635cfc6cdc079da01ae94e856170340d4b
  # ls -l
  total 0
  lrwxrwxrwx. 1 root root 18 Apr 20 17:58 70856f7f66e6f3354be5b1ce212c0c299e0b652ea0a7100e1595b1bbac5ea91c -> /proc/33051/ns/net
  lrwxrwxrwx. 1 root root 18 Apr 20 17:58 d8b5b2e5f869a35357eb79487f9c3e2e55c2378af2484f218e6152a43517dc39 -> /proc/33130/ns/net
  lrwxrwxrwx. 1 root root 18 Apr 20 17:58 f66c01f7d829162c137dd9803c035e635cfc6cdc079da01ae94e856170340d4b -> /proc/32852/ns/net

  # ls -l /proc/33051/ns/net
  ls: cannot access /proc/33051/ns/net: No such file or directory

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1573165/+subscriptions

Follow ups

  Thread Previous • Date Previous • • Thread Next