yahoo-eng-team team mailing list archive

[Paul Michali <1258952@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

I'm running tox -e py27 under Ubuntu 12.04 desktop 64 bit in a VM on my
laptop with 4 GB or RAM allocated. The memory in the run creeps up to
4GB and then near the end, it drops back down to about 2GB and in the
end, I get numerous tox failures, many indicating out of memory and too
many open files:

...
======================================================================
FAIL: neutron.tests.unit.test_agent_linux_utils.AgentUtilsExecuteTest.test_without_helper
tags: worker-0
----------------------------------------------------------------------
Empty attachments:
  pythonlogging:''
  stderr
  stdout

Traceback (most recent call last):
  File "/opt/stack/neutron/neutron/tests/unit/test_agent_linux_utils.py", line 39, in test_without_helper
    result = utils.execute(["ls", self.test_file])
  File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 65, in execute
    addl_env=addl_env)
  File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 56, in create_process
    env=env)
  File "/opt/stack/neutron/neutron/common/utils.py", line 125, in subprocess_popen
    close_fds=True, env=env)
  File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/eventlet/green/subprocess.py", line 44, in __init__
    subprocess_orig.Popen.__init__(self, args, 0, *argss, **kwds)
  File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1143, in _execute_child
    self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory
======================================================================
FAIL: process-returncode
tags: worker-0
----------------------------------------------------------------------
Binary content:
  traceback (test/plain; charset="utf8")
======================================================================
FAIL: neutron.tests.unit.test_agent_linux_utils.AgentUtilsExecuteTest.test_check_exit_code
tags: worker-1
----------------------------------------------------------------------
Empty attachments:
  pythonlogging:''
  stderr
  stdout

Traceback (most recent call last):
  File "/opt/stack/neutron/neutron/tests/unit/test_agent_linux_utils.py", line 59, in test_check_exit_code
    check_exit_code=False)
  File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 65, in execute
    addl_env=addl_env)
  File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 56, in create_process
    env=env)
  File "/opt/stack/neutron/neutron/common/utils.py", line 125, in subprocess_popen
    close_fds=True, env=env)
  File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/eventlet/green/subprocess.py", line 44, in __init__
    subprocess_orig.Popen.__init__(self, args, 0, *argss, **kwds)
  File "/usr/lib/python2.7/subprocess.py", line 672, in __init__
    errread, errwrite) = self._get_handles(stdin, stdout, stderr)
  File "/usr/lib/python2.7/subprocess.py", line 1038, in _get_handles
    p2cread, p2cwrite = self.pipe_cloexec()
  File "/usr/lib/python2.7/subprocess.py", line 1091, in pipe_cloexec
    r, w = os.pipe()
OSError: [Errno 24] Too many open files
======================================================================
...

Ran 9290 (+1703) tests in 2704.305s (+299.051s)
FAILED (id=1, failures=194 (+185), skips=324)

Here is the version info:

cm@vpn-a[2853] $ git log -n 1
commit 3014e1e021b3fe59c75daae1734472c3a11582ee
Merge: e22626e 5652e20
Author: Jenkins <jenkins@xxxxxxxxxxxxxxxxxxxx>
Date:   Sat Dec 7 23:05:30 2013 +0000

    Merge "Preserve floating ips when initializing l3 gateway interface"

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1258952

Title:
  tox failues due to out of memory

Status in OpenStack Neutron (virtual network service):
  New

Bug description:
  I'm running tox -e py27 under Ubuntu 12.04 desktop 64 bit in a VM on
  my laptop with 4 GB or RAM allocated. The memory in the run creeps up
  to 4GB and then near the end, it drops back down to about 2GB and in
  the end, I get numerous tox failures, many indicating out of memory
  and too many open files:

  ...
  ======================================================================
  FAIL: neutron.tests.unit.test_agent_linux_utils.AgentUtilsExecuteTest.test_without_helper
  tags: worker-0
  ----------------------------------------------------------------------
  Empty attachments:
    pythonlogging:''
    stderr
    stdout

  Traceback (most recent call last):
    File "/opt/stack/neutron/neutron/tests/unit/test_agent_linux_utils.py", line 39, in test_without_helper
      result = utils.execute(["ls", self.test_file])
    File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 65, in execute
      addl_env=addl_env)
    File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 56, in create_process
      env=env)
    File "/opt/stack/neutron/neutron/common/utils.py", line 125, in subprocess_popen
      close_fds=True, env=env)
    File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/eventlet/green/subprocess.py", line 44, in __init__
      subprocess_orig.Popen.__init__(self, args, 0, *argss, **kwds)
    File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
      errread, errwrite)
    File "/usr/lib/python2.7/subprocess.py", line 1143, in _execute_child
      self.pid = os.fork()
  OSError: [Errno 12] Cannot allocate memory
  ======================================================================
  FAIL: process-returncode
  tags: worker-0
  ----------------------------------------------------------------------
  Binary content:
    traceback (test/plain; charset="utf8")
  ======================================================================
  FAIL: neutron.tests.unit.test_agent_linux_utils.AgentUtilsExecuteTest.test_check_exit_code
  tags: worker-1
  ----------------------------------------------------------------------
  Empty attachments:
    pythonlogging:''
    stderr
    stdout

  Traceback (most recent call last):
    File "/opt/stack/neutron/neutron/tests/unit/test_agent_linux_utils.py", line 59, in test_check_exit_code
      check_exit_code=False)
    File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 65, in execute
      addl_env=addl_env)
    File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 56, in create_process
      env=env)
    File "/opt/stack/neutron/neutron/common/utils.py", line 125, in subprocess_popen
      close_fds=True, env=env)
    File "/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/eventlet/green/subprocess.py", line 44, in __init__
      subprocess_orig.Popen.__init__(self, args, 0, *argss, **kwds)
    File "/usr/lib/python2.7/subprocess.py", line 672, in __init__
      errread, errwrite) = self._get_handles(stdin, stdout, stderr)
    File "/usr/lib/python2.7/subprocess.py", line 1038, in _get_handles
      p2cread, p2cwrite = self.pipe_cloexec()
    File "/usr/lib/python2.7/subprocess.py", line 1091, in pipe_cloexec
      r, w = os.pipe()
  OSError: [Errno 24] Too many open files
  ======================================================================
  ...

  Ran 9290 (+1703) tests in 2704.305s (+299.051s)
  FAILED (id=1, failures=194 (+185), skips=324)

  Here is the version info:

  cm@vpn-a[2853] $ git log -n 1
  commit 3014e1e021b3fe59c75daae1734472c3a11582ee
  Merge: e22626e 5652e20
  Author: Jenkins <jenkins@xxxxxxxxxxxxxxxxxxxx>
  Date:   Sat Dec 7 23:05:30 2013 +0000

      Merge "Preserve floating ips when initializing l3 gateway
  interface"

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1258952/+subscriptions