yahoo-eng-team team mailing list archive

[OpenStack Infra <1595878@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/333827
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=a5d19b41a57141ef51208b8a821bb7d2e0b6c159
Submitter: Jenkins
Branch:    master

commit a5d19b41a57141ef51208b8a821bb7d2e0b6c159
Author: Oleg Bondarev <obondarev@xxxxxxxxxxxx>
Date:   Fri Jun 24 11:42:52 2016 +0300

    Mock threading.Thread to prevent daemon creation by unit tests
    
    tests.unit.agent.ovsdb.native.test_connection.TestOVSNativeConnection
    calls Connection.start() which starts a daemon with a while True loop
    full of mocks. mock._CallList of those mocks start to grow very
    quick and finally eat all available memory.
    
    Closes-Bug: #1595878
    Change-Id: Ie053a2248925ce5bb960207c16c23b261d1d458c


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1595878

Title:
  Memory leak in unit tests

Status in neutron:
  Fix Released

Bug description:
  tests.unit.agent.ovsdb.native.test_connection.TestOVSNativeConnection
  calls Connection.start() which starts a daemon with a while True loop
  full of mocks. mock._CallList of those mocks start to grow very quick
  and finally eat all available memory.

  mem_top output during unit tests run:

  refs:
  18118	<class 'mock.mock._CallList'> [call(1),
   call().get_nowait(),
   call().get_nowait().do_commit(),
   call().get_nowait().results.put(<MagicMock name='TransactionQueue().get_nowait().do_commit()' id='139952637975312'>),
   call().task_do
  18117	<class 'mock.mock._CallList'> [call.get_nowait(),
   call.get_nowait().do_commit(),
   call.get_nowait().results.put(<MagicMock name='TransactionQueue().get_nowait().do_commit()' id='139952637975312'>),
   call.task_done(),
   call.get_no
  17990	<class 'mock.mock._CallList'> [call(1),
   call().get_nowait(),
   call().get_nowait().do_commit(),
   call().get_nowait().results.put(<MagicMock name='TransactionQueue().get_nowait().do_commit()' id='139952663067600'>),
   call().task_do
  17989	<class 'mock.mock._CallList'> [call.get_nowait(),
   call.get_nowait().do_commit(),
   call.get_nowait().results.put(<MagicMock name='TransactionQueue().get_nowait().do_commit()' id='139952663067600'>),
   call.task_done(),
   call.get_no
  13592	<class 'mock.mock._CallList'> [call(),
   call().fd_wait(<MagicMock name='TransactionQueue().alert_fileno' id='139952638377296'>, 1),
   call().timer_wait(<Mock id='139952614993680'>),
   call().block(),
   call().fd_wait(<MagicMock name=
  13591	<class 'mock.mock._CallList'> [call.fd_wait(<MagicMock name='TransactionQueue().alert_fileno' id='139952638377296'>, 1),
   call.timer_wait(<Mock id='139952614993680'>),
   call.block(),
   call.fd_wait(<MagicMock name='TransactionQueue
  13591	<class 'mock.mock._CallList'> [call.fd_wait(<MagicMock name='TransactionQueue().alert_fileno' id='139952638377296'>, 1),
   call.timer_wait(<Mock id='139952614993680'>),
   call.block(),
   call.fd_wait(<MagicMock name='TransactionQueue
  13588	<class 'mock.mock._CallList'> [call(),
   call().do_commit(),
   call().results.put(<MagicMock name='TransactionQueue().get_nowait().do_commit()' id='139952637975312'>),
   call(),
   call().do_commit(),
   call().results.put(<MagicMock nam
  13496	<class 'mock.mock._CallList'> [call(),
   call().fd_wait(<MagicMock name='TransactionQueue().alert_fileno' id='139952689448208'>, 1),
   call().timer_wait(<Mock id='139952687478800'>),
   call().block(),
   call().fd_wait(<MagicMock name=
  13495	<class 'mock.mock._CallList'> [call.fd_wait(<MagicMock name='TransactionQueue().alert_fileno' id='139952689448208'>, 1),
   call.timer_wait(<Mock id='139952687478800'>),
   call.block(),
   call.fd_wait(<MagicMock name='TransactionQueue
  13495	<class 'mock.mock._CallList'> [call.fd_wait(<MagicMock name='TransactionQueue().alert_fileno' id='139952689448208'>, 1),
   call.timer_wait(<Mock id='139952687478800'>),
   call.block(),
   call.fd_wait(<MagicMock name='TransactionQueue
  13492	<class 'mock.mock._CallList'> [call(),
   call().do_commit(),
   call().results.put(<MagicMock name='TransactionQueue().get_nowait().do_commit()' id='139952663067600'>),
   call(),
   call().do_commit(),
   call().results.put(<MagicMock nam
  11550	<type 'dict'> {'keystoneclient.service_catalog': <module 'keystoneclient.service_catalog' from '/opt/stack/neutron/.tox/py27/local/lib/python2.7/site-packages/keystoneclient/service_catalog.pyc'>, 'oslo_messaging.r
  9061	<class 'mock.mock._CallList'> [call(<Mock id='139952630613712'>, <Mock name='get_schema_helper()' id='139952630616016'>),
   call().wait(<MagicMock name='Poller()' id='139952641693968'>),
   call().run(),
   call().wait(<MagicMock name=
  9060	<class 'mock.mock._CallList'> [call.wait(<MagicMock name='Poller()' id='139952641693968'>),
   call.run(),
   call.wait(<MagicMock name='Poller()' id='139952641693968'>),
   call.run(),
   call.wait(<MagicMock name='Poller()' id='13995264
  9060	<class 'mock.mock._CallList'> [call.wait(<MagicMock name='Poller()' id='139952641693968'>),
   call.run(),
   call.wait(<MagicMock name='Poller()' id='139952641693968'>),
   call.run(),
   call.wait(<MagicMock name='Poller()' id='13995264
  9059	<class 'mock.mock._CallList'> [call.do_commit(),
   call.results.put(<MagicMock name='TransactionQueue().get_nowait().do_commit()' id='139952637975312'>),
   call.do_commit(),
   call.results.put(<MagicMock name='TransactionQueue().get_
  9059	<class 'mock.mock._CallList'> [call.do_commit(),
   call.results.put(<MagicMock name='TransactionQueue().get_nowait().do_commit()' id='139952637975312'>),
   call.do_commit(),
   call.results.put(<MagicMock name='TransactionQueue().get_
  9059	<class 'mock.mock._CallList'> [call.get_nowait(),
   call.task_done(),
   call.get_nowait(),
   call.task_done(),
   call.get_nowait(),
   call.task_done(),
   call.get_nowait(),
   call.task_done(),
   call.get_nowait(),
   call.task_done(),
   call
  8997	<class 'mock.mock._CallList'> [call(<Mock id='139952686333712'>, <Mock name='get_schema_helper()' id='139952686333840'>),
   call().wait(<MagicMock name='Poller()' id='139952689107984'>),
   call().run(),
   call().wait(<MagicMock name=

  types:
  460418	 <class 'mock.mock._Call'>
  79091	 <type 'tuple'>
  47269	 <type 'function'>
  45542	 <type 'dict'>
  30758	 <type 'list'>
  14696	 <type 'weakref'>
  8601	 <type 'set'>
  6579	 <type 'cell'>
  5639	 <type 'type'>
  4940	 <class 'mock.mock.MagicProxy'>
  3858	 <type 'getset_descriptor'>
  3291	 <class 'unittest.suite.TestSuite'>
  3275	 <type 'itertools.count'>
  3267	 <class 'unittest2.case._TypeEqualityDict'>
  2439	 <type 'module'>
  2304	 <class '_weakrefset.WeakSet'>
  2222	 <type 'instancemethod'>
  2219	 <type 'builtin_function_or_method'>
  1869	 <type 'property'>
  1424	 <type 'wrapper_descriptor'>

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1595878/+subscriptions