yahoo-eng-team team mailing list archive

[OpenStack Infra <1602794@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/341794
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ed34f18916c62d77358d4c5910ff17694828576a
Submitter: Jenkins
Branch:    master

commit ed34f18916c62d77358d4c5910ff17694828576a
Author: Brian Haley <brian.haley@xxxxxxx>
Date:   Wed Jul 13 16:06:06 2016 -0400

    DVR: Fix ItemAllocator class to handle exceptions
    
    The ItemAllocator class blindly reads from a file
    when initialized, assuming each line is always in
    "key,value" syntax.  It's possible that the file
    may become corrupted, leading to ValueErrors being
    thrown which it doesn't handle.
    
    This was found running the unit tests, more specifically
    those doing DVR FIP namespace operations, since they were
    always reading/writing the same files in /tmp.  Typically
    the failure was random and would go away with a recheck,
    but could be the cause of check job instability.  All
    occurrences were fixed to use a test-created temporary
    directory to avoid collision with each other.
    
    Change-Id: I39d116aa8261b50bfcb3269416c1a307cafa134e
    Closes-bug: #1602794


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1602794

Title:
  ItemAllocator class can throw a ValueError when file is corrupted

Status in neutron:
  Fix Released

Bug description:
  While running tests for an unrelated change, I saw this failure:

  neutron.tests.unit.agent.l3.test_dvr_fip_ns.TestDvrFipNs.test_subscribe
  -----------------------------------------------------------------------

  Captured traceback:
  ~~~~~~~~~~~~~~~~~~~
      Traceback (most recent call last):
        File "neutron/tests/unit/agent/l3/test_dvr_fip_ns.py", line 40, in setUp
          use_ipv6=True)
        File "neutron/agent/l3/dvr_fip_ns.py", line 57, in __init__
          FIP_PR_END)
        File "neutron/agent/l3/fip_rule_priority_allocator.py", line 53, in __init__
          pool)
        File "neutron/agent/l3/item_allocator.py", line 46, in __init__
          key, saved_value = line.strip().split(delimiter)
      ValueError: need more than 1 value to unpack

  This is due to the fact that all the tests in this file using the
  ItemAllocator class are using/re-using the same files in /tmp, and
  when the right ones are run simultaneously, the file can be truncated,
  leading to a corruption.  To recreate it you just have to do this:

  $ echo "111.2.3.466652" > /tmp/fip-priorities
  $ tox -e py27 neutron.tests.unit.agent.l3.test_dvr_fip_ns.TestDvrFipNs.test_subscribe

  So the tests need to change to use unique files, and clean them up on
  exit.

  But it also shows that the ItemAllocator class cannot deal with a
  corrupt file.  In this case we should catch the error and skip the
  invalid entry, since it could cause the l3-agent to not start.

  The same is probably true of other files in /tmp the tests are using
  as well.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1602794/+subscriptions