yahoo-eng-team team mailing list archive

[Alan Pevec <1184041@xxxxxxxxxxxxxxxxxx>]

** Changed in: nova/grizzly
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1184041

Title:
  [OSSA 2013-020] Denial of Service in Nova network source security
  groups (CVE-2013-4185)

Status in OpenStack Compute (Nova):
  Fix Committed
Status in OpenStack Compute (nova) folsom series:
  Fix Committed
Status in OpenStack Compute (nova) grizzly series:
  Fix Released
Status in OpenStack Security Advisories:
  Fix Released

Bug description:
  The code for retrieving the list of ips to make source security groups
  is very inefficient. It is possible to create a set of rules using
  source security groups that will lead to an explosion of
  get_network_info calls.

  Example:

  # create a security group
  nova secgroup-create foo foo

  # add source group rules referencing the same group
  for i in {1..10}; do
    nova secgroup-add-group-rule foo foo tcp $i $i
  done

  # launch 10 instances
  nova boot --flavor 1 --image precise --num-instances 10 --security-groups foo test

  When each instance boots it sends out a notification to each of the
  other instances to update their iptables rules. Here is what happens
  when the 10th instance boots:

  for num instances in group that is referenced in a source rule: 10
    for num rules referencing a source group: 10
      for num instances in the source group: 10
        call get_nw_info()

  That means 1000 individual requests to get_nw_info for a single instance launch. In this case you have You can see how these numbers could get out of hand very quickly, for example launching 20 instances:
  instance number(n) : num calls (n^2) * 10 
  1 :  10
  2 :  20
  3 :  90
  ...
  18 : 3240
  19 : 3610
  20 : 4000
  ...
  $ python -c "print sum(n * n * 10 for n in xrange(1, 21))"
  28700

  28,700 calls to get_network info by the time the launch completes. In
  reality this number of calls creates a DOS where nova-network can't
  respond in time and instance launches start to timeout.

  There are multiple fixes needed:

  a) constructing a list of all needed instances instead of making a separate request for each rule.
  b) a single call to retrieve nw_info (or at least fixed ip info) for multiple instances at once.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1184041/+subscriptions