yahoo-eng-team team mailing list archive

[OpenStack Infra <1523780@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/254586
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=f54cba053556a43d51ccd895cdf8232c51210299
Submitter: Jenkins
Branch:    master

commit f54cba053556a43d51ccd895cdf8232c51210299
Author: LIU Yulong <liuyulong@xxxxxxxx>
Date:   Tue Dec 8 14:13:44 2015 +0800

    Catch known exceptions during deleting last HA router
    
    In some scenarios, for instance rally test create_and_delete_routers,
    it will get some exceptions, such as the network in use exception,
    during the router deleting api call, but actually the router has
    been deleted. There has race between HA router create and delete,
    if set more api and rpc worker race raises exception more frequently.
    Because the inconsistent error message was not useful for user,
    this patch will catch those know exceptions ObjectDeletedError,
    NetworkInUse when user delete last HA router.
    
    At the same time, when user create the first HA router, but because
    of the failure of HA network creation, the router will be deleted,
    then the deleting HA network will raise AttributeError, this patch
    also move HA network deleting procedure under ha_network exist check
    block.
    
    Change-Id: I8cda00c1e7caffc4dfb20a817a11c60736855bb5
    Closes-Bug: #1523780
    Related-Bug: #1367157


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1523780

Title:
  Race between HA router create and HA router delete

Status in neutron:
  Fix Released

Bug description:
  Set more than one API worker and RPC worker,  and then run rally scenario test  create_and_delete_routers:
  you may get such errors:

  1.DBReferenceError: (IntegrityError) (1452, 'Cannot add or update a
  child row: a foreign key constraint fails
  (`neutron`.`ha_router_agent_port_bindings`, CONSTRAINT
  `ha_router_agent_port_bindings_ibfk_2` FOREIGN KEY (`router_id`)
  REFERENCES `routers` (`id`) ON DELETE CASCADE)') 'INSERT INTO
  ha_router_agent_port_bindings (port_id, router_id, l3_agent_id, state)
  VALUES (%s, %s, %s, %s)' ('xxxxxxxxxxx', 'xxxxxxxxxxxxxxxxxxx', None,
  'standby')

  (InvalidRequestError: This Session's transaction has been rolled back
  by a nested rollback() call.  To begin a new transaction, issue
  Session.rollback() first.)

  2. AttributeError: 'NoneType' object has no attribute 'config' (l3
  agent process router in router_delete function)

  3. DBError: UPDATE statement on table 'ports' expected to update 1
  row(s); 0 were matched.

  4. res = {"id": port["id"],
     TypeError: 'NoneType' object is unsubscriptable

  5. delete HA network during deleting the last router, get error
  message: "Unable to complete operation on network xxxxxxxxxxxx. There
  are one or more ports still in use on the network."

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1523780/+subscriptions