← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #91634

[Bug 1615502] Re: LBAAS - housekeeping serive does not cleanup stale amphora VMs

  Thread Previous • Date Previous • • Thread Next 
Abandoned after re-enabling the Octavia launchpad.

** Changed in: octavia
       Status: In Progress => Invalid

** Changed in: octavia
     Assignee: Ravikumar (ravikumar-vallabhu) => (unassigned)

** Tags added: auto-abandon

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1615502

Title:
  LBAAS - housekeeping serive does not cleanup stale amphora VMs

Status in neutron:
  Invalid
Status in octavia:
  Invalid

Bug description:
  1.	Initially there were no spare VMs since the “spare_amphora_pool_size = 0”
   .

   [house_keeping]
  # Pool size for the spare pool
  spare_amphora_pool_size = 0

  
  stack@hlm:~/scratch/ansible/next/hos/ansible$ nova list --all
  WARNING: Option "--all_tenants" is deprecated; use "--all-tenants"; this option will be removed in novaclient 3.3.0.
  +--------------------------------------+----------------------------------------------+----------------------------------+--------+------------+-------------+-------------------------------+
  | ID                                   | Name                                         | Tenant ID                        | Status | Task State | Power State | Networks                      |
  +--------------------------------------+----------------------------------------------+----------------------------------+--------+------------+-------------+-------------------------------+
  | 91eef324-0c51-4b91-8a54-e16abdb64e55 | vm1                                          | d15f2abc106740499a453260ae6522f3 | ACTIVE | -          | Running     | n1=4.5.6.5                    |
  | 7d85921c-e7d9-4b70-9023-0478c66b7e7c | vm2                                          | d15f2abc106740499a453260ae6522f3 | ACTIVE | -          | Running     | n1=4.5.6.6                    |
  +--------------------------------------+----------------------------------------------+----------------------------------+--------+------------+-------------+-------------------------------+

  2.      Change the spare pool size to 1 and restart Octavia-
  housekeeping service. Spare Amphora VM gets created as below.

  
  stack@hlm:~/scratch/ansible/next/hos/ansible$  nova list --all
  WARNING: Option "--all_tenants" is deprecated; use "--all-tenants"; this option will be removed in novaclient 3.3.0.
  +--------------------------------------+----------------------------------------------+----------------------------------+--------+------------+-------------+-------------------------------+
  | ID                                   | Name                                         | Tenant ID                        | Status | Task State | Power State | Networks                      |
  +--------------------------------------+----------------------------------------------+----------------------------------+--------+------------+-------------+-------------------------------+
  | 6a1101cd-d9d3-4c8e-aa1d-0790f7f4ac8b | amphora-18f4d90f-fe6e-4085-851e-7571cba0c65a | a5e6e87d402847e7b4210e035a0fceec | ACTIVE | -          | Running     | OCTAVIA-MGMT-NET=100.74.25.13 |
  | 91eef324-0c51-4b91-8a54-e16abdb64e55 | vm1                                          | d15f2abc106740499a453260ae6522f3 | ACTIVE | -          | Running     | n1=4.5.6.5                    |
  | 7d85921c-e7d9-4b70-9023-0478c66b7e7c | vm2                                          | d15f2abc106740499a453260ae6522f3 | ACTIVE | -          | Running     | n1=4.5.6.6                    |
  +--------------------------------------+----------------------------------------------+----------------------------------+--------+------------+-------------+-------------------------------+

  3.      Now change the spare pool size to 0 and restart Octavia-
  housekeeping service. Spare Amphora VM does not get deleted.

  stack@hlm:~/scratch/ansible/next/hos/ansible$  nova list --all
  WARNING: Option "--all_tenants" is deprecated; use "--all-tenants"; this option will be removed in novaclient 3.3.0.
  +--------------------------------------+----------------------------------------------+----------------------------------+--------+------------+-------------+-------------------------------+
  | ID                                   | Name                                         | Tenant ID                        | Status | Task State | Power State | Networks                      |
  +--------------------------------------+----------------------------------------------+----------------------------------+--------+------------+-------------+-------------------------------+
  | 6a1101cd-d9d3-4c8e-aa1d-0790f7f4ac8b | amphora-18f4d90f-fe6e-4085-851e-7571cba0c65a | a5e6e87d402847e7b4210e035a0fceec | ACTIVE | -          | Running     | OCTAVIA-MGMT-NET=100.74.25.13 |
  | 91eef324-0c51-4b91-8a54-e16abdb64e55 | vm1                                          | d15f2abc106740499a453260ae6522f3 | ACTIVE | -          | Running     | n1=4.5.6.5                    |
  | 7d85921c-e7d9-4b70-9023-0478c66b7e7c | vm2                                          | d15f2abc106740499a453260ae6522f3 | ACTIVE | -          | Running     | n1=4.5.6.6                    |
  +--------------------------------------+----------------------------------------------+----------------------------------+--------+------------+-------------+-------------------------------+
  stack@hlm:~/scratch/ansible/next/hos/ansible$

  4.      I see the requirement is not satisfied as per the config so
  hosue keeping should take care of cleaning up the spare amphora VM.But
  in the logs I see the statement “ requirement is satisfied “.

  In the logs

  2016-08-22 06:37:56.901 63471 DEBUG octavia.cmd.house_keeping [-] Initiating the cleanup of old amphora... db_cleanup /opt/stack/venv/octavia-20160815T061034Z/lib/python2.7/site-packages/octavia/cmd/house_keeping.py:64
  2016-08-22 06:37:57.194 63471 DEBUG octavia.cmd.house_keeping [-] Initiating spare amphora check... spare_amphora_check /opt/stack/venv/octavia-20160815T061034Z/lib/python2.7/site-packages/octavia/cmd/house_keeping.py:49
  2016-08-22 06:37:57.203 63471 DEBUG octavia.controller.housekeeping.house_keeping [-] Required Spare Amphora count : 0 spare_check /opt/stack/venv/octavia-20160815T061034Z/lib/python2.7/site-packages/octavia/controller/housekeeping/house_keeping.py:45
  2016-08-22 06:37:57.204 63471 DEBUG octavia.controller.housekeeping.house_keeping [-] Current Spare Amphora count : 1 spare_check /opt/stack/venv/octavia-20160815T061034Z/lib/python2.7/site-packages/octavia/controller/housekeeping/house_keeping.py:46
  2016-08-22 06:37:57.204 63471 DEBUG octavia.controller.housekeeping.house_keeping [-] Current spare amphora count satisfies the requirement spare_check /opt/stack/venv/octavia-20160815T061034Z/lib/python2.7/site-packages/octavia/controller/housekeeping/house_keeping.py:60 2016-08-22 06:37:56.901 63471 DEBUG octavia.cmd.house_keeping [-] Initiating the cleanup of old amphora... db_cleanup /opt/stack/venv/octavia-20160815T061034Z/lib/python2.7/site-packages/octavia/cmd/house_keeping.py:64
  2016-08-22 06:37:57.194 63471 DEBUG octavia.cmd.house_keeping [-] Initiating spare amphora check... spare_amphora_check /opt/stack/venv/octavia-20160815T061034Z/lib/python2.7/site-packages/octavia/cmd/house_keeping.py:49
  2016-08-22 06:37:57.203 63471 DEBUG octavia.controller.housekeeping.house_keeping [-] Required Spare Amphora count : 0 spare_check /opt/stack/venv/octavia-20160815T061034Z/lib/python2.7/site-packages/octavia/controller/housekeeping/house_keeping.py:45
  2016-08-22 06:37:57.204 63471 DEBUG octavia.controller.housekeeping.house_keeping [-] Current Spare Amphora count : 1 spare_check /opt/stack/venv/octavia-20160815T061034Z/lib/python2.7/site-packages/octavia/controller/housekeeping/house_keeping.py:46
  2016-08-22 06:37:57.204 63471 DEBUG octavia.controller.housekeeping.house_keeping [-] Current spare amphora count satisfies the requirement spare_check /opt/stack/venv/octavia-20160815T061034Z/lib/python2.7/site-packages/octavia/controller/housekeeping/house_keeping.py:60

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1615502/+subscriptions

References

  Thread Previous • Date Previous • • Thread Next