yahoo-eng-team team mailing list archive

[Assaf Muller <amuller@xxxxxxxxxx>]

Public bug reported:

Problem statement:

1) Tests overlap between Tempest and Neutron repos - 264 tests last I checked. The effect is:
1.1) Confusion amongst QA & dev contributors and reviewers. I'm writing a test, where should it go? Someone just proposed a Tempest patch for a new Neutron API, what should I do with this patch?
1.2) Wasteful from a computing resources point of view - The same tests are being run multiple times for every Neutron patchset.
2) New APIs (Subnet pools, address scopes, QoS, RBAC, port security, service flavors and availability zones), are not covered by Tempest tests. Consumers have to adapt and run both the Tempest tests and the Neutron tests in two separate runs. This causes a surprising amount of grief.

Proposed solution:

For problem 1, we eliminate the overlap. We do this by defining a set of
tests that will live in Tempest, and another set of tests that will live
in Neutron. More information may be found here:
https://review.openstack.org/#/c/280427/. After deciding on the line in
the sand, we will delete any tests in Neutron that should continue to
live in Tempest. Some Neutron tests were modified after they were copied
from Tempest, these modifications will have to be examined and then
proposed to Tempest. Afterwards these tests may be removed from Neutron,
eliminating the overlap from the Neutron side. Once this is done,
overlapping tests may be deleted from Tempest.

For problem 2, we will develop a Neutron Tempest plugin. This will be
tracked in a separate bug. Note that there's already a patch for this up
for review: https://review.openstack.org/#/c/274023/

* The work is also being tracked here: https://etherpad.openstack.org/p
/neutron-tempest-defork

** Affects: neutron
     Importance: High
     Assignee: Assaf Muller (amuller)
         Status: In Progress

** Description changed:

  Problem statement:
  
  1) Tests overlap between Tempest and Neutron repos - 264 tests last I checked. The effect is:
  1.1) Confusion amongst QA & dev contributors and reviewers. I'm writing a test, where should it go? Someone just proposed a Tempest patch for a new Neutron API, what should I do with this patch?
  1.2) Wasteful from a computing resources point of view - The same tests are being run multiple times for every Neutron patchset.
  2) New APIs (Subnet pools, address scopes, QoS, RBAC, port security, service flavors and availability zones), are not covered by Tempest tests. Consumers have to adapt and run both the Tempest tests and the Neutron tests in two separate runs. This causes a surprising amount of grief.
  
  Proposed solution:
  
  For problem 1, we eliminate the overlap. We do this by defining a set of
  tests that will live in Tempest, and another set of tests that will live
  in Neutron. More information may be found here:
  https://review.openstack.org/#/c/280427/. After deciding on the line in
  the sand, we will delete any tests in Neutron that should continue to
  live in Tempest. Some Neutron tests were modified after they were copied
  from Tempest, these modifications will have to be examined and then
  proposed to Tempest. Afterwards these tests may be removed from Neutron,
  eliminating the overlap from the Neutron side. Once this is done,
  overlapping tests may be deleted from Tempest.
  
  For problem 2, we will develop a Neutron Tempest plugin. This will be
  tracked in a separate bug. Note that there's already a patch for this up
  for review: https://review.openstack.org/#/c/274023/
+ 
+ * The work is also being tracked here: https://etherpad.openstack.org/p
+ /neutron-tempest-defork

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1552960

Title:
  Tempest and Neutron duplicate tests

Status in neutron:
  In Progress

Bug description:
  Problem statement:

  1) Tests overlap between Tempest and Neutron repos - 264 tests last I checked. The effect is:
  1.1) Confusion amongst QA & dev contributors and reviewers. I'm writing a test, where should it go? Someone just proposed a Tempest patch for a new Neutron API, what should I do with this patch?
  1.2) Wasteful from a computing resources point of view - The same tests are being run multiple times for every Neutron patchset.
  2) New APIs (Subnet pools, address scopes, QoS, RBAC, port security, service flavors and availability zones), are not covered by Tempest tests. Consumers have to adapt and run both the Tempest tests and the Neutron tests in two separate runs. This causes a surprising amount of grief.

  Proposed solution:

  For problem 1, we eliminate the overlap. We do this by defining a set
  of tests that will live in Tempest, and another set of tests that will
  live in Neutron. More information may be found here:
  https://review.openstack.org/#/c/280427/. After deciding on the line
  in the sand, we will delete any tests in Neutron that should continue
  to live in Tempest. Some Neutron tests were modified after they were
  copied from Tempest, these modifications will have to be examined and
  then proposed to Tempest. Afterwards these tests may be removed from
  Neutron, eliminating the overlap from the Neutron side. Once this is
  done, overlapping tests may be deleted from Tempest.

  For problem 2, we will develop a Neutron Tempest plugin. This will be
  tracked in a separate bug. Note that there's already a patch for this
  up for review: https://review.openstack.org/#/c/274023/

  * The work is also being tracked here:
  https://etherpad.openstack.org/p/neutron-tempest-defork

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1552960/+subscriptions