yahoo-eng-team team mailing list archive

[Dave Walker <email@xxxxxxxxxx>]

** Also affects: neutron/kilo
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1552960

Title:
  Tempest and Neutron duplicate tests

Status in neutron:
  In Progress
Status in neutron kilo series:
  New

Bug description:
  Problem statement:

  1) Tests overlap between Tempest and Neutron repos - 264 tests last I checked. The effect is:
  1.1) Confusion amongst QA & dev contributors and reviewers. I'm writing a test, where should it go? Someone just proposed a Tempest patch for a new Neutron API, what should I do with this patch?
  1.2) Wasteful from a computing resources point of view - The same tests are being run multiple times for every Neutron patchset.
  2) New APIs (Subnet pools, address scopes, QoS, RBAC, port security, service flavors and availability zones), are not covered by Tempest tests. Consumers have to adapt and run both the Tempest tests and the Neutron tests in two separate runs. This causes a surprising amount of grief.

  Proposed solution:

  For problem 1, we eliminate the overlap. We do this by defining a set
  of tests that will live in Tempest, and another set of tests that will
  live in Neutron. More information may be found here:
  https://review.openstack.org/#/c/280427/. After deciding on the line
  in the sand, we will delete any tests in Neutron that should continue
  to live in Tempest. Some Neutron tests were modified after they were
  copied from Tempest, these modifications will have to be examined and
  then proposed to Tempest. Afterwards these tests may be removed from
  Neutron, eliminating the overlap from the Neutron side. Once this is
  done, overlapping tests may be deleted from Tempest.

  For problem 2, we will develop a Neutron Tempest plugin. This will be
  tracked in a separate bug. Note that there's already a patch for this
  up for review: https://review.openstack.org/#/c/274023/

  * The work is also being tracked here:
  https://etherpad.openstack.org/p/neutron-tempest-defork

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1552960/+subscriptions