yahoo-eng-team team mailing list archive

[OpenStack Infra <1822575@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/622972
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=e5269f2fbb773953266fccbfd9df3f75e351eeb9
Submitter: Zuul
Branch:    master

commit e5269f2fbb773953266fccbfd9df3f75e351eeb9
Author: Chris Dent <cdent@xxxxxxxxxxxxx>
Date:   Wed Dec 5 14:01:04 2018 +0000

    Correct lower-constraints.txt and the related tox job
    
    In the review of a similar change in placement [1], it was realized that
    the nova lower-constraints tox job probably had the same problems.
    Testing revealed this to be the case. This change fixes the job and
    updates the related requirements problems accordingly.
    
    The are two main factors at play here:
    
    * The default install_command in tox.ini uses the upper_contraints.txt
      file. When there is more than one constraints.txt they are merged and
      the higher constraints win. Using upper and lower at the same time
      violates the point of lower (which is to indicate the bare minimum
      we are capable of using).
    
    * When usedevelop is true in tox, the command that is run to install the
      current projects code is something like 'python setup.py develop',
      which installs a project's requirements _after_ the install_command has
      run, clobbering the constrained installs. When using pbr,
      'python setup.py install' (used when usedevelop is False) does not do
      this.
    
    Fixing those then makes it possible to use the test to fix the
    lower-constraints.txt and *requirements.txt files, changes include:
    
    * Defining 'usedevelop = False' in the 'lower-constraints' target and
      removing the otherwise superfluous 'skipsdist' global setting to
      ensure requirements aren't clobbered.
    
    * Removing packages which show up in lower-constraints.txt but not in
      the created virtualenv. Note that the job only runs unit tests, so
      this may be incomplete. In the placement version of this both unit and
      functional are run. We may want to consider that here.
    
    * Updating cryptography. This version is needed with more recent
      pyopenssl.
    
    * Updated keystonemiddleware. This is needed for some tests which
      confirm passing configuration to the middleware.
    
    * Update psycopg2 to a version that can talk with postgresql 10.
    
    * Add PyJWT, used by zVMCloudConnector
    
    * Update zVMCloudConnector to a version that works with Python 3.5 and
      beyond.
    
    * Update olso.messaging to versions that work with the tests, under
      Python 3.
    
    * Adding missing transitive packages.
    
    * Adjusting alpha-ordering to map to how pip freeze does it.
    
    * setuptools is removed from requirements.txt because the created
      virtualenv doesn't contain it
    
    NOTE: The lower-constraints.txt file makes no commitment to expressing
    minimum requirements for anything other than the current basepython.
    So the fact that a different set of lower-constraints would be present
    if we were using python2 is not relevant. See discussion at [1].
    However, since requirements.txt _is_ used for python2, the
    requirements-check gate job requires that enum34 be present in
    lower-constraints.txt because it is in requirements.txt.
    
    NOTE: A test is removed because it cannot work in the
    lower-constraints context: 'test_policy_generator_from_command_line'
    forks a call to 'oslopolicy-policy-generator --namespace nova' which
    fails because stevedore fails to pick up nova-based entry points when
    in a different process. This is because of the change to usedevelop.
    After discussion with the original author of the test removal was
    considered an acceptable choice.
    
    [1] http://eavesdrop.openstack.org/irclogs/%23openstack-dev/%23openstack-dev.2019-03-05.log.html#t2019-03-05T13:28:23
    
    Closes-Bug: #1822575
    
    Change-Id: Ic6466b0440a4fe012731a63715cf5d793b6ae4dd


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1822575

Title:
  lower-constraints are not used in gate job

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  the lower constraints tox env attempts to run nova's unit tests with
  the minium supported software versions declared in nova lower-constraints.txt

  due to the way the install command is specified in the default tox env

  install_command = pip install
  -c{env:UPPER_CONSTRAINTS_FILE:https://git.openstack.org/cgit/openstack/requirements/plain
  /upper-constraints.txt} {opts} {packages}

  the upper-constraints.txt was also passed to pip.

  pips constraint solver takes the first deffintion of a constraint and
  discards all redfinitoins.

  because upper-constraints.txt was included before lower-constraints.txt the lower constraints were
  ignored.

  there are two patchs proposed to fix this 
  https://review.openstack.org/#/c/622972 and https://review.openstack.org/#/c/645392

  we should merge one of them.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1822575/+subscriptions