← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1277168] Re: having oslo.sphinx in namespace package causes issues with devstack

 

Uggh, digging up the details behind this was pretty terrible ...

There is clearly some underlying issue we're papering over here ... but
nowhere do I see any analysis of that underlying issue recorded for
posterity

For example, in https://review.openstack.org/71589

  "Having the sphinx theme in the same namespace package
    as the production code has been causing issues with
    devstack installations."

The thread linked above ends with http://lists.openstack.org/pipermail
/openstack-dev/2013-December/023002.html

 "Basically you can very easily get yourself in a situation where *just
   enough* of the distro package is left behind that pip thinks its there,
   so won't install it, but the python loader doesn't, so won't work.

   Then much sadness."

Or in https://review.openstack.org/#/c/65414/1//COMMIT_MSG

  "After merging https://review.openstack.org/64226 devstack gate
started failing."

Oh, now I'm getting somewhere http://eavesdrop.openstack.org/irclogs
/%23openstack-infra/%23openstack-infra.2014-01-08.log

   <dhellmann> fungi: the tl;dr is that when oslo.config is installed globally with "pip install -e" and oslo.sphinx is installed in a virtualenv with "pip install" python in the virtualenv can't find oslo.config
  ...
  <dhellmann> sdague: I think bnemec worked out that it's caused by a combination of pip install -e and something else in the same namespace package not being installed in that mode
  <dhellmann> sdague: alternatives to fix it seem to be install oslo.sphinx with pip install -e or change its package name
  <dhellmann> sdague: or don't install it at all, of course
  <bnemec> dhellmann: sdague: Right, the problem is that we pip install -e oslo.config in the system site packages, then pip install oslo.sphinx in the venv.
  <bnemec> That combination results in oslo.config being unavailable in the venv.
  <dhellmann> fungi: common namespace, split, *and* installed with .egg-link in some cases (I think)
  <dstufft> namespaces don't play well with -e
  <dhellmann> so for normal production oslo libs, we'd just install them from devstack with pip install -e and be done
  <dhellmann> but because oslo.sphinx is not a production lib, that's not necessarily the best answer
  <dhellmann> we should probably just reserve the oslo namespace for production libraries, and rename the theme package
  <dhellmann> well, it could also be said that if devstack didn't use that -e option this wouldn't be an issue, so it's a combination of all of it

And, at last, I find Ben's email ! http://lists.openstack.org/pipermail
/openstack-dev/2014-January/023759.html

  The problem seems to be when we pip install -e oslo.config on the 
  system, then pip install oslo.sphinx in a venv.  oslo.config is 
  unavailable in the venv, apparently because the namespace package for 
  o.s causes the egg-link for o.c to be ignored.  Pretty much every other 
  combination I've tried (regular pip install of both, or pip install -e 
  of both, regardless of where they are) works fine, but there seem to be 
  other issues with all of the other options we've explored so far.

  We can't remove the pip install -e of oslo.config because it has to be 
  used for gating, and we can't pip install -e oslo.sphinx because it's 
  not a runtime dep so it doesn't belong in the gate.  Changing the 
  toplevel package for oslo.sphinx was also mentioned, but has obvious 
  drawbacks too.

And then this pip bug: https://github.com/pypa/pip/issues/3

** Also affects: oslo.messaging
   Importance: Undecided
       Status: New

** Changed in: oslo.messaging
   Importance: Undecided => High

** Changed in: oslo.messaging
       Status: New => Fix Committed

** Changed in: oslo.messaging
     Assignee: (unassigned) => Dirk Mueller (dmllr)

** Description changed:

  http://lists.openstack.org/pipermail/openstack-
  dev/2013-December/022586.html
  
  We've decided to rename oslo.sphinx to oslosphinx. This will require
  small changes in the doc builds for a lot of the other projects.
+ 
+ The problem seems to be when we pip install -e oslo.config on the
+ system, then pip install oslo.sphinx in a venv. oslo.config is
+ unavailable in the venv, apparently because the namespace package for
+ o.s causes the egg-link for o.c to be ignored.

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1277168

Title:
  having oslo.sphinx in namespace package causes issues with devstack

Status in OpenStack Compute (Nova):
  Confirmed
Status in Oslo - a Library of Common OpenStack Code:
  In Progress
Status in Messaging API for OpenStack:
  Fix Committed
Status in Tempest:
  New

Bug description:
  http://lists.openstack.org/pipermail/openstack-
  dev/2013-December/022586.html

  We've decided to rename oslo.sphinx to oslosphinx. This will require
  small changes in the doc builds for a lot of the other projects.

  The problem seems to be when we pip install -e oslo.config on the
  system, then pip install oslo.sphinx in a venv. oslo.config is
  unavailable in the venv, apparently because the namespace package for
  o.s causes the egg-link for o.c to be ignored.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1277168/+subscriptions