← Back to team overview

fuel-dev team mailing list archive

Re: [iso build] Inconsistency in eggs requirement files

 

Hi Ivan,

>> I vote for less strong solution like "pakage_name>=min.version<=max.version'.
Such solution is more friendly for integration with any other project.

How it can help to integrate with any other projects?
As far as I know pip has problems [0] with such version declaration.
And it will be harder to develop/test/debug if other environments (e.g.
development, production) may have different versions of eggs.

[0] https://github.com/pypa/pip/issues/988


On Mon, Feb 17, 2014 at 2:01 PM, Ivan Kolodyazhny <e0ne@xxxxxxxxx> wrote:

> Hi Dmitry,
>
> I'm absolutely agree with you about keeping requirements.txt closer to
> sources.
>
> And by default pypi packages are taken from our mirrors, where we have
>> packages only for current master.
>
>
> Unfortunately, we don't have all packages in mirrors. For now, several of
> them are building during ISO creation. AFAIK, it will be fixed in a next
> release. Also, we can't install all requirements from
> https://github.com/stackforge/fuel-web/blob/master/nailgun/requirements.txt because
> the are no Shotgun package in any mirror or original PyPI.
>
> Because dependencies usually are written in ">=" format, we will get
>> latest versions of modules. Solution: use "==" format in requirement.txt
>> files
>
>
> I vote for less strong solution like
> "pakage_name>=min.version<=max.version'. Such solution is more friendly for
> integration with any other project. And don't forget for OpenStack global
> requirements.
>
> Regards,
> Ivan Kolodyazhny
>
>
> On Fri, Feb 14, 2014 at 4:16 PM, Dmitry Pyzhov <dpyzhov@xxxxxxxxxxxx>wrote:
>
>> Guys,
>>
>>
>> We have several overcomplicated places in our iso build system. It
>> produces pain for developers and for anyone who wants to build custom iso.
>> On of the places is in eggs requirements lists.
>>
>>
>> We have two requirements files for the iso:
>>
>> https://github.com/stackforge/fuel-main/blob/master/requirements-eggs.txt
>>
>>
>> https://github.com/stackforge/fuel-library/blob/master/deployment/puppet/nailgun/files/venv-ostf.txt
>>
>>
>> And we have several python modules installed from eggs:
>>
>> nailgun, with dep-list in
>> https://github.com/stackforge/fuel-web/blob/master/nailgun/requirements.txt
>>
>> shotgun with dep-list hardcoded in
>> https://github.com/stackforge/fuel-web/blob/master/shotgun/setup.py
>>
>> ostf with dep-list hardcoded in
>> https://github.com/stackforge/fuel-ostf/blob/master/setup.py
>>
>> fuelclient with dep-list hardcoded in
>> https://github.com/stackforge/fuel-web/blob/master/fuelclient/setup.py
>>
>>
>> And by default pypi packages are taken from our mirrors, where we have
>> packages only for current master.
>>
>>
>> It is inconvenient because everyone has to keep in mind this information
>> in order to build an iso or update module dependencies.
>>
>>
>> I have a proposal to keep requirements.txt files next to sources, update
>> our build system for use of this files. And learn our build system to use
>> global pypi database as a fallback for missed packages.
>>
>>
>> As an option, ostf requirements.txt can be split into several parts in
>> order to keep savanna/murano/heat clients dependencies separately.
>>
>>
>> Possible issues:
>>
>> Because dependencies usually are written in ">=" format, we will get
>> latest versions of modules. Solution: use "==" format in requirement.txt
>> files.
>>
>> OSTF installation can be broken because for some unclear reason it
>> requires installation of dependencies in one pip run and installation of
>> the module in another. Solution: create venv-ostf.txt file during build
>> time.
>>
>> We will get bunch of versions for every single module. Solution is not
>> required, python is ok with it.
>>
>>
>> I think this will be done with little effort and will make our life
>> easier for next three months.
>>
>>
>> --
>> Mailing list: https://launchpad.net/~fuel-dev
>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~fuel-dev
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
> --
> Mailing list: https://launchpad.net/~fuel-dev
> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~fuel-dev
> More help   : https://help.launchpad.net/ListHelp
>
>

References