fuel-dev team mailing list archive

[Dmitry Borodaenko <dborodaenko@xxxxxxxxxxxx>]

Having a reliable and *repeatable* way to update a deployed fuel
master node from local nailgun and fuel-library (and astute and ostf)
changes would be a very nice thing to have. Reducing the number of
cases where an ISO rebuild is needed to test a change will certainly
help speed up development, it might also increase the amount of
testing done by developers before changes are published and merged.

As Andrew has already mentioned, Fuel is not the only thing present on
the ISO that may need to be changed during development. There's
nailgun DB, CentOS and Ubuntu package repositories, packages installed
on the fuel-master itself, the bootstrap image. We need a way for
developers to test changes to all of these, two.

And to reiterate the concerns I've mentioned yesterday, even if you
have a post-deployment update tool that can take care of updating all
types of components on a deployed master node (which I suspect is a
bit more than what DmitryP is talking about), that tool still cannot
be the only option available to developers. If incremental updates are
the only way you test things, the difference between that and the
production deployment process (installing fuel master from a new ISO
image and installing all components from packages) will make you miss
all ISO and packaging specific bugs until much later when QA picks it
up.

So yeah, we need to keep the ability to build our own custom ISO
images. And we need to improve it, too. I think we need to bring the
OSCI packaging process closer to Fuel development and make it open to
external contributions, so that anyone can build their own packages
and test them by including them in a custom ISO before submitting a
review request to OSCI. It would also help take some load off the OSCI
team and let them focus more on improving their tools rather than
dealing with internal tickets asking for package rebuilds.

BTW here's how WikiMedia combines git-buildpackage with gerrit:
https://wikitech.wikimedia.org/wiki/Git-buildpackage

-DmitryB


On Wed, Jan 15, 2014 at 10:01 AM, Andrew Woodward <xarses@xxxxxxxxx> wrote:
> +1 to Dmitry P, I much prefer hacking my changes into a deployed env over
> waiting for an ISO to build, most of the time Its trivial and results in
> more stable code. The only hard thing to do is to inject deb packages since
> the CentOS master lacks the tools to rebuild the repo metadata.
>
>
> On Wed, Jan 15, 2014 at 7:47 AM, Dmitry Pyzhov <dpyzhov@xxxxxxxxxxxx> wrote:
>>
>> My vision about packages versus development:
>>
>> We should pack nailgun and fuel with OSCI tools. For development we should
>> update files in place on pre-deployed nodes. And we should have jenkins job
>> for building custom isos from branches. And we should say good-bye to local
>> iso build from the sources.
>>
>> So developer should test changes on pre-deployed node (we should create
>> instruction for that). And if he or she wants to be absolutely sure there is
>> an option to test on custom iso.
>>
>> We are moving toward this process, but have lack of resources in OSCI
>> team.
>>
>>
>>
>> On Wed, Jan 15, 2014 at 8:28 AM, Oleg Gelbukh <ogelbukh@xxxxxxxxxxxx>
>> wrote:
>>>
>>> JFYI, this thread contains multiple opinions about packages applicability
>>> to mass scale in production:
>>>
>>> http://lists.openstack.org/pipermail/openstack-dev/2014-January/023628.html
>>>
>>> Main idea is that at scale packages are too granular to be managed
>>> effectively. However, packages could be used to create base overcloud images
>>> with all benefits Alexander mentioned (i.e. verisioning etc).
>>>
>>> FWIW, OSCI project did some job to make packaging a part of development
>>> process in the past. It could continue down that path.
>>>
>>> -Oleg
>>>
>>>
>>> On Wed, Jan 15, 2014 at 2:50 AM, Dmitry Borodaenko
>>> <dborodaenko@xxxxxxxxxxxx> wrote:
>>>>
>>>> Unless we move away from package based deployment altogether (which
>>>> btw I think would be a bad idea: I agree with Aleksandr and Ivan that
>>>> packages are more suitable for production), we have to make generation
>>>> of rpm and deb packages a part of our development process. I think the
>>>> best way to achieve that is have a public git-buildpackage repository
>>>> for each package we build, and use mr/myrepos to make rebuilding all
>>>> these packages a part of make iso, as an alternative to "make
>>>> deep_clean".
>>>>
>>>> Having one deployment process for development and another process for
>>>> production is even worse than picking the wrong process. It would push
>>>> discovery of packaging-related bugs from the very beginning of the
>>>> development cycle (before the change is even pushed to gerrit) towards
>>>> the very end of the cycle (after a release candidate build was passed
>>>> to QA). It is widely accepted that the cost of fixing a bug grows
>>>> exponentially related to the time elapsed since the bug was
>>>> introduced.
>>>>
>>>> My 2c,
>>>> -DmitryB
>>>>
>>>>
>>>> On Tue, Jan 14, 2014 at 1:13 PM, Ivan Kolodyazhny <e0ne@xxxxxxxxx>
>>>> wrote:
>>>> > RPMs is good for production use. For development process we need to
>>>> > specify
>>>> > some workflow and create guidelines like a Nailgun docs to easy get
>>>> > development environment.
>>>> >
>>>> >
>>>> > --
>>>> > Regards,
>>>> > Ivan "e0ne" Kolodyazhny
>>>> >
>>>> > On Tue, Jan 14, 2014 at 7:35 PM, Andrey Korolyov
>>>> > <akorolev@xxxxxxxxxxxx>
>>>> > wrote:
>>>> >>
>>>> >> On 01/14/2014 01:59 PM, Aleksandr Didenko wrote:
>>>> >> > One more thing to consider: plugins packaging into RPMs. This will
>>>> >> > provide a set of benefits "for free", for example:
>>>> >> >
>>>> >> > - versioning
>>>> >> > - easy installation/removal/upgrades/downgrades/distribution
>>>> >> > - dependencies (including cross plugin dependencies)
>>>> >> > - conflicts (including cross plugin conflicts)
>>>> >> > - plugin consistency check (rpm -qV)
>>>> >> >
>>>> >> > Thanks
>>>> >> > --
>>>> >> > Aleksandr
>>>> >> >
>>>> >> >
>>>> >>
>>>> >> Sounds great but will be complete mess during development stage. Can
>>>> >> be
>>>> >> fixed via tgz archives during development window and 'equal' packages
>>>> >> during code freeze testing.
>>>> >>
>>>> >> >
>>>> >> > On Sat, Jan 4, 2014 at 8:38 PM, Sergey Vasilenko
>>>> >> > <svasilenko@xxxxxxxxxxxx <mailto:svasilenko@xxxxxxxxxxxx>> wrote:
>>>> >> >
>>>> >> >     I think we shouldn't tell about plugable architecture for fuel,
>>>> >> >     before divide one global manifest to set of small. First step
>>>> >> >     (master-less) already done.
>>>> >> >
>>>> >> >
>>>> >> >     On Fri, Jan 3, 2014 at 12:00 AM, Andrew Woodward
>>>> >> > <xarses@xxxxxxxxx
>>>> >> >     <mailto:xarses@xxxxxxxxx>> wrote:
>>>> >> >
>>>> >> >         We need to address that a lot (if not all) of puppet
>>>> >> > "plugins"
>>>> >> >         will be pre-baked from upstream sources. For example
>>>> >> > nagios. A
>>>> >> >         user will want to be able to use that module, and even keep
>>>> >> > it
>>>> >> >         up to date with the upstream, we should never modify the
>>>> >> > module
>>>> >> >         existing files.
>>>> >> >
>>>> >> >             Each plugin has to be implemented as a separate module.
>>>> >> > Init
>>>> >> >             manifest withouta input parameters (init.pp) should be
>>>> >> > used
>>>> >> >             as an entry point to each plugin.
>>>> >> >
>>>> >> >         No, a separate bridging file plugin_fuel_<module>.pp should
>>>> >> > be
>>>> >> >         created.
>>>> >> >
>>>> >> >
>>>> >> >             All parameters for plugin should be kept in astute.yaml
>>>> >> >
>>>> >> >         We need to fail if a required parameter is not provided.
>>>> >> >
>>>> >> >             Function which automatically loads each plugin (e.g. it
>>>> >> >             should execute init manifest of each class with name
>>>> >> > mask
>>>> >> >             "plugin_.*") is required.
>>>> >> >
>>>> >> >         This likely need to be a function of astute and even our
>>>> >> > core
>>>> >> >         modules should be loaded this way. modules included should
>>>> >> > be
>>>> >> >         based from data in astute.yaml.
>>>> >> >
>>>> >> >
>>>> >> >             Staging system should be implemented. All plugins have
>>>> >> > to be
>>>> >> >             executed in a separate stage (e.g. Stage['main'] ->
>>>> >> > stage
>>>> >> >             {'plugins':} )
>>>> >> >
>>>> >> >         We can't assume that this is OK. we can encourage plugins
>>>> >> > to use
>>>> >> >         a separate phase, however this is not enforceable in all
>>>> >> > cases
>>>> >> >         (everything must be loaded this way). We should create a
>>>> >> > set of
>>>> >> >         standard stages, and create pre and post stages as-well
>>>> >> > this way
>>>> >> >         a plugin can insert its self where necessary. A plugin
>>>> >> > without a
>>>> >> >         stage should run last. Which stage might be operated by
>>>> >> > astute.yaml
>>>> >> >
>>>> >> >         example stages might include: init, pre-networking,
>>>> >> > networking,
>>>> >> >         post-networking ...
>>>> >> >         (actual stages
>>>> >> >         are
>>>> >> >
>>>> >> > https://github.com/stackforge/fuel-library/blob/master/deployment/puppet/osnailyfacter/examples/site.pp#L14-21
>>>> >> > )
>>>> >> >
>>>> >> >
>>>> >> >             Each puppet class which is used with library has to
>>>> >> > have the
>>>> >> >             ability to be disabled from plugin
>>>> >> >             This is necessary because plugins may require to
>>>> >> > disable
>>>> >> >             some services. Alternatively this can be implemented by
>>>> >> >             using resource collections (e.g. Service <| title ==
>>>> >> >             l3-agent|> { noop => true }) but we can't execute
>>>> >> > plugin in
>>>> >> >             a separate stage in this case.
>>>> >> >
>>>> >> >         This is mostly addressed by loading all modules, including
>>>> >> > core
>>>> >> >         components the same way. If you dont wat a core module to
>>>> >> > run,
>>>> >> >         you simply get astute to not run it. Some modules might
>>>> >> > still
>>>> >> >         need further tweaking to allow more flexible overriding. If
>>>> >> > all
>>>> >> >         modules load the same way, we should be able to simply
>>>> >> > replace
>>>> >> >         the parameter into it's plugin_fuel_<module> class by
>>>> >> > calling it
>>>> >> >         instead of allowing it to be defined with the defaults.
>>>> >> >
>>>> >> >         Andrew Woodward
>>>> >> >         Mirantis
>>>> >> >
>>>> >> >         --
>>>> >> >         Mailing list: https://launchpad.net/~fuel-dev
>>>> >> >         Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>> >> >         <mailto:fuel-dev@xxxxxxxxxxxxxxxxxxx>
>>>> >> >         Unsubscribe : https://launchpad.net/~fuel-dev
>>>> >> >         More help   : https://help.launchpad.net/ListHelp
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> >     --
>>>> >> >     Mailing list: https://launchpad.net/~fuel-dev
>>>> >> >     Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>> >> >     <mailto:fuel-dev@xxxxxxxxxxxxxxxxxxx>
>>>> >> >     Unsubscribe : https://launchpad.net/~fuel-dev
>>>> >> >     More help   : https://help.launchpad.net/ListHelp
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >>
>>>> >>
>>>> >> --
>>>> >> Mailing list: https://launchpad.net/~fuel-dev
>>>> >> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>> >> Unsubscribe : https://launchpad.net/~fuel-dev
>>>> >> More help   : https://help.launchpad.net/ListHelp
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Mailing list: https://launchpad.net/~fuel-dev
>>>> > Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>> > Unsubscribe : https://launchpad.net/~fuel-dev
>>>> > More help   : https://help.launchpad.net/ListHelp
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> Dmitry Borodaenko
>>>>
>>>> --
>>>> Mailing list: https://launchpad.net/~fuel-dev
>>>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~fuel-dev
>>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>>
>>> --
>>> Mailing list: https://launchpad.net/~fuel-dev
>>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~fuel-dev
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>
>>
>> --
>> Mailing list: https://launchpad.net/~fuel-dev
>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~fuel-dev
>> More help   : https://help.launchpad.net/ListHelp
>>
>
>
>
> --
> If google has done it, Google did it right!
>
> --
> Mailing list: https://launchpad.net/~fuel-dev
> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~fuel-dev
> More help   : https://help.launchpad.net/ListHelp
>



-- 
Dmitry Borodaenko