fuel-dev team mailing list archive

[Igor Kalnitsky <ikalnitsky@xxxxxxxxxxxx>]

Hello,

I've launched upgrade process a lot of times on my working env, so I
can share my experience with these issues.

> https://bugs.launchpad.net/fuel/+bug/1359725 (Docker dies during the upgrade)

I never met that issue myself, so tt looks like it was appeared twice
on QA side. Maybe Evgeniy could provide more info this one., but I
believe it's really rare.

> https://bugs.launchpad.net/fuel/+bug/1357357 (Upgrades, sometimes docker allocates the same IPs for different containers)

I met that issue two times on my own env. Meantime, the issue is quite
common on CI (I'm not sure, but not long time ago it falls with
~10-15% probability). I believe the fall probability is strongly
depends on the hardware and how it handles multi-threaded
applications.

Thanks,
Igor

On Tue, Aug 26, 2014 at 10:11 AM, Mike Scherbakov
<mscherbakov@xxxxxxxxxxxx> wrote:
>> There's been a number of decisions already impacting UX in a negative way
>> (confusing versioning in OpenStack patching, large size of the upgrade
>> tarball, issues with containers) and we just can't afford going down the
>> slippery slope.
> These are rather side effects than "decisions".
>
> "confusing versioning in OpenStack patching" - if we didn't change puppet
> manifests and Fuel/OpenStack reference architecture in next Fuel versions,
> then it would be as simple as patching from 5.0 to 5.1. But it appeared to
> be more complicated system than you would initially think of, so in general
> 5.0.2 may not be equal to 5.1, that's where all things come up. If we had
> OpenStack upgrades, then we could just say 5.0 -> 6.0 - easy.
> "large size of the upgrade tarball" - it goes from the feature which is not
> yet complete. We can choose here - to deliver it as is (with large tarball),
> or wait one more release when it will be fixed. We chose approach which
> seemed to be balanced: users can start using upgrades, and we can improve UX
> over time.
> "issues with containers" - we have same issues with everything. Let's take
> Galera, for example. It's just issues. We can question maturity of tools we
> use, and here I'd agree - we spent too much fixing issues around Docker. At
> the same time, if we were about taking our own journey with LXC, we would
> likely spend even more time inventing our own bicycle.
>
>> I would like to understand how bad the issue is. I.e.
> Evgeny, please respond on those. Also, I'd like to ask everyone to provide
> such information in every bug you report if possible (or if get this info
> later, put comments): in many bug reports it is unclear to understand how
> severe issue is.
>
> Thanks,
>
>
> On Tue, Aug 26, 2014 at 4:18 AM, Roman Alekseenkov
> <ralekseenkov@xxxxxxxxxxxx> wrote:
>>
>> There's been a parallel thread, but still. I have a question regarding
>> Docker-related issues:
>> - https://bugs.launchpad.net/fuel/+bug/1359725 (Docker dies during the
>> upgrade)
>> - https://bugs.launchpad.net/fuel/+bug/1357357 (Upgrades, sometimes docker
>> allocates the same IPs for different containers)
>>
>> How consistently we can reproduce these problems?
>>
>> I would like to understand how bad the issue is. I.e.
>> - what are the chances that upgrade will fail on the 1st attempt (because
>> of either of those issues)
>> - what are the chances that upgrade will fail on the 2nd attempt (i.e.
>> user will have to run upgrade.sh more than two times)
>>
>> There's been a number of decisions already impacting UX in a negative way
>> (confusing versioning in OpenStack patching, large size of the upgrade
>> tarball, issues with containers) and we just can't afford going down the
>> slippery slope.
>>
>> Thanks,
>> Roman
>>
>>
>> On Mon, Aug 25, 2014 at 6:52 AM, Evgeniy L <eli@xxxxxxxxxxxx> wrote:
>>>
>>> Hi, here you can find a list of problems which we
>>> have for the current release
>>>
>>> Problem with keystone container during the upgrade
>>> https://bugs.launchpad.net/fuel/+bug/1353574
>>> It happens often on my environment, there is a patch from
>>> Matthew to fix the problem
>>> https://review.openstack.org/#/c/116613/
>>>
>>> Docker dies during the upgrade
>>> https://bugs.launchpad.net/fuel/+bug/1359725
>>> It happens not often, but I saw it 2 times I
>>> wrote the instruction how to work around it, also
>>> it affects 5.0.1 upgrade tar ball, we should add
>>> it to known issues list
>>> https://bugs.launchpad.net/fuel/+bug/1359725/comments/5
>>>
>>> Build iptables 1.4.11 for centos
>>> https://bugs.launchpad.net/fuel/+bug/1359096
>>> Waiting for mos-linux team, it should help to
>>> solve problems with duplicated iptables rules
>>> in some cases
>>>
>>> 5.1 upgrade script failed with UpgradeVerificationError:
>>> Failed to run services ['integration_postgres_nailgun_nginx',
>>> 'integration_nginx_nailgun']
>>> https://bugs.launchpad.net/fuel/+bug/1349287
>>> It was reopened, we are trying to reproduce it to get
>>> the environment with the problem
>>>
>>> Upgrades, sometimes docker allocates the same IPs
>>> for different containers
>>> https://bugs.launchpad.net/fuel/+bug/1357357
>>> Decided to move to 6.0, because it can be fixed only
>>> by patched docker, it's very risky to upgrade docker,
>>> decided to postpone.
>>> Details are here
>>> https://www.mail-archive.com/fuel-dev@xxxxxxxxxxxxxxxxxxx/msg01328.html
>>>
>>>
>>> Thanks
>>>
>>> --
>>> Mailing list: https://launchpad.net/~fuel-dev
>>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~fuel-dev
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>
>>
>> --
>> Mailing list: https://launchpad.net/~fuel-dev
>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~fuel-dev
>> More help   : https://help.launchpad.net/ListHelp
>>
>
>
>
> --
> Mike Scherbakov
> #mihgen
>
>
> --
> Mailing list: https://launchpad.net/~fuel-dev
> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~fuel-dev
> More help   : https://help.launchpad.net/ListHelp
>