fuel-dev team mailing list archive

[Mike Scherbakov <mscherbakov@xxxxxxxxxxxx>]

> ship two bootstrap images: 2.6.32 and 3.10, - and allow a user to switch
between them if something goes wrong by manually relinking the image used
on the master node

I think it would be a good compromise. I suggest to have 3.10 by default.


On Sat, Mar 29, 2014 at 1:16 PM, Vladimir Kuklin <vkuklin@xxxxxxxxxxxx>wrote:

> Andrew,
> >> For example Cisco UCS is not validated on CentOS using 3.10.
> 1) RHEL 7 Beta is already using 3.10 kernel. AFAIK, we created this
> workaround to help Alex Shaposhnikov exactly with UCS servers.
> >> This is the first that I've heard of hardware problems other than OVS
> and just plain missing drivers that the newer kernels have embeded (Which
> is our bad). Please provide examples.
> 2) Let me provide some examples
> https://bugs.launchpad.net/fuel/+bug/1291424
> https://bugs.launchpad.net/fuel/+bug/1260492
> https://bugs.launchpad.net/fuel/+bug/1275663
>
> >>  Again, its no longer CentOS and no longer something that we can test
> effectively.
> 3) As Andrey already said, we do already have a bunch of packages rebuilt
> by ourselves, thus it is no longer CentOS even without 3.10 kernel in
> bootstrap
>  >> We are going down the slope of needing to validate a kernel against a
> myriad of hardware platforms
> 4) As I said, 3.10 is an LTS kernel and is actively tested by Red Hat now.
> We are using the version from Fedora guys, thus using the same code.
> Speaking of 2.6.32 kernel we are also bound to testing of this kernel with
> the myriad of hardware platforms as some vendors (e.g. look into
> aforementioned NUC-related Intel bug) are not going to provide support for
> 2.6.x kernels anymore.
>
> My suggestion is that we build and ship two bootstrap images: 2.6.32 and
> 3.10, - and allow a user to switch between them if something goes wrong by
> manually relinking the image used on the master node.
>
> On Sat, Mar 29, 2014 at 2:06 AM, Andrew Woodward <xarses@xxxxxxxxx> wrote:
>
>> Firstly, 3.10-lt kernel is an LTS kernel supported by Linux Foundation
>>
>> For example Cisco UCS is not validated on CentOS using 3.10. Cisco won't
>> provide support because they didn't validate it. Another example is if a
>> vendor provides kernel drives they may be bound to 2.6 and not provide
>> packages that cant be bound to 3.10 again because they dont test it. We
>> will have issues with this until RHEL is on 3.10 and vendors start to test
>> and validate against it.
>>
>>
>>> We've already experienced issues with hardware unsupported by 2.6.32
>>> kernel
>>
>> This is the first that I've heard of hardware problems other than OVS and
>> just plain missing drivers that the newer kernels have embeded (Which is
>> our bad). Please provide examples.
>>
>>  If you want to upgrade CentOS 6.x to 3.10 and make it default, you
>> can't this is Red Hat's job. This is a limitation that is known to the
>> people who select CentOS. If they dont like it, then they can use the
>> fedora-lt kernel at their own volition, wait for CentOS 7, or use Ubuntu.
>> Taking away the default kernel from CentOS 6.x violates a core assumption
>> users are going to make if they select to install it and makes it NOT
>> CentOS.
>>
>> Yes I object to using the 3.10 kernel any were for CentOS by default.
>> Again, its no longer CentOS and no longer something that we can test
>> effectively. We are going down the slope of needing to validate a kernel
>> against a myriad of hardware platforms. This is something that we
>> absolutely should not be attempting to do. Hardware vendors and Distos do
>> this for us with their default kernel for distributions they support.
>>
>> again this goes back to what problem are we trying to solve by moving the
>> bootstrap kernel to 3.10? I'm still guessing here so you really need to
>> come up with a story here so that we can actually solve the problem.
>>
>> It sounds like you are looking to improve UX when installing, great!.
>> However this is the wrong way. We can either continue to have this A/B
>> split between the kernel that is used by Ubuntu and the kernel we are using
>> for CentOS (no moving to 3.10 in CentOS does not resolve this issue) or we
>> can fix this by improving the approach.
>>
>> 1) I think that if we just want to improve the bootstrap kernel, then we
>> should switch bootstrap to Ubuntu, it gets a 3.8 kernel
>> 2) If we want to solve this A/B issue then we need to prepare both CentOS
>> and Ubuntu bootstrap images and allow the user to switch. It's the only
>> complete solution and I'd guess that most of the time they will only use
>> one, or the other. (Unless they test one and then switch to the other).
>>
>>
>>
>> On Fri, Mar 28, 2014 at 2:17 PM, Vladimir Kuklin <vkuklin@xxxxxxxxxxxx>wrote:
>>
>>> Andrew, Dmitry N,
>>>
>>> Firstly, 3.10-lt kernel is an LTS kernel supported by Linux Foundation,
>>> this is not a piece of some short-term supported code. Moreover, this
>>> kernel has a support of much more hardware as it is not so ancient as
>>> 2.6.32 kernel is. We've already experienced issues with hardware
>>> unsupported by 2.6.32 kernel. This makes our users pretty unhappy, that
>>> they cannot use their newer hardware, even though it is supported by 3.10
>>> and Ubuntu 3.x kernels. Thus, changing bootstrap kernel allows users to use
>>> their newer hardware without breaking things as user is still able to
>>> choose the kernel for main OS installation. Thus we faced the fact that we
>>> need either to provide bootstrap with the newest LTS kernel or create
>>> several bootstrap images, which looks much more complicated. We chose the
>>> compromise. This is obviously a mistake, that we did not dicsuss this in
>>> this ML. So, if you have any objections and you think that we should revert
>>> this change, please, provide you arguments. In my opinion, changing only
>>> the bootstrap image kernel is not a big deal.
>>>
>>>
>>>
>>>
>>> On Fri, Mar 28, 2014 at 10:31 PM, Dmitriy Novakovskiy <
>>> dnovakovskiy@xxxxxxxxxxxx> wrote:
>>>
>>>> +1 to Andrew's points. I've also been dealing with a bulk of vendors'
>>>> pain - plugins (Cinder, Neutron) not working properly due to our CentOS
>>>> being not actually CentOS (kernel+qemu versions).
>>>>
>>>> ---
>>>> Regards,
>>>> Dmitriy
>>>>
>>>>
>>>> On Fri, Mar 28, 2014 at 11:22 AM, Andrew Woodward <xarses@xxxxxxxxx>wrote:
>>>>
>>>>> Right now master node has 2.6 kernel, bootstrap has 3.10 and target
>>>>>> node has an option in settings with 2.6 kernel as default.
>>>>>
>>>>>
>>>>> WHAT? Why is the bootstrap 3.10 already? I already spoke to the issues
>>>>> that some users have had about this. This should have been discussed on the
>>>>> ML before making a change like this.
>>>>>
>>>>> First off, I'm sorry but making the 3.10 the default kernel for our
>>>>> "CentOS 6.x" installs inherently makes it NOT CentOS, it becomes Fedora 18.
>>>>> If we are going to do this, then we might as well switch from CentOS base
>>>>> to a proper Fedora base. At the same time, making 3.10 the default kernel
>>>>> impacts vendor support. Vendors that might have supported CentOS 6.x now
>>>>> wont. I've already been in on a conversation with with a user, and
>>>>> separately two vendors that are having a hard time using our "CentOS"
>>>>> because we don't have a stock kernel. This removes support-ability from the
>>>>> list of reasons to use CentOS, which is one of the core reasons I've seen
>>>>> organizations select it.
>>>>>
>>>>> Second, what is the goal here?
>>>>>
>>>>> * if we are trying reduce issues from master -> bootstrap -> target
>>>>> node? switching kernel around? If this is the case then we really should
>>>>> just change master and bootstrap to Ubuntu (which I've been considering
>>>>> raising anyways).
>>>>>
>>>>> * If we are trying to reduce the OVS issues (or other old kernel
>>>>> issues), then the current workup is correct, the user should have to
>>>>> explicitly choose a kernel that wasn't part of the base distribution, this
>>>>> is important as is must be their choice and not default to enter into a
>>>>> mode that reduces their support.
>>>>>
>>>>> * If we are trying to get CentOS to a more modern kernel, then we
>>>>> really just have to wait for CentOS 7 otherwise we are really better off
>>>>> tracking Fedora instead of CentOS
>>>>>
>>>>> Third, this is changing our user story of "No vendor lock-in" to
>>>>> "You're stuck with Mirantis". We really need to take a step back an
>>>>> evaluate where we are going. If we want to maintain our on proprietary
>>>>> distro then we need to stop beating around and do it, and do it well. If we
>>>>> are going to fight vendor lock-in, then we need to be more cautions about
>>>>> what distro packages we are modifying and why. We also need to not replace
>>>>> distro packages but instead maintain a clean distro repos and a MOS repo,
>>>>> this will grant us a clear line of what we are doing and better allow
>>>>> others to participate (like bringing in other distros, vendor
>>>>> compatibility, and others)
>>>>>
>>>>>
>>>>> On Fri, Mar 28, 2014 at 5:08 AM, Matthew Mosesohn <
>>>>> mmosesohn@xxxxxxxxxxxx> wrote:
>>>>>
>>>>>> I propose that since we do kernel-lt 3.10.30 as a default for installs
>>>>>> and bootstrap, we should default to it for Fuel master node as well.
>>>>>> I believe we should keep the 2.6.32 kernel around just in case there
>>>>>> is a regression in using the newer kernel, but maybe we can drop it
>>>>>> after a release cycle or two if there are no major issues.
>>>>>>
>>>>>> The build tiem option I mentioned to you would be to optionally
>>>>>> generate a bootstrap image with the 2.6.32 kernel during ISO build. It
>>>>>> isn't likely we will need to build it, but it should be easy to swap
>>>>>> back if the need arises.
>>>>>>
>>>>>> On Fri, Mar 28, 2014 at 4:03 PM, Dmitry Pyzhov <dpyzhov@xxxxxxxxxxxx>
>>>>>> wrote:
>>>>>> > Guys,
>>>>>> >
>>>>>> > we have two kernels: default 2.6.32 and kernel-lt 3.10.30. And we
>>>>>> have three
>>>>>> > types of CentOS nodes: master node, bootstrap, target node.
>>>>>> >
>>>>>> > Right now master node has 2.6 kernel, bootstrap has 3.10 and target
>>>>>> node has
>>>>>> > an option in settings with 2.6 kernel as default.
>>>>>> >
>>>>>> > There is a suggestion from our team members to use 3.10 kernel
>>>>>> everywhere
>>>>>> > and add build-time option for 2.6.
>>>>>> >
>>>>>> > Any concerns?
>>>>>> >
>>>>>> > --
>>>>>> > Mailing list: https://launchpad.net/~fuel-dev
>>>>>> > Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>>>> > Unsubscribe : https://launchpad.net/~fuel-dev
>>>>>> > More help   : https://help.launchpad.net/ListHelp
>>>>>> >
>>>>>>
>>>>>> --
>>>>>> Mailing list: https://launchpad.net/~fuel-dev
>>>>>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>>>> Unsubscribe : https://launchpad.net/~fuel-dev
>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Andrew
>>>>> Mirantis
>>>>> Ceph community
>>>>>
>>>>> --
>>>>> Mailing list: https://launchpad.net/~fuel-dev
>>>>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>>> Unsubscribe : https://launchpad.net/~fuel-dev
>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>
>>>>>
>>>>
>>>> --
>>>> Mailing list: https://launchpad.net/~fuel-dev
>>>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~fuel-dev
>>>> More help   : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>>
>>>
>>> --
>>> Yours Faithfully,
>>> Vladimir Kuklin,
>>> Fuel Library Tech Lead,
>>> Mirantis, Inc.
>>> +7 (495) 640-49-04
>>> +7 (926) 702-39-68
>>> Skype kuklinvv
>>> 45bk3, Vorontsovskaya Str.
>>> Moscow, Russia,
>>> www.mirantis.com <http://www.mirantis.ru/>
>>> www.mirantis.ru
>>> vkuklin@xxxxxxxxxxxx
>>>
>>
>>
>>
>> --
>> Andrew
>> Mirantis
>> Ceph community
>>
>
>
>
> --
> Yours Faithfully,
> Vladimir Kuklin,
> Fuel Library Tech Lead,
> Mirantis, Inc.
> +7 (495) 640-49-04
> +7 (926) 702-39-68
> Skype kuklinvv
> 45bk3, Vorontsovskaya Str.
> Moscow, Russia,
> www.mirantis.com <http://www.mirantis.ru/>
> www.mirantis.ru
> vkuklin@xxxxxxxxxxxx
>
> --
> Mailing list: https://launchpad.net/~fuel-dev
> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~fuel-dev
> More help   : https://help.launchpad.net/ListHelp
>
>


-- 
Mike Scherbakov
#mihgen