fuel-dev team mailing list archive

[Mike Scherbakov <mscherbakov@xxxxxxxxxxxx>]

Branches are created.
Consider CODE FREEZED.

Please note, that all further fixes must be landed first in master branch,
and must not be accepted into stable/4.1 before they are accepted into
master. There are can be exceptions, but please mark those in commit
message then and explain why we don't need those in master.

Thanks all for the hard work!!!
And let's nail down remaining criticals in Murano and figure out what's up
with bonding.


On Tue, Feb 25, 2014 at 8:58 PM, Matthew Mosesohn <mmosesohn@xxxxxxxxxxxx>wrote:

> Just do stable/4.1 for all and get it over with faster.
>
> On Tue, Feb 25, 2014 at 8:55 PM, Mike Scherbakov
> <mscherbakov@xxxxxxxxxxxx> wrote:
> > Thanks Matt. #2 must be fixed, but that's trivial change, so I propose
> to do
> > code freezing and land package then into both "master" mirror &
> stable/4.1.
> >
> > So what repos do I need to skip for creating stable/4.1 for any reason?
> >
> > stackforge/fuel-astute
> > stackforge/fuel-devops
> > stackforge/fuel-docs
> > stackforge/fuel-library
> > stackforge/fuel-main
> > stackforge/fuel-ostf
> > stackforge/fuel-ostf-plugin
> > stackforge/fuel-provision
> > stackforge/fuel-web
> >
> >
> > On Tue, Feb 25, 2014 at 8:34 PM, Matthew Mosesohn <
> mmosesohn@xxxxxxxxxxxx>
> > wrote:
> >>
> >> There are 2 issues with glance:
> >> 1 - glance API needs filesystem_store_data setting added to
> >> glance-api.conf (even though it's defined also in glance-cache.conf).
> >> This is a non-fatal error and doesn't block normal usage. I proposed
> >> https://review.openstack.org/76252 - a one line trivial fix
> >>
> >> 2 - /var/lib/glance/image-cache/{queue,invalid,incomplete} are set to
> >> root:root ownership instead of glance:glance. I put an OSCI ticket in
> >> to fix this. It is really the job of the package to set file ownership
> >> and this one doesn't even make sense because
> >> /var/lib/glance/image-cache (its parent dir) is glance:glance. OSCI
> >> bug is https://mirantis.jira.com/browse/OSCI-1091 and I think it
> >> should be marked a blocker, but it also is a one line fix for
> >> glance-api post-install script.
> >>
> >> On Tue, Feb 25, 2014 at 8:28 PM, Andrew Woodward <
> awoodward@xxxxxxxxxxxx>
> >> wrote:
> >> > +1
> >> >
> >> >
> >> > On Tue, Feb 25, 2014 at 8:17 AM, Mike Scherbakov
> >> > <mscherbakov@xxxxxxxxxxxx>
> >> > wrote:
> >> >>
> >> >> A few more things were nailed down during these hours.
> >> >>
> >> >> I've moved https://bugs.launchpad.net/fuel/+bug/1284261 to 5.0, I
> hope
> >> >> no
> >> >> objections to it.
> >> >>
> >> >> OpenVSwitch with bonding sometimes stops accepting ARPs - Critical,
> but
> >> >> looks like we can't configure LACP on our switches. Active-backup
> mode
> >> >> works
> >> >> fine.
> >> >> OSError: [Errno 13] Permission denied: for murano dashboard - should
> be
> >> >> already fixed by package update, waiting for someone to confirm
> >> >> Murano has too open dirs (o+w) - security issue. Waiting for patches
> >> >> from
> >> >> Murano team
> >> >> Glance Error: Could not find filesystem_store_datadir in
> configuration
> >> >> options. - waiting from Matt more details on this issue, but looks
> like
> >> >> it
> >> >> low-priority glance caching issue
> >> >>
> >> >> I believe we can call for code freeze for all components except
> Murano.
> >> >> It
> >> >> means that we will still need to merge trivial changes into Murano
> DEB
> >> >> specs, and start accepting testing of Murano only after it is done.
> >> >> For all other pieces of Fuel I believe we can start acceptance
> testing.
> >> >>
> >> >> Any objections?
> >> >>
> >> >>
> >> >> On Tue, Feb 25, 2014 at 3:42 PM, Mike Scherbakov
> >> >> <mscherbakov@xxxxxxxxxxxx> wrote:
> >> >>>
> >> >>> Folks,
> >> >>> few more things were tested and merged. Please pay high attention to
> >> >>> every new bug created now, and try to triage immediately - we need
> to
> >> >>> know
> >> >>> whether it's low priority and can be deferred to 5.0 or critical,
> >> >>> which must
> >> >>> be addressed in 4.1.
> >> >>>
> >> >>> We are one step away from calling for a code-freeze.
> >> >>> Largest thing to verify is kernel 3.10
> >> >>> (https://bugs.launchpad.net/fuel/+bug/1263072,
> >> >>> https://bugs.launchpad.net/fuel/+bug/1282493)
> >> >>>
> >> >>> Needs to be triaged/try/comments provided:
> >> >>> https://bugs.launchpad.net/fuel/+bug/1284092
> >> >>> https://bugs.launchpad.net/fuel/+bug/1284236
> >> >>> https://bugs.launchpad.net/fuel/+bug/1284177
> >> >>>
> >> >>> Update about bonding is needed from Andrey:
> >> >>> https://bugs.launchpad.net/fuel/+bug/1272842
> >> >>>
> >> >>> Murano-related issues:
> >> >>> https://bugs.launchpad.net/fuel/+bug/1282613 - Critical
> >> >>> https://bugs.launchpad.net/fuel/+bug/1284574
> >> >>>
> >> >>> Other:
> >> >>> https://bugs.launchpad.net/fuel/+bug/1284571 - slow getting of
> master
> >> >>> IP.
> >> >>> Let's discuss if it's Ok to merge in 4.1.
> >> >>>
> >> >>> Thanks,
> >> >>>
> >> >>>
> >> >>> On Tue, Feb 25, 2014 at 2:39 PM, Aleksey Kasatkin
> >> >>> <akasatkin@xxxxxxxxxxxx> wrote:
> >> >>>>
> >> >>>> >>> #1283805 In fuelmenu, if you set IP for master, static pool
> >> >>>> >>> starts
> >> >>>> >>> from same IP -- merged,
> >> >>>> This particular problem is fixed, ticket can be closed.
> >> >>>> Andrey pointed another problem:
> >> >>>> https://bugs.launchpad.net/fuel/+bug/1283805/comments/11
> >> >>>> that can be discussed within separate ticket.
> >> >>>>
> >> >>>>
> >> >>>> Aleksey Kasatkin
> >> >>>>
> >> >>>> S. Software Developer | Mirantis, Inc. | http://www.mirantis.com
> >> >>>> cell: +380938330852 | skype: alexeyk_ru
> >> >>>>
> >> >>>>
> >> >>>> On Tue, Feb 25, 2014 at 8:44 AM, Andrew Woodward
> >> >>>> <awoodward@xxxxxxxxxxxx> wrote:
> >> >>>>>
> >> >>>>> I agree kernel-lt-firmware and kernel-lt-headers conflict, headers
> >> >>>>> can
> >> >>>>> only be installed by the package section because it has deps that
> it
> >> >>>>> won't
> >> >>>>> resolve otherwise, but kernel-lt wont replace kernel in that
> >> >>>>> section, hence
> >> >>>>> the hokey adding it removing the ones we don't want and double
> >> >>>>> checking we
> >> >>>>> have the ones we do.
> >> >>>>>
> >> >>>>> Created https://bugs.launchpad.net/fuel/+bug/1284482 to track
> defect
> >> >>>>>
> >> >>>>>
> >> >>>>> On Mon, Feb 24, 2014 at 9:57 PM, Matthew Mosesohn
> >> >>>>> <mmosesohn@xxxxxxxxxxxx> wrote:
> >> >>>>>>
> >> >>>>>> I'll test the updated CentOS kernel for you today and review.
> >> >>>>>>
> >> >>>>>> I see why you had to do the ugly hack for kernel vs kernel-ml,
> and
> >> >>>>>> normally this is handled via "conflicts" or "obsoletes" flags. I
> >> >>>>>> know
> >> >>>>>> we can't put both kernels in the same repo with obsoletes flag,
> but
> >> >>>>>> maybe we can make it blend with conflicts instead.  If this code
> >> >>>>>> works, let's ship it for 4.1 since we're down to the wire, but
> I'd
> >> >>>>>> like to refactor it into something a little less hackish and
> >> >>>>>> doesn't
> >> >>>>>> make systems complain.
> >> >>>>>>
> >> >>>>>> On Tue, Feb 25, 2014 at 9:40 AM, Andrew Woodward
> >> >>>>>> <awoodward@xxxxxxxxxxxx> wrote:
> >> >>>>>> > https://review.openstack.org/#/c/76090/ for fuel-web to
> convert
> >> >>>>>> > names from
> >> >>>>>> > kernel_ml to kernel_lt
> >> >>>>>> >
> >> >>>>>> > https://review.openstack.org/#/c/76089/ to fix the package
> names
> >> >>>>>> > and
> >> >>>>>> > crazy
> >> >>>>>> > amount of work that is required to effectively get anaconda to
> >> >>>>>> > change the
> >> >>>>>> > kernel before it finishes.
> >> >>>>>> >
> >> >>>>>> > https://review.openstack.org/76092 to fix the rpm requires and
> >> >>>>>> > should be
> >> >>>>>> > merged after OSCI has all packages.
> >> >>>>>> >
> >> >>>>>> > Andrew.
> >> >>>>>> >
> >> >>>>>> >
> >> >>>>>> >
> >> >>>>>> > On Mon, Feb 24, 2014 at 8:41 PM, Dmitry Borodaenko
> >> >>>>>> > <dborodaenko@xxxxxxxxxxxx> wrote:
> >> >>>>>> >>
> >> >>>>>> >> Team,
> >> >>>>>> >>
> >> >>>>>> >> We are almost ready to announce code freeze for 4.1.
> >> >>>>>> >> Unfortunately,
> >> >>>>>> >> we
> >> >>>>>> >> couldn't merge everything that we need today, so we can't
> freeze
> >> >>>>>> >> just
> >> >>>>>> >> yet. The good news is that we didn't find any major
> regressions
> >> >>>>>> >> as
> >> >>>>>> >> of
> >> >>>>>> >> ISO #195, and there is only a handful of problems that we
> still
> >> >>>>>> >> need
> >> >>>>>> >> to address:
> >> >>>>>> >>
> >> >>>>>> >> 1) Bonding
> >> >>>>>> >> 2) Kernel 3.10 for CentOS
> >> >>>>>> >> 3) Post-deploy orchestration workaround
> >> >>>>>> >>
> >> >>>>>> >> I believe that other outstanding issues are not critical
> enough
> >> >>>>>> >> to
> >> >>>>>> >> be
> >> >>>>>> >> considered release blockers. If you know of any other bugs
> that
> >> >>>>>> >> must
> >> >>>>>> >> be fixed in 4.1, please speak up now. Otherwise, I propose a
> >> >>>>>> >> soft
> >> >>>>>> >> code
> >> >>>>>> >> freeze: lets not merge any other changes except fixes for the
> >> >>>>>> >> above
> >> >>>>>> >> three issues, and declare code freeze as soon as these are
> >> >>>>>> >> fixed.
> >> >>>>>> >>
> >> >>>>>> >> Bonding:
> >> >>>>>> >>
> >> >>>>>> >> There was a lot of confusion today about #1272842 (OpenVSwitch
> >> >>>>>> >> with
> >> >>>>>> >> bonding sometimes stops accepting ARPs). We thought we have a
> >> >>>>>> >> way
> >> >>>>>> >> to
> >> >>>>>> >> reliably reproduce this bug, but all we found was a lot of
> >> >>>>>> >> problems
> >> >>>>>> >> with our test methodology for NIC bonding, and a real and
> easily
> >> >>>>>> >> fixed
> >> >>>>>> >> bug #1284384 (Remove balance-tcp bond mode option). Our
> >> >>>>>> >> conclusion
> >> >>>>>> >> so
> >> >>>>>> >> far is that bonding isn't going to work in our devops/kvm
> based
> >> >>>>>> >> test
> >> >>>>>> >> environment, and we will need a real switch with LACP to have
> >> >>>>>> >> another
> >> >>>>>> >> go at trying to reproduce the bonding/ARP problem. If that is
> >> >>>>>> >> the
> >> >>>>>> >> case
> >> >>>>>> >> and we can't really reproduce #1272842 tomorrow, I propose
> that
> >> >>>>>> >> we
> >> >>>>>> >> fix
> >> >>>>>> >> #1284384 and leave #1272842 as a known limitation.
> >> >>>>>> >>
> >> >>>>>> >> Kernel 3.10 for CentOS
> >> >>>>>> >>
> >> >>>>>> >> Andrew's fuel-library patch was merged prematurely (before the
> >> >>>>>> >> required kernel rpm's where created and added to the package
> >> >>>>>> >> repositories). Andrew is close to getting his Kickstart script
> >> >>>>>> >> changes
> >> >>>>>> >> work with the new kernel-lt packages, but unfortunately the
> new
> >> >>>>>> >> packages still miss the matching firmware binary package, he
> had
> >> >>>>>> >> to
> >> >>>>>> >> download that package from the upstream site to manually test
> >> >>>>>> >> his
> >> >>>>>> >> changes. I think we need some close coordination between
> Andrew,
> >> >>>>>> >> Matt,
> >> >>>>>> >> and OSCI team tomorrow morning Pacific time to tie all loose
> >> >>>>>> >> ends
> >> >>>>>> >> here: we need the firmware package from OSCI, and it would be
> >> >>>>>> >> very
> >> >>>>>> >> nice if Matt could review Andrew's fixes before we merge an
> >> >>>>>> >> updated
> >> >>>>>> >> version of them.
> >> >>>>>> >>
> >> >>>>>> >> Post-deploy orchestration workaround
> >> >>>>>> >>
> >> >>>>>> >> There's a -1 from QA on
> https://review.openstack.org/#/c/75389/
> >> >>>>>> >> ,
> >> >>>>>> >> it
> >> >>>>>> >> can't be merged until that is resolved. There's hope that this
> >> >>>>>> >> was
> >> >>>>>> >> a
> >> >>>>>> >> one-off test failure unrelated to the code change: Aleksandr
> >> >>>>>> >> Didenko
> >> >>>>>> >> commented that at least the /etc/hosts part worked fine,
> >> >>>>>> >> although
> >> >>>>>> >> he
> >> >>>>>> >> wasn't able to check the radosgw side of the fix. If that's
> the
> >> >>>>>> >> case,
> >> >>>>>> >> we should be able to merge this tomorrow.
> >> >>>>>> >>
> >> >>>>>> >> Other critical and high priority bugs:
> >> >>>>>> >>
> >> >>>>>> >> #1284002 Request for HA deployment -- the consensus this
> morning
> >> >>>>>> >> was
> >> >>>>>> >> this this bug report is invalid, I think we need a document on
> >> >>>>>> >> how
> >> >>>>>> >> to
> >> >>>>>> >> manually test Neutron, something similar to
> >> >>>>>> >> fuel-library/deployment/puppet/ceph/README.md, so that
> everyone
> >> >>>>>> >> can
> >> >>>>>> >> agree and what is and isn't supposed to work. having an OSTF
> >> >>>>>> >> test
> >> >>>>>> >> script helps, but is not enough, there should be a document
> >> >>>>>> >> explains
> >> >>>>>> >> the test steps, and may include checks that are not easily
> >> >>>>>> >> scripted
> >> >>>>>> >>
> >> >>>>>> >> #1263934 fuelmenu NTP settings prevent me from proceeding --
> >> >>>>>> >> merged
> >> >>>>>> >>
> >> >>>>>> >> #1267431 Get rid of hardcoded "admin" tenant in puppet modules
> >> >>>>>> >> -- a
> >> >>>>>> >> short-term fix was merged, the rest should be postponed to 5.0
> >> >>>>>> >>
> >> >>>>>> >> #1283805 In fuelmenu, if you set IP for master, static pool
> >> >>>>>> >> starts
> >> >>>>>> >> from same IP -- merged, not sure if there's more work left for
> >> >>>>>> >> 5.0
> >> >>>>>> >> or
> >> >>>>>> >> this can be closed?
> >> >>>>>> >>
> >> >>>>>> >> #1283812 Stop deploy tasks fails if any node was inaccessible
> --
> >> >>>>>> >> doesn't look like this can be done in 4.1, unless a really
> safe
> >> >>>>>> >> and
> >> >>>>>> >> non-intrusive fix is proposed tomorrow
> >> >>>>>> >>
> >> >>>>>> >> #1284092 Instances can't get metadata and IP addresses --
> didn't
> >> >>>>>> >> have
> >> >>>>>> >> time to triage, is this confirmed?
> >> >>>>>> >>
> >> >>>>>> >> #1284177 OS not erased from node after cluster is deleted -- a
> >> >>>>>> >> duplicate of #1283825?
> >> >>>>>> >>
> >> >>>>>> >> #1284236 Glance Error: Could not find filesystem_store_datadir
> >> >>>>>> >> in
> >> >>>>>> >> configuration options -- didn't have time to triage
> >> >>>>>> >>
> >> >>>>>> >> #1284261 Astute must compare a sets of UIDs when collect data
> >> >>>>>> >> from
> >> >>>>>> >> netprobe.py -- proposed fix is a simple one-liner, can this be
> >> >>>>>> >> done
> >> >>>>>> >> tomorrow?
> >> >>>>>> >>
> >> >>>>>> >> #1284270 Please, add notification for user of fuel-cli about
> >> >>>>>> >> custom
> >> >>>>>> >> settings -- I think this should be postpoed to 5.0
> >> >>>>>> >>
> >> >>>>>> >> #1284193 ImageNotFound: error opening image
> >> >>>>>> >> /var/lib/nova/instances/_base/ -- Nova bug (live migrations
> work
> >> >>>>>> >> but
> >> >>>>>> >> non-live migrations don't work with ephemeral RBD backend),
> >> >>>>>> >> postponed
> >> >>>>>> >> to 5.0
> >> >>>>>> >>
> >> >>>>>> >> --
> >> >>>>>> >> Dmitry Borodaenko
> >> >>>>>> >>
> >> >>>>>> >> --
> >> >>>>>> >> You received this message because you are subscribed to the
> >> >>>>>> >> Google
> >> >>>>>> >> Groups
> >> >>>>>> >> "fuel-core-team" group.
> >> >>>>>> >> To unsubscribe from this group and stop receiving emails from
> >> >>>>>> >> it,
> >> >>>>>> >> send an
> >> >>>>>> >> email to fuel-core-team+unsubscribe@xxxxxxxxxxxx.
> >> >>>>>> >> For more options, visit
> >> >>>>>> >> https://groups.google.com/a/mirantis.com/groups/opt_out.
> >> >>>>>> >
> >> >>>>>> >
> >> >>>>>> > --
> >> >>>>>> > You received this message because you are subscribed to the
> >> >>>>>> > Google
> >> >>>>>> > Groups
> >> >>>>>> > "fuel-core-team" group.
> >> >>>>>> > To unsubscribe from this group and stop receiving emails from
> it,
> >> >>>>>> > send an
> >> >>>>>> > email to fuel-core-team+unsubscribe@xxxxxxxxxxxx.
> >> >>>>>> > For more options, visit
> >> >>>>>> > https://groups.google.com/a/mirantis.com/groups/opt_out.
> >> >>>>>
> >> >>>>>
> >> >>>>> --
> >> >>>>> You received this message because you are subscribed to the Google
> >> >>>>> Groups "fuel-core-team" group.
> >> >>>>> To unsubscribe from this group and stop receiving emails from it,
> >> >>>>> send
> >> >>>>> an email to fuel-core-team+unsubscribe@xxxxxxxxxxxx.
> >> >>>>> For more options, visit
> >> >>>>> https://groups.google.com/a/mirantis.com/groups/opt_out.
> >> >>>>
> >> >>>>
> >> >>>> --
> >> >>>> You received this message because you are subscribed to the Google
> >> >>>> Groups "fuel-core-team" group.
> >> >>>> To unsubscribe from this group and stop receiving emails from it,
> >> >>>> send
> >> >>>> an email to fuel-core-team+unsubscribe@xxxxxxxxxxxx.
> >> >>>> For more options, visit
> >> >>>> https://groups.google.com/a/mirantis.com/groups/opt_out.
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>> --
> >> >>> Mike Scherbakov
> >> >>> #mihgen
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Mike Scherbakov
> >> >> #mihgen
> >> >>
> >> >> --
> >> >> You received this message because you are subscribed to the Google
> >> >> Groups
> >> >> "fuel-core-team" group.
> >> >> To unsubscribe from this group and stop receiving emails from it,
> send
> >> >> an
> >> >> email to fuel-core-team+unsubscribe@xxxxxxxxxxxx.
> >> >> For more options, visit
> >> >> https://groups.google.com/a/mirantis.com/groups/opt_out.
> >> >
> >> >
> >> > --
> >> > You received this message because you are subscribed to the Google
> >> > Groups
> >> > "fuel-core-team" group.
> >> > To unsubscribe from this group and stop receiving emails from it, send
> >> > an
> >> > email to fuel-core-team+unsubscribe@xxxxxxxxxxxx.
> >> > For more options, visit
> >> > https://groups.google.com/a/mirantis.com/groups/opt_out.
> >
> >
> >
> >
> > --
> > Mike Scherbakov
> > #mihgen
>



-- 
Mike Scherbakov
#mihgen