← Back to team overview

fuel-dev team mailing list archive

Re: Issues with RabbitMQ update

 

Vladimir,

There's no pull request since we don't yet have rabbitmq-server 3.2
packages ready (this command won't work with 2.8). It can be run at
any point after starting rabbitmq-server on the primary controller, no
special orchestration for that is needed so it's fine to do it from
puppet.


On Fri, Apr 4, 2014 at 9:10 AM, Vladimir Sharshov
<vsharshov@xxxxxxxxxxxx> wrote:
> Hi all!
>
> Dmitry, when we should run 'rabbitmqctl set_policy HA . '{"ha-mode":
> "all"}'' command? I do not find pull request link for such command.
>
> Lukasz, great work! Now i can confirm that problem in Astute part (Naily).
> For some strange reason publish asynchronous call work with so big delay. At
> now moment i have no ideas why.
>
> Such small fix solve the problem:
>
> -          @exchange.publish(message.to_json, options)
> +          EM.next_tick { @exchange.publish(message.to_json, options) }
>
> I will send all requests to review and build control iso for QA team when
> Dmitry give info about bug/1296922.
>
> On Thu, Apr 3, 2014 at 1:37 AM, Dmitry Borodaenko <dborodaenko@xxxxxxxxxxxx>
> wrote:
>>
>> Please make sure you include a fix for this problem in fuel-library
>> when using RabbitMQ3 with OpenStack:
>> https://bugs.launchpad.net/fuel/+bug/1296922
>>
>> On Wed, Apr 2, 2014 at 6:53 AM, Vladimir Sharshov
>> <vsharshov@xxxxxxxxxxxx> wrote:
>> > First of all: QA says what bug reproduced. Now i try to fix it based on
>> > Lukasz investigations.
>> >
>> > Short how info how to make such iso
>> > (http://172.18.8.252/fuelweb_rabbitmq3-5.0.iso):
>> >
>> > We need this things:
>> >
>> > 1. Patch for fuel-library (https://review.openstack.org/#/c/84707/)
>> > 2. Patch for fuel-main (https://review.openstack.org/#/c/84717/)
>> > 3. Custom RabbitMQ repo:
>> > http://osci-obs.vm.mirantis.net:82/centos-fuel-5.0-stable-14055/centos
>> >
>> > Steps:
>> >
>> > http://docs.mirantis.com/fuel-dev/develop/env.html#building-the-fuel-iso
>> > (steps 1-4)
>> > cd fuel-main
>> > Apply patch for fuel-main:
>> > git fetch https://review.openstack.org/stackforge/fuel-main
>> > refs/changes/17/84717/1 && git cherry-pick FETCH_HEAD
>> > Edit config.mk:
>> > FUELLIB_GERRIT_COMMIT?=refs/changes/07/84707/1
>> > Build iso (in example i use Moscow mirror, but it is not necessary) and
>> > custom RabbitMQ3 package:
>> > make iso USE_MIRROR=msk
>> >
>> > EXTRA_RPM_REPOS="rabbitmq3,http://osci-obs.vm.mirantis.net:82/centos-fuel-5.0-stable-14055/centos";
>> > Check iso in 'build/iso' folder
>> >
>> >
>> >
>> >
>> >
>> > On Wed, Apr 2, 2014 at 1:40 PM, Vladimir Sharshov
>> > <vsharshov@xxxxxxxxxxxx>
>> > wrote:
>> >>
>> >> Long awaited iso: http://172.18.8.252/fuelweb_rabbitmq3-5.0.iso
>> >> (internal
>> >> network only)
>> >>
>> >> Using this iso we can try to repeat problem with progress bar. TTL
>> >> changes
>> >> included.
>> >>
>> >>
>> >> On Mon, Mar 31, 2014 at 3:38 PM, Vladimir Sharshov
>> >> <vsharshov@xxxxxxxxxxxx> wrote:
>> >>>
>> >>> Hi all.
>> >>>
>> >>> Current version of RabbitMQ 3 package have issue which does not allow
>> >>> install it in fresh system. OSCI team now work on it. After it will be
>> >>> done,
>> >>> i will build iso for test.
>> >>>
>> >>> Sorry for delay.
>> >>>
>> >>>
>> >>> On Thu, Mar 27, 2014 at 4:19 PM, Vladimir Sharshov
>> >>> <vsharshov@xxxxxxxxxxxx> wrote:
>> >>>>
>> >>>> Thanks!
>> >>>>
>> >>>> > Do you know easy way to do it?
>> >>>> Yes. I do this soon.
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> On Thu, Mar 27, 2014 at 3:30 PM, Lukasz Oles <loles@xxxxxxxxxxxx>
>> >>>> wrote:
>> >>>>>
>> >>>>> hi, comments inline
>> >>>>>
>> >>>>>
>> >>>>> On Thu, Mar 27, 2014 at 10:50 AM, Vladimir Sharshov
>> >>>>> <vsharshov@xxxxxxxxxxxx> wrote:
>> >>>>>>
>> >>>>>> I try to reproduce problem with new rabbitmq from this email and
>> >>>>>> all
>> >>>>>> works fine without any delay with 7 astute workers.
>> >>>>>>
>> >>>>>> I think we should build ISO with new RabbitMQ and ttl fixes (today
>> >>>>>> i
>> >>>>>> spend time to reinvestigate already solved problem with ttl).
>> >>>>>
>> >>>>> Do you know easy way to do it?
>> >>>>>>
>> >>>>>>
>> >>>>>> Several questions for you:
>> >>>>>>
>> >>>>>> does this problem repeat only in HA-case
>> >>>>>> https://bugs.launchpad.net/fuel/+bug/1278336 ?
>> >>>>>
>> >>>>> No, I'm testing it by adding two nodes to cluster and pressing
>> >>>>> "Verify
>> >>>>> netoworks' in Netowork Tab. I'm waiting 10 min to see results on
>> >>>>> GUI.
>> >>>>>>
>> >>>>>> does problem repeat every time or episodically?
>> >>>>>
>> >>>>> Almost every time. Once it worked, with 2 astute workers, but after
>> >>>>> restart it stopped to work.
>> >>>>>>
>> >>>>>> does this problem affect 'generate diagnostic snapshot'  (it take
>> >>>>>> much
>> >>>>>> longer time)?
>> >>>>>
>> >>>>> I don't know. I tested it on 4.1 branch. I will repeat tests on
>> >>>>> trunk
>> >>>>> version.
>> >>>>>
>> >>>>>>
>> >>>>>> I use simple CentOS cluster (1 controller + 1 compute) and
>> >>>>>> 'generate
>> >>>>>> diagnostic snapshot', because a few month ago in this cases UI have
>> >>>>>> significant delays. As result — all work without delay. Now try
>> >>>>>> with HA.
>> >>>>>>
>> >>>>>>
>> >>>>>> On Tue, Mar 25, 2014 at 3:41 PM, Mike Scherbakov
>> >>>>>> <mscherbakov@xxxxxxxxxxxx> wrote:
>> >>>>>>>
>> >>>>>>> Great findings, Lukasz!
>> >>>>>>>
>> >>>>>>> Adding larger audience of fuel-dev..
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Tue, Mar 25, 2014 at 3:37 PM, Lukasz Oles <loles@xxxxxxxxxxxx>
>> >>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>> Vladimir,
>> >>>>>>>>
>> >>>>>>>> there is no ISO, just install newest rabbitmq. I attached rpm
>> >>>>>>>> package for centos.
>> >>>>>>>>
>> >>>>>>>> I have done some more investigation and number of workers
>> >>>>>>>> actually
>> >>>>>>>> doesn't matter. It just gives random results, but I think I found
>> >>>>>>>> solution.
>> >>>>>>>>
>> >>>>>>>> Naily is using asynchronous library to communicate with Rabbitmq.
>> >>>>>>>> It
>> >>>>>>>> uses amqp library which uses EventMachine. In Naily event loop is
>> >>>>>>>> running in
>> >>>>>>>> main thread but consumer is running in another and publisher in
>> >>>>>>>> yet another
>> >>>>>>>> thread.
>> >>>>>>>>
>> >>>>>>>> To solve the problem with hanging I moved the code for publisher
>> >>>>>>>> and
>> >>>>>>>> consumer to EM::next_tick block. After this everything is working
>> >>>>>>>> again now.
>> >>>>>>>> EM::next_tick does two things. First, it schedules code to run in
>> >>>>>>>> next event loop iteration. Second it runs this code in event loop
>> >>>>>>>> thread.
>> >>>>>>>> I'm not sure which of this things helps.
>> >>>>>>>> Debugging async code in threads is really hard. Why in the first
>> >>>>>>>> place Naily is using async library?
>> >>>>>>>>
>> >>>>>>>> What do you think about it? Maybe it would be better just move to
>> >>>>>>>> synchronous library like bunny?
>> >>>>>>>>
>> >>>>>>>> Please remember I'm not ruby programmer so I can be missing
>> >>>>>>>> something here.
>> >>>>>>>>
>> >>>>>>>> Regards
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On Tue, Mar 25, 2014 at 10:42 AM, Vladimir Sharshov
>> >>>>>>>> <vsharshov@xxxxxxxxxxxx> wrote:
>> >>>>>>>>>
>> >>>>>>>>> Guys, please share link to ISO. Without it i could not say
>> >>>>>>>>> anything
>> >>>>>>>>> useful about potencial problem with naily. Thanks!
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Mon, Mar 24, 2014 at 4:34 PM, Andrey Danin
>> >>>>>>>>> <adanin@xxxxxxxxxxxx>
>> >>>>>>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>> A huge ttl value was set in order to allow nodes with
>> >>>>>>>>>> unsynchronized time be able to use mcollective. if a master
>> >>>>>>>>>> node has a local
>> >>>>>>>>>> time more than 6000 seconds in past in comparison with target
>> >>>>>>>>>> nodes, these
>> >>>>>>>>>> target nodes will not be able to answer via mcollective.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> On Mon, Mar 24, 2014 at 2:07 PM, Vladimir Sharshov
>> >>>>>>>>>> <vsharshov@xxxxxxxxxxxx> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>> Hi all!
>> >>>>>>>>>>>
>> >>>>>>>>>>> > When I changed number of workers in naily from 3 to 2
>> >>>>>>>>>>> > everything started to work
>> >>>>>>>>>>> At now moment we increase this value to 7 as i remember. What
>> >>>>>>>>>>> about shared connection - at now moment it works without any
>> >>>>>>>>>>> issues. Due to
>> >>>>>>>>>>> this limitation only 2 clients in new version looks very
>> >>>>>>>>>>> strange.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Please share link to iso, i try to reproduce and investigate
>> >>>>>>>>>>> this
>> >>>>>>>>>>> problem. Thanks!
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Wed, Mar 19, 2014 at 1:36 PM, Mike Scherbakov
>> >>>>>>>>>>> <mscherbakov@xxxxxxxxxxxx> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Vladimir - I think you've been working with Naily workers,
>> >>>>>>>>>>>> any
>> >>>>>>>>>>>> thoughts on the issue?
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Wed, Mar 19, 2014 at 1:04 PM, Andrey Korolyov
>> >>>>>>>>>>>> <akorolev@xxxxxxxxxxxx> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On 03/19/2014 12:52 PM, Dmitry Pyzhov wrote:
>> >>>>>>>>>>>>> > + more guys.
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> > On Wed, Mar 19, 2014 at 12:10 PM, Lukasz Oles
>> >>>>>>>>>>>>> > <loles@xxxxxxxxxxxx
>> >>>>>>>>>>>>> > <mailto:loles@xxxxxxxxxxxx>> wrote:
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >     Hello guys,
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >     After a lot of testing and debugging finally I have
>> >>>>>>>>>>>>> > something to share.
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >     First change is in mcollective settings. In
>> >>>>>>>>>>>>> >     /etc/mcollective/server.cfg value  ttl = 2000000000 is
>> >>>>>>>>>>>>> > too big.
>> >>>>>>>>>>>>> >     Rabbitmq returns error. I changed it to 6000, without
>> >>>>>>>>>>>>> > this
>> >>>>>>>>>>>>> >     mcollective will not work.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> The problem is not in value itself but in way how rmq
>> >>>>>>>>>>>>> drivers
>> >>>>>>>>>>>>> pushes it.
>> >>>>>>>>>>>>> Somehow it turns as a *concatenation* of default value
>> >>>>>>>>>>>>> around
>> >>>>>>>>>>>>> 10k and
>> >>>>>>>>>>>>> this one, which is definitely too large for first one`
>> >>>>>>>>>>>>> concatenation
>> >>>>>>>>>>>>> result. Just remove this value entirely as I did before from
>> >>>>>>>>>>>>> the config,
>> >>>>>>>>>>>>> three hours are acceptable enough.
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >     Another problem is with task status update. In my
>> >>>>>>>>>>>>> > tests
>> >>>>>>>>>>>>> > it hangs for
>> >>>>>>>>>>>>> >     about 10 minutes. After that task is updated.
>> >>>>>>>>>>>>> > Unfortunately it's not
>> >>>>>>>>>>>>> >     a problem with python but with naily. When I changed
>> >>>>>>>>>>>>> > number of
>> >>>>>>>>>>>>> >     workers in naily from 3 to 2 everything started to
>> >>>>>>>>>>>>> > work.
>> >>>>>>>>>>>>> > I think
>> >>>>>>>>>>>>> >     it's because all thread are using the same connection
>> >>>>>>>>>>>>> > and
>> >>>>>>>>>>>>> > chanel to
>> >>>>>>>>>>>>> >     publish results but I'm still investigating it.
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >     Regards
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >     On Thu, Mar 13, 2014 at 12:38 PM, Dmitry Pyzhov
>> >>>>>>>>>>>>> >     <dpyzhov@xxxxxxxxxxxx <mailto:dpyzhov@xxxxxxxxxxxx>>
>> >>>>>>>>>>>>> > wrote:
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >         Lukasz,
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >         Feel free to contact us if you need anything else.
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >         On Thu, Mar 13, 2014 at 3:22 PM, Lukasz Oles
>> >>>>>>>>>>>>> > <loles@xxxxxxxxxxxx
>> >>>>>>>>>>>>> >         <mailto:loles@xxxxxxxxxxxx>> wrote:
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >             ok, thx for rpm
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >             On Thu, Mar 13, 2014 at 11:40 AM, Dmitry
>> >>>>>>>>>>>>> > Burmistrov
>> >>>>>>>>>>>>> >             <dburmistrov@xxxxxxxxxxxx
>> >>>>>>>>>>>>> > <mailto:dburmistrov@xxxxxxxxxxxx>>
>> >>>>>>>>>>>>> >             wrote:
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >                 Package rabbitmq-server has been built
>> >>>>>>>>>>>>> > from
>> >>>>>>>>>>>>> > changeset:
>> >>>>>>>>>>>>> >                 http://gerrit.mirantis.com/13455
>> >>>>>>>>>>>>> >                 RPM Repository URL:
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> > http:///osci-obs.vm.mirantis.net:82/centos-fuel-5.0-stable-13455/centos
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> > <http://osci-obs.vm.mirantis.net:82/centos-fuel-5.0-stable-13455/centos>
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >                 Package rabbitmq-server has been built
>> >>>>>>>>>>>>> > from
>> >>>>>>>>>>>>> > changeset:
>> >>>>>>>>>>>>> >                 http://gerrit.mirantis.com/13457
>> >>>>>>>>>>>>> >                 DEB Repository URL:
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> > http:///osci-obs.vm.mirantis.net:82/ubuntu-fuel-5.0-stable-13457/ubuntu
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> > <http://osci-obs.vm.mirantis.net:82/ubuntu-fuel-5.0-stable-13457/ubuntu>
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >                 On Thu, Mar 13, 2014 at 2:11 PM, Dmitry
>> >>>>>>>>>>>>> > Pyzhov
>> >>>>>>>>>>>>> >                 <dpyzhov@xxxxxxxxxxxx
>> >>>>>>>>>>>>> > <mailto:dpyzhov@xxxxxxxxxxxx>> wrote:
>> >>>>>>>>>>>>> >                 > Lukasz,
>> >>>>>>>>>>>>> >                 >
>> >>>>>>>>>>>>> >                 > Sorry for long response. Our OSCI team
>> >>>>>>>>>>>>> > will
>> >>>>>>>>>>>>> > build a
>> >>>>>>>>>>>>> >                 package. Dmitry B, could
>> >>>>>>>>>>>>> >                 > you reply with the download link when it
>> >>>>>>>>>>>>> > is
>> >>>>>>>>>>>>> > available?
>> >>>>>>>>>>>>> >                 Ticket OSCI-1016.
>> >>>>>>>>>>>>> >                 >
>> >>>>>>>>>>>>> >                 >
>> >>>>>>>>>>>>> >                 > On Wed, Mar 12, 2014 at 2:18 PM, Lukasz
>> >>>>>>>>>>>>> > Oles
>> >>>>>>>>>>>>> >                 <loles@xxxxxxxxxxxx
>> >>>>>>>>>>>>> > <mailto:loles@xxxxxxxxxxxx>> wrote:
>> >>>>>>>>>>>>> >                 >>
>> >>>>>>>>>>>>> >                 >> Sure, I will look into it. Can you give
>> >>>>>>>>>>>>> > me
>> >>>>>>>>>>>>> > a link to
>> >>>>>>>>>>>>> >                 rabbitmq rpm which
>> >>>>>>>>>>>>> >                 >> you used?
>> >>>>>>>>>>>>> >                 >>
>> >>>>>>>>>>>>> >                 >> Regards,
>> >>>>>>>>>>>>> >                 >>
>> >>>>>>>>>>>>> >                 >>
>> >>>>>>>>>>>>> >                 >> On Wed, Mar 12, 2014 at 11:10 AM,
>> >>>>>>>>>>>>> > Dmitry
>> >>>>>>>>>>>>> > Pyzhov
>> >>>>>>>>>>>>> >                 <dpyzhov@xxxxxxxxxxxx
>> >>>>>>>>>>>>> > <mailto:dpyzhov@xxxxxxxxxxxx>>
>> >>>>>>>>>>>>> >                 >> wrote:
>> >>>>>>>>>>>>> >                 >>>
>> >>>>>>>>>>>>> >                 >>> Lukasz,
>> >>>>>>>>>>>>> >                 >>>
>> >>>>>>>>>>>>> >                 >>> actually we have no idea what is wrong
>> >>>>>>>>>>>>> > with fresh
>> >>>>>>>>>>>>> >                 rabbitmq. For some
>> >>>>>>>>>>>>> >                 >>> reason refresh of task status takes
>> >>>>>>>>>>>>> > too
>> >>>>>>>>>>>>> > much time.
>> >>>>>>>>>>>>> >                 Dmitry tried to find the
>> >>>>>>>>>>>>> >                 >>> root cause, but did not succeed. Could
>> >>>>>>>>>>>>> > you
>> >>>>>>>>>>>>> >                 investigate the issue?
>> >>>>>>>>>>>>> >                 >>>
>> >>>>>>>>>>>>> >                 >>>
>> >>>>>>>>>>>>> >                 >>> On Tue, Mar 11, 2014 at 9:49 PM,
>> >>>>>>>>>>>>> > Lukasz
>> >>>>>>>>>>>>> > Oles
>> >>>>>>>>>>>>> >                 <loles@xxxxxxxxxxxx
>> >>>>>>>>>>>>> > <mailto:loles@xxxxxxxxxxxx>> wrote:
>> >>>>>>>>>>>>> >                 >>>>
>> >>>>>>>>>>>>> >                 >>>> Dmitry,
>> >>>>>>>>>>>>> >                 >>>>
>> >>>>>>>>>>>>> >                 >>>> rabbitmq update looks interesting, I
>> >>>>>>>>>>>>> > can
>> >>>>>>>>>>>>> > look into
>> >>>>>>>>>>>>> >                 it. Do I need any
>> >>>>>>>>>>>>> >                 >>>> additional information?
>> >>>>>>>>>>>>> >                 >>>>
>> >>>>>>>>>>>>> >                 >>>> regards,
>> >>>>>>>>>>>>> >                 >>>>
>> >>>>>>>>>>>>> >                 >>>>
>> >>>>>>>>>>>>> >                 >>>> On Tue, Mar 11, 2014 at 2:05 PM,
>> >>>>>>>>>>>>> > Dmitry
>> >>>>>>>>>>>>> > Pyzhov
>> >>>>>>>>>>>>> >                 <dpyzhov@xxxxxxxxxxxx
>> >>>>>>>>>>>>> > <mailto:dpyzhov@xxxxxxxxxxxx>>
>> >>>>>>>>>>>>> >                 >>>> wrote:
>> >>>>>>>>>>>>> >                 >>>>>
>> >>>>>>>>>>>>> >                 >>>>> Great!
>> >>>>>>>>>>>>> >                 >>>>>
>> >>>>>>>>>>>>> >                 >>>>> Lukasz, could you help us with
>> >>>>>>>>>>>>> > rabbitmq
>> >>>>>>>>>>>>> > update? We
>> >>>>>>>>>>>>> >                 faced an issue with
>> >>>>>>>>>>>>> >                 >>>>> it:
>> >>>>>>>>>>>>> > https://bugs.launchpad.net/fuel/+bug/1278336
>> >>>>>>>>>>>>> >                 >>>>>
>> >>>>>>>>>>>>> >                 >>>>> Also, could you participate it
>> >>>>>>>>>>>>> > design
>> >>>>>>>>>>>>> > review:
>> >>>>>>>>>>>>> >                 >>>>>
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> > https://docs.google.com/document/d/1zqV58LZBLQ-0gllb_i3MyIKIMj-Qx8ELJohjcWs459s/edit?usp=sharing
>> >>>>>>>>>>>>> >                 >>>>>
>> >>>>>>>>>>>>> >                 >>>>>
>> >>>>>>>>>>>>> >                 >>>>> On Mon, Mar 10, 2014 at 9:01 PM,
>> >>>>>>>>>>>>> > Mike
>> >>>>>>>>>>>>> > Scherbakov
>> >>>>>>>>>>>>> >                 >>>>> <mscherbakov@xxxxxxxxxxxx
>> >>>>>>>>>>>>> >                 <mailto:mscherbakov@xxxxxxxxxxxx>> wrote:
>> >>>>>>>>>>>>> >                 >>>>>>
>> >>>>>>>>>>>>> >                 >>>>>> Lukasz,
>> >>>>>>>>>>>>> >                 >>>>>> please take any bugs from
>> >>>>>>>>>>>>> >                 https://launchpad.net/fuel/+milestone/5.0
>> >>>>>>>>>>>>> >                 >>>>>> which are not assigned to
>> >>>>>>>>>>>>> > particular
>> >>>>>>>>>>>>> > person. You
>> >>>>>>>>>>>>> >                 are likely to be interested
>> >>>>>>>>>>>>> >                 >>>>>> in those which are assigned to
>> >>>>>>>>>>>>> > "fuel-python". Of
>> >>>>>>>>>>>>> >                 course, it's preferred to
>> >>>>>>>>>>>>> >                 >>>>>> work on Critical and High priority
>> >>>>>>>>>>>>> > bugs in a
>> >>>>>>>>>>>>> >                 first order.
>> >>>>>>>>>>>>> >                 >>>>>>
>> >>>>>>>>>>>>> >                 >>>>>> We are in a design phase for 5.0.
>> >>>>>>>>>>>>> > Please take a
>> >>>>>>>>>>>>> >                 look at
>> >>>>>>>>>>>>> >                 >>>>>>
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> > https://mirantis.jira.com/wiki/display/PRD/5.0+-+Mirantis+OpenStack+release+home+page.
>> >>>>>>>>>>>>> >                 >>>>>> I'm discussing this still with
>> >>>>>>>>>>>>> > management, and we
>> >>>>>>>>>>>>> >                 will likely have only part
>> >>>>>>>>>>>>> >                 >>>>>> of what is on the page. Your
>> >>>>>>>>>>>>> > comments
>> >>>>>>>>>>>>> > and input
>> >>>>>>>>>>>>> >                 into design docs (which you
>> >>>>>>>>>>>>> >                 >>>>>> can find following blueprint link,
>> >>>>>>>>>>>>> > then "Read the
>> >>>>>>>>>>>>> >                 full spec") is very
>> >>>>>>>>>>>>> >                 >>>>>> welcome.
>> >>>>>>>>>>>>> >                 >>>>>>
>> >>>>>>>>>>>>> >                 >>>>>> Start looking over and try to
>> >>>>>>>>>>>>> > identify
>> >>>>>>>>>>>>> > spot which
>> >>>>>>>>>>>>> >                 is in most interest
>> >>>>>>>>>>>>> >                 >>>>>> of you. Dmitry/Evgeny will help to
>> >>>>>>>>>>>>> > identify areas
>> >>>>>>>>>>>>> >                 where help is mostly
>> >>>>>>>>>>>>> >                 >>>>>> needed. Sorry for not responding to
>> >>>>>>>>>>>>> > you in time.
>> >>>>>>>>>>>>> >                 I'll get my team to fix
>> >>>>>>>>>>>>> >                 >>>>>> this.
>> >>>>>>>>>>>>> >                 >>>>>>
>> >>>>>>>>>>>>> >                 >>>>>> FYI: Today is holiday in Russia &
>> >>>>>>>>>>>>> > Ukraine
>> >>>>>>>>>>>>> >                 >>>>>> Thanks,
>> >>>>>>>>>>>>> >                 >>>>>>
>> >>>>>>>>>>>>> >                 >>>>>>
>> >>>>>>>>>>>>> >                 >>>>>> On Thu, Mar 6, 2014 at 12:37 AM,
>> >>>>>>>>>>>>> > Mike
>> >>>>>>>>>>>>> > Scherbakov
>> >>>>>>>>>>>>> >                 >>>>>> <mscherbakov@xxxxxxxxxxxx
>> >>>>>>>>>>>>> >                 <mailto:mscherbakov@xxxxxxxxxxxx>> wrote:
>> >>>>>>>>>>>>> >                 >>>>>>>
>> >>>>>>>>>>>>> >                 >>>>>>> It's great. I would be happy to
>> >>>>>>>>>>>>> > see
>> >>>>>>>>>>>>> > Lukasz
>> >>>>>>>>>>>>> >                 working on Fuel.
>> >>>>>>>>>>>>> >                 >>>>>>> Actually, Lukasz already doing
>> >>>>>>>>>>>>> > great
>> >>>>>>>>>>>>> > job helping
>> >>>>>>>>>>>>> >                 us with Nailgun
>> >>>>>>>>>>>>> >                 >>>>>>> scalability issues resolution.
>> >>>>>>>>>>>>> >                 >>>>>>>
>> >>>>>>>>>>>>> >                 >>>>>>> Dmitry, please arrange meeting
>> >>>>>>>>>>>>> > between our
>> >>>>>>>>>>>>> >                 Python engineers and
>> >>>>>>>>>>>>> >                 >>>>>>> Lukasz, and identify areas where
>> >>>>>>>>>>>>> > contribution of
>> >>>>>>>>>>>>> >                 Lukasz will be the most
>> >>>>>>>>>>>>> >                 >>>>>>> effective. It should be aligned
>> >>>>>>>>>>>>> > with
>> >>>>>>>>>>>>> > the
>> >>>>>>>>>>>>> >                 development of our engineers too.
>> >>>>>>>>>>>>> >                 >>>>>>>
>> >>>>>>>>>>>>> >                 >>>>>>> Thanks,
>> >>>>>>>>>>>>> >                 >>>>>>>
>> >>>>>>>>>>>>> >                 >>>>>>>
>> >>>>>>>>>>>>> >                 >>>>>>> On Tue, Mar 4, 2014 at 3:27 PM,
>> >>>>>>>>>>>>> > Piotr
>> >>>>>>>>>>>>> > Siwczak
>> >>>>>>>>>>>>> >                 <psiwczak@xxxxxxxxxxxx
>> >>>>>>>>>>>>> > <mailto:psiwczak@xxxxxxxxxxxx>>
>> >>>>>>>>>>>>> >                 >>>>>>> wrote:
>> >>>>>>>>>>>>> >                 >>>>>>>>
>> >>>>>>>>>>>>> >                 >>>>>>>> Mike,
>> >>>>>>>>>>>>> >                 >>>>>>>>
>> >>>>>>>>>>>>> >                 >>>>>>>> For now I see Lukasz has finished
>> >>>>>>>>>>>>> > his work for
>> >>>>>>>>>>>>> >                 Softlayer/Express (at
>> >>>>>>>>>>>>> >                 >>>>>>>> least for the next few weeks)and
>> >>>>>>>>>>>>> > can
>> >>>>>>>>>>>>> > use his
>> >>>>>>>>>>>>> >                 time to engage into Fuel
>> >>>>>>>>>>>>> >                 >>>>>>>> development. Please feel free to
>> >>>>>>>>>>>>> > assign him to
>> >>>>>>>>>>>>> >                 Fuel tasks.
>> >>>>>>>>>>>>> >                 >>>>>>>>
>> >>>>>>>>>>>>> >                 >>>>>>>> -Piotr
>> >>>>>>>>>>>>> >                 >>>>>>>
>> >>>>>>>>>>>>> >                 >>>>>>>
>> >>>>>>>>>>>>> >                 >>>>>>>
>> >>>>>>>>>>>>> >                 >>>>>>>
>> >>>>>>>>>>>>> >                 >>>>>>> --
>> >>>>>>>>>>>>> >                 >>>>>>> Mike Scherbakov
>> >>>>>>>>>>>>> >                 >>>>>>> #mihgen
>> >>>>>>>>>>>>> >                 >>>>>>
>> >>>>>>>>>>>>> >                 >>>>>>
>> >>>>>>>>>>>>> >                 >>>>>>
>> >>>>>>>>>>>>> >                 >>>>>>
>> >>>>>>>>>>>>> >                 >>>>>> --
>> >>>>>>>>>>>>> >                 >>>>>> Mike Scherbakov
>> >>>>>>>>>>>>> >                 >>>>>> #mihgen
>> >>>>>>>>>>>>> >                 >>>>>
>> >>>>>>>>>>>>> >                 >>>>>
>> >>>>>>>>>>>>> >                 >>>>
>> >>>>>>>>>>>>> >                 >>>>
>> >>>>>>>>>>>>> >                 >>>>
>> >>>>>>>>>>>>> >                 >>>> --
>> >>>>>>>>>>>>> >                 >>>> Łukasz Oleś
>> >>>>>>>>>>>>> >                 >>>
>> >>>>>>>>>>>>> >                 >>>
>> >>>>>>>>>>>>> >                 >>
>> >>>>>>>>>>>>> >                 >>
>> >>>>>>>>>>>>> >                 >>
>> >>>>>>>>>>>>> >                 >> --
>> >>>>>>>>>>>>> >                 >> Łukasz Oleś
>> >>>>>>>>>>>>> >                 >
>> >>>>>>>>>>>>> >                 >
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >             --
>> >>>>>>>>>>>>> >             Łukasz Oleś
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >     --
>> >>>>>>>>>>>>> >     Łukasz Oleś
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> >
>> >>>>>>>>>>>>> > --
>> >>>>>>>>>>>>> > You received this message because you are subscribed to
>> >>>>>>>>>>>>> > the
>> >>>>>>>>>>>>> > Google
>> >>>>>>>>>>>>> > Groups "fuel-core-team" group.
>> >>>>>>>>>>>>> > To unsubscribe from this group and stop receiving emails
>> >>>>>>>>>>>>> > from
>> >>>>>>>>>>>>> > it, send
>> >>>>>>>>>>>>> > an email to fuel-core-team+unsubscribe@xxxxxxxxxxxx
>> >>>>>>>>>>>>> > <mailto:fuel-core-team+unsubscribe@xxxxxxxxxxxx>.
>> >>>>>>>>>>>>> > For more options, visit
>> >>>>>>>>>>>>> > https://groups.google.com/a/mirantis.com/d/optout.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> --
>> >>>>>>>>>>>>> You received this message because you are subscribed to the
>> >>>>>>>>>>>>> Google Groups "fuel-core-team" group.
>> >>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails
>> >>>>>>>>>>>>> from
>> >>>>>>>>>>>>> it, send an email to
>> >>>>>>>>>>>>> fuel-core-team+unsubscribe@xxxxxxxxxxxx.
>> >>>>>>>>>>>>> For more options, visit
>> >>>>>>>>>>>>> https://groups.google.com/a/mirantis.com/d/optout.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> --
>> >>>>>>>>>>>> Mike Scherbakov
>> >>>>>>>>>>>> #mihgen
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> --
>> >>>>>>>>>>> You received this message because you are subscribed to the
>> >>>>>>>>>>> Google Groups "fuel-core-team" group.
>> >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from
>> >>>>>>>>>>> it,
>> >>>>>>>>>>> send an email to fuel-core-team+unsubscribe@xxxxxxxxxxxx.
>> >>>>>>>>>>> For more options, visit
>> >>>>>>>>>>> https://groups.google.com/a/mirantis.com/d/optout.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> --
>> >>>>>>>>>> Andrey Danin
>> >>>>>>>>>> adanin@xxxxxxxxxxxx
>> >>>>>>>>>> skype: gcon.monolake
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> --
>> >>>>>>>> Łukasz Oleś
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> Mike Scherbakov
>> >>>>>>> #mihgen
>> >>>>>>
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> Łukasz Oleś
>> >>>>
>> >>>>
>> >>>
>> >>
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "fuel-core-team" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an
>> > email to fuel-core-team+unsubscribe@xxxxxxxxxxxx.
>> > For more options, visit
>> > https://groups.google.com/a/mirantis.com/d/optout.
>>
>>
>>
>> --
>> Dmitry Borodaenko
>
>



-- 
Dmitry Borodaenko


Follow ups

References