← Back to team overview

fuel-dev team mailing list archive

Re: Issues with RabbitMQ update

 

Update. Please review and accept.
https://review.openstack.org/#/c/84707/
https://review.openstack.org/#/c/84717/


On Thu, Apr 10, 2014 at 11:06 AM, Mike Scherbakov
<mscherbakov@xxxxxxxxxxxx>wrote:

> on #2, there is -1 from Dmitry B... why it is so?
>
>
> On Thu, Apr 10, 2014 at 10:59 AM, Vladimir Sharshov <
> vsharshov@xxxxxxxxxxxx> wrote:
>
>> Step to merge:
>>
>> 1. https://review.openstack.org/#/c/86366/2 (can be safely merge)
>> 2. 3. TTL fixs: https://review.openstack.org/#/c/84707/ &&
>> https://review.openstack.org/#/c/84717/ (can be safely merge)
>> 3. Vladimir Kuklin backport patch from Andrey's old pull: 'rabbitmqctl
>> set_policy HA . '{"ha-mode": "all"}'
>> 4. OSCI: publish RabbitMQ3 package to repository.
>>
>>
>> On Wed, Apr 9, 2014 at 7:30 PM, Lukasz Oles <loles@xxxxxxxxxxxx> wrote:
>>
>>> Gerrit version https://review.openstack.org/#/c/86366/2
>>>
>>>
>>> On Tue, Apr 8, 2014 at 8:40 AM, Vladimir Kuklin <vkuklin@xxxxxxxxxxxx>wrote:
>>>
>>>> Guys
>>>>
>>>> It would be awesome if this change was somehow introduced into our
>>>> gerrit instead of being posted in the ML.
>>>>
>>>>
>>>> On Tue, Apr 8, 2014 at 12:32 PM, Lukasz Oles <loles@xxxxxxxxxxxx>wrote:
>>>>
>>>>> I finally was able to download and test ISO. I'm glad that you also
>>>>> were able to reproduce this bug.
>>>>> I think you should include whole begine/rescue block in EM.next_tick
>>>>> function. This code block will be executed in different thread and if there
>>>>> will be exception, nothing will catch it. Something like this should work:
>>>>>
>>>>> --- a/lib/astute/server/producer.rb
>>>>> +++ b/lib/astute/server/producer.rb
>>>>> @@ -25,11 +25,13 @@ module Astute
>>>>>                             :content_type => 'application/json'}
>>>>>          options = default_options.merge(options)
>>>>>
>>>>> -        begin
>>>>> -          @exchange.publish(message.to_json, options)
>>>>> -        rescue
>>>>> -          Astute.logger.error "Error publishing message: #{$!}"
>>>>> -        end
>>>>> +        EM.next_tick {
>>>>> +          begin
>>>>> +            @exchange.publish(message.to_json, options)
>>>>> +          rescue
>>>>> +            Astute.logger.error "Error publishing message: #{$!}"
>>>>> +          end
>>>>> +        }
>>>>>        end
>>>>>      end
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Apr 4, 2014 at 4:10 PM, Vladimir Sharshov <
>>>>> vsharshov@xxxxxxxxxxxx> wrote:
>>>>>
>>>>>> Hi all!
>>>>>>
>>>>>> Dmitry, when we should run 'rabbitmqctl set_policy HA . '{"ha-mode":
>>>>>> "all"}'' command? I do not find pull request link for such command.
>>>>>>
>>>>>> Lukasz, great work! Now i can confirm that problem in Astute part
>>>>>> (Naily). For some strange reason publish asynchronous call work with so big
>>>>>> delay. At now moment i have no ideas why.
>>>>>>
>>>>>> Such small fix solve the problem:
>>>>>>
>>>>>> -          @exchange.publish(message.to_json, options)
>>>>>> +          EM.next_tick { @exchange.publish(message.to_json, options)
>>>>>> }
>>>>>>
>>>>>> I will send all requests to review and build control iso for QA team
>>>>>> when Dmitry give info about bug/1296922<https://bugs.launchpad.net/fuel/+bug/1296922>
>>>>>> .
>>>>>>
>>>>>> On Thu, Apr 3, 2014 at 1:37 AM, Dmitry Borodaenko <
>>>>>> dborodaenko@xxxxxxxxxxxx> wrote:
>>>>>>
>>>>>>> Please make sure you include a fix for this problem in fuel-library
>>>>>>> when using RabbitMQ3 with OpenStack:
>>>>>>> https://bugs.launchpad.net/fuel/+bug/1296922
>>>>>>>
>>>>>>> On Wed, Apr 2, 2014 at 6:53 AM, Vladimir Sharshov
>>>>>>> <vsharshov@xxxxxxxxxxxx> wrote:
>>>>>>> > First of all: QA says what bug reproduced. Now i try to fix it
>>>>>>> based on
>>>>>>> > Lukasz investigations.
>>>>>>> >
>>>>>>> > Short how info how to make such iso
>>>>>>> > (http://172.18.8.252/fuelweb_rabbitmq3-5.0.iso):
>>>>>>> >
>>>>>>> > We need this things:
>>>>>>> >
>>>>>>> > 1. Patch for fuel-library (https://review.openstack.org/#/c/84707/
>>>>>>> )
>>>>>>> > 2. Patch for fuel-main (https://review.openstack.org/#/c/84717/)
>>>>>>> > 3. Custom RabbitMQ repo:
>>>>>>> >
>>>>>>> http://osci-obs.vm.mirantis.net:82/centos-fuel-5.0-stable-14055/centos
>>>>>>> >
>>>>>>> > Steps:
>>>>>>> >
>>>>>>> >
>>>>>>> http://docs.mirantis.com/fuel-dev/develop/env.html#building-the-fuel-iso
>>>>>>> > (steps 1-4)
>>>>>>> > cd fuel-main
>>>>>>> > Apply patch for fuel-main:
>>>>>>> > git fetch https://review.openstack.org/stackforge/fuel-main
>>>>>>> > refs/changes/17/84717/1 && git cherry-pick FETCH_HEAD
>>>>>>> > Edit config.mk:
>>>>>>> > FUELLIB_GERRIT_COMMIT?=refs/changes/07/84707/1
>>>>>>> > Build iso (in example i use Moscow mirror, but it is not
>>>>>>> necessary) and
>>>>>>> > custom RabbitMQ3 package:
>>>>>>> > make iso USE_MIRROR=msk
>>>>>>> > EXTRA_RPM_REPOS="rabbitmq3,
>>>>>>> http://osci-obs.vm.mirantis.net:82/centos-fuel-5.0-stable-14055/centos
>>>>>>> "
>>>>>>> > Check iso in 'build/iso' folder
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> > On Wed, Apr 2, 2014 at 1:40 PM, Vladimir Sharshov <
>>>>>>> vsharshov@xxxxxxxxxxxx>
>>>>>>> > wrote:
>>>>>>> >>
>>>>>>> >> Long awaited iso: http://172.18.8.252/fuelweb_rabbitmq3-5.0.iso(internal
>>>>>>> >> network only)
>>>>>>> >>
>>>>>>> >> Using this iso we can try to repeat problem with progress bar.
>>>>>>> TTL changes
>>>>>>> >> included.
>>>>>>> >>
>>>>>>> >>
>>>>>>> >> On Mon, Mar 31, 2014 at 3:38 PM, Vladimir Sharshov
>>>>>>> >> <vsharshov@xxxxxxxxxxxx> wrote:
>>>>>>> >>>
>>>>>>> >>> Hi all.
>>>>>>> >>>
>>>>>>> >>> Current version of RabbitMQ 3 package have issue which does not
>>>>>>> allow
>>>>>>> >>> install it in fresh system. OSCI team now work on it. After it
>>>>>>> will be done,
>>>>>>> >>> i will build iso for test.
>>>>>>> >>>
>>>>>>> >>> Sorry for delay.
>>>>>>> >>>
>>>>>>> >>>
>>>>>>> >>> On Thu, Mar 27, 2014 at 4:19 PM, Vladimir Sharshov
>>>>>>> >>> <vsharshov@xxxxxxxxxxxx> wrote:
>>>>>>> >>>>
>>>>>>> >>>> Thanks!
>>>>>>> >>>>
>>>>>>> >>>> > Do you know easy way to do it?
>>>>>>> >>>> Yes. I do this soon.
>>>>>>> >>>>
>>>>>>> >>>>
>>>>>>> >>>>
>>>>>>> >>>>
>>>>>>> >>>> On Thu, Mar 27, 2014 at 3:30 PM, Lukasz Oles <
>>>>>>> loles@xxxxxxxxxxxx> wrote:
>>>>>>> >>>>>
>>>>>>> >>>>> hi, comments inline
>>>>>>> >>>>>
>>>>>>> >>>>>
>>>>>>> >>>>> On Thu, Mar 27, 2014 at 10:50 AM, Vladimir Sharshov
>>>>>>> >>>>> <vsharshov@xxxxxxxxxxxx> wrote:
>>>>>>> >>>>>>
>>>>>>> >>>>>> I try to reproduce problem with new rabbitmq from this email
>>>>>>> and all
>>>>>>> >>>>>> works fine without any delay with 7 astute workers.
>>>>>>> >>>>>>
>>>>>>> >>>>>> I think we should build ISO with new RabbitMQ and ttl fixes
>>>>>>> (today i
>>>>>>> >>>>>> spend time to reinvestigate already solved problem with ttl).
>>>>>>> >>>>>
>>>>>>> >>>>> Do you know easy way to do it?
>>>>>>> >>>>>>
>>>>>>> >>>>>>
>>>>>>> >>>>>> Several questions for you:
>>>>>>> >>>>>>
>>>>>>> >>>>>> does this problem repeat only in HA-case
>>>>>>> >>>>>> https://bugs.launchpad.net/fuel/+bug/1278336 ?
>>>>>>> >>>>>
>>>>>>> >>>>> No, I'm testing it by adding two nodes to cluster and pressing
>>>>>>> "Verify
>>>>>>> >>>>> netoworks' in Netowork Tab. I'm waiting 10 min to see results
>>>>>>> on GUI.
>>>>>>> >>>>>>
>>>>>>> >>>>>> does problem repeat every time or episodically?
>>>>>>> >>>>>
>>>>>>> >>>>> Almost every time. Once it worked, with 2 astute workers, but
>>>>>>> after
>>>>>>> >>>>> restart it stopped to work.
>>>>>>> >>>>>>
>>>>>>> >>>>>> does this problem affect 'generate diagnostic snapshot'  (it
>>>>>>> take much
>>>>>>> >>>>>> longer time)?
>>>>>>> >>>>>
>>>>>>> >>>>> I don't know. I tested it on 4.1 branch. I will repeat tests
>>>>>>> on trunk
>>>>>>> >>>>> version.
>>>>>>> >>>>>
>>>>>>> >>>>>>
>>>>>>> >>>>>> I use simple CentOS cluster (1 controller + 1 compute) and
>>>>>>> 'generate
>>>>>>> >>>>>> diagnostic snapshot', because a few month ago in this cases
>>>>>>> UI have
>>>>>>> >>>>>> significant delays. As result — all work without delay. Now
>>>>>>> try with HA.
>>>>>>> >>>>>>
>>>>>>> >>>>>>
>>>>>>> >>>>>> On Tue, Mar 25, 2014 at 3:41 PM, Mike Scherbakov
>>>>>>> >>>>>> <mscherbakov@xxxxxxxxxxxx> wrote:
>>>>>>> >>>>>>>
>>>>>>> >>>>>>> Great findings, Lukasz!
>>>>>>> >>>>>>>
>>>>>>> >>>>>>> Adding larger audience of fuel-dev..
>>>>>>> >>>>>>>
>>>>>>> >>>>>>>
>>>>>>> >>>>>>> On Tue, Mar 25, 2014 at 3:37 PM, Lukasz Oles <
>>>>>>> loles@xxxxxxxxxxxx>
>>>>>>> >>>>>>> wrote:
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>> Vladimir,
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>> there is no ISO, just install newest rabbitmq. I attached
>>>>>>> rpm
>>>>>>> >>>>>>>> package for centos.
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>> I have done some more investigation and number of workers
>>>>>>> actually
>>>>>>> >>>>>>>> doesn't matter. It just gives random results, but I think I
>>>>>>> found solution.
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>> Naily is using asynchronous library to communicate with
>>>>>>> Rabbitmq. It
>>>>>>> >>>>>>>> uses amqp library which uses EventMachine. In Naily event
>>>>>>> loop is running in
>>>>>>> >>>>>>>> main thread but consumer is running in another and
>>>>>>> publisher in yet another
>>>>>>> >>>>>>>> thread.
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>> To solve the problem with hanging I moved the code for
>>>>>>> publisher and
>>>>>>> >>>>>>>> consumer to EM::next_tick block. After this everything is
>>>>>>> working again now.
>>>>>>> >>>>>>>> EM::next_tick does two things. First, it schedules code to
>>>>>>> run in
>>>>>>> >>>>>>>> next event loop iteration. Second it runs this code in
>>>>>>> event loop thread.
>>>>>>> >>>>>>>> I'm not sure which of this things helps.
>>>>>>> >>>>>>>> Debugging async code in threads is really hard. Why in the
>>>>>>> first
>>>>>>> >>>>>>>> place Naily is using async library?
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>> What do you think about it? Maybe it would be better just
>>>>>>> move to
>>>>>>> >>>>>>>> synchronous library like bunny?
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>> Please remember I'm not ruby programmer so I can be missing
>>>>>>> >>>>>>>> something here.
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>> Regards
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>> On Tue, Mar 25, 2014 at 10:42 AM, Vladimir Sharshov
>>>>>>> >>>>>>>> <vsharshov@xxxxxxxxxxxx> wrote:
>>>>>>> >>>>>>>>>
>>>>>>> >>>>>>>>> Guys, please share link to ISO. Without it i could not say
>>>>>>> anything
>>>>>>> >>>>>>>>> useful about potencial problem with naily. Thanks!
>>>>>>> >>>>>>>>>
>>>>>>> >>>>>>>>>
>>>>>>> >>>>>>>>> On Mon, Mar 24, 2014 at 4:34 PM, Andrey Danin <
>>>>>>> adanin@xxxxxxxxxxxx>
>>>>>>> >>>>>>>>> wrote:
>>>>>>> >>>>>>>>>>
>>>>>>> >>>>>>>>>> A huge ttl value was set in order to allow nodes with
>>>>>>> >>>>>>>>>> unsynchronized time be able to use mcollective. if a
>>>>>>> master node has a local
>>>>>>> >>>>>>>>>> time more than 6000 seconds in past in comparison with
>>>>>>> target nodes, these
>>>>>>> >>>>>>>>>> target nodes will not be able to answer via mcollective.
>>>>>>> >>>>>>>>>>
>>>>>>> >>>>>>>>>>
>>>>>>> >>>>>>>>>> On Mon, Mar 24, 2014 at 2:07 PM, Vladimir Sharshov
>>>>>>> >>>>>>>>>> <vsharshov@xxxxxxxxxxxx> wrote:
>>>>>>> >>>>>>>>>>>
>>>>>>> >>>>>>>>>>> Hi all!
>>>>>>> >>>>>>>>>>>
>>>>>>> >>>>>>>>>>> > When I changed number of workers in naily from 3 to 2
>>>>>>> >>>>>>>>>>> > everything started to work
>>>>>>> >>>>>>>>>>> At now moment we increase this value to 7 as i remember.
>>>>>>> What
>>>>>>> >>>>>>>>>>> about shared connection - at now moment it works without
>>>>>>> any issues. Due to
>>>>>>> >>>>>>>>>>> this limitation only 2 clients in new version looks very
>>>>>>> strange.
>>>>>>> >>>>>>>>>>>
>>>>>>> >>>>>>>>>>> Please share link to iso, i try to reproduce and
>>>>>>> investigate this
>>>>>>> >>>>>>>>>>> problem. Thanks!
>>>>>>> >>>>>>>>>>>
>>>>>>> >>>>>>>>>>>
>>>>>>> >>>>>>>>>>> On Wed, Mar 19, 2014 at 1:36 PM, Mike Scherbakov
>>>>>>> >>>>>>>>>>> <mscherbakov@xxxxxxxxxxxx> wrote:
>>>>>>> >>>>>>>>>>>>
>>>>>>> >>>>>>>>>>>> Vladimir - I think you've been working with Naily
>>>>>>> workers, any
>>>>>>> >>>>>>>>>>>> thoughts on the issue?
>>>>>>> >>>>>>>>>>>>
>>>>>>> >>>>>>>>>>>>
>>>>>>> >>>>>>>>>>>> On Wed, Mar 19, 2014 at 1:04 PM, Andrey Korolyov
>>>>>>> >>>>>>>>>>>> <akorolev@xxxxxxxxxxxx> wrote:
>>>>>>> >>>>>>>>>>>>>
>>>>>>> >>>>>>>>>>>>> On 03/19/2014 12:52 PM, Dmitry Pyzhov wrote:
>>>>>>> >>>>>>>>>>>>> > + more guys.
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> > On Wed, Mar 19, 2014 at 12:10 PM, Lukasz Oles
>>>>>>> >>>>>>>>>>>>> > <loles@xxxxxxxxxxxx
>>>>>>> >>>>>>>>>>>>> > <mailto:loles@xxxxxxxxxxxx>> wrote:
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >     Hello guys,
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >     After a lot of testing and debugging finally I
>>>>>>> have
>>>>>>> >>>>>>>>>>>>> > something to share.
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >     First change is in mcollective settings. In
>>>>>>> >>>>>>>>>>>>> >     /etc/mcollective/server.cfg value  ttl =
>>>>>>> 2000000000 is
>>>>>>> >>>>>>>>>>>>> > too big.
>>>>>>> >>>>>>>>>>>>> >     Rabbitmq returns error. I changed it to 6000,
>>>>>>> without
>>>>>>> >>>>>>>>>>>>> > this
>>>>>>> >>>>>>>>>>>>> >     mcollective will not work.
>>>>>>> >>>>>>>>>>>>>
>>>>>>> >>>>>>>>>>>>> The problem is not in value itself but in way how rmq
>>>>>>> drivers
>>>>>>> >>>>>>>>>>>>> pushes it.
>>>>>>> >>>>>>>>>>>>> Somehow it turns as a *concatenation* of default value
>>>>>>> around
>>>>>>> >>>>>>>>>>>>> 10k and
>>>>>>> >>>>>>>>>>>>> this one, which is definitely too large for first one`
>>>>>>> >>>>>>>>>>>>> concatenation
>>>>>>> >>>>>>>>>>>>> result. Just remove this value entirely as I did
>>>>>>> before from
>>>>>>> >>>>>>>>>>>>> the config,
>>>>>>> >>>>>>>>>>>>> three hours are acceptable enough.
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >     Another problem is with task status update. In
>>>>>>> my tests
>>>>>>> >>>>>>>>>>>>> > it hangs for
>>>>>>> >>>>>>>>>>>>> >     about 10 minutes. After that task is updated.
>>>>>>> >>>>>>>>>>>>> > Unfortunately it's not
>>>>>>> >>>>>>>>>>>>> >     a problem with python but with naily. When I
>>>>>>> changed
>>>>>>> >>>>>>>>>>>>> > number of
>>>>>>> >>>>>>>>>>>>> >     workers in naily from 3 to 2 everything started
>>>>>>> to work.
>>>>>>> >>>>>>>>>>>>> > I think
>>>>>>> >>>>>>>>>>>>> >     it's because all thread are using the same
>>>>>>> connection and
>>>>>>> >>>>>>>>>>>>> > chanel to
>>>>>>> >>>>>>>>>>>>> >     publish results but I'm still investigating it.
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >     Regards
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >     On Thu, Mar 13, 2014 at 12:38 PM, Dmitry Pyzhov
>>>>>>> >>>>>>>>>>>>> >     <dpyzhov@xxxxxxxxxxxx <mailto:
>>>>>>> dpyzhov@xxxxxxxxxxxx>>
>>>>>>> >>>>>>>>>>>>> > wrote:
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >         Lukasz,
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >         Feel free to contact us if you need anything
>>>>>>> else.
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >         On Thu, Mar 13, 2014 at 3:22 PM, Lukasz Oles
>>>>>>> >>>>>>>>>>>>> > <loles@xxxxxxxxxxxx
>>>>>>> >>>>>>>>>>>>> >         <mailto:loles@xxxxxxxxxxxx>> wrote:
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >             ok, thx for rpm
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >             On Thu, Mar 13, 2014 at 11:40 AM, Dmitry
>>>>>>> >>>>>>>>>>>>> > Burmistrov
>>>>>>> >>>>>>>>>>>>> >             <dburmistrov@xxxxxxxxxxxx
>>>>>>> >>>>>>>>>>>>> > <mailto:dburmistrov@xxxxxxxxxxxx>>
>>>>>>> >>>>>>>>>>>>> >             wrote:
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >                 Package rabbitmq-server has been
>>>>>>> built from
>>>>>>> >>>>>>>>>>>>> > changeset:
>>>>>>> >>>>>>>>>>>>> >                 http://gerrit.mirantis.com/13455
>>>>>>> >>>>>>>>>>>>> >                 RPM Repository URL:
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> > http:///
>>>>>>> osci-obs.vm.mirantis.net:82/centos-fuel-5.0-stable-13455/centos
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> > <
>>>>>>> http://osci-obs.vm.mirantis.net:82/centos-fuel-5.0-stable-13455/centos
>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >                 Package rabbitmq-server has been
>>>>>>> built from
>>>>>>> >>>>>>>>>>>>> > changeset:
>>>>>>> >>>>>>>>>>>>> >                 http://gerrit.mirantis.com/13457
>>>>>>> >>>>>>>>>>>>> >                 DEB Repository URL:
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> > http:///
>>>>>>> osci-obs.vm.mirantis.net:82/ubuntu-fuel-5.0-stable-13457/ubuntu
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> > <
>>>>>>> http://osci-obs.vm.mirantis.net:82/ubuntu-fuel-5.0-stable-13457/ubuntu
>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >                 On Thu, Mar 13, 2014 at 2:11 PM,
>>>>>>> Dmitry
>>>>>>> >>>>>>>>>>>>> > Pyzhov
>>>>>>> >>>>>>>>>>>>> >                 <dpyzhov@xxxxxxxxxxxx
>>>>>>> >>>>>>>>>>>>> > <mailto:dpyzhov@xxxxxxxxxxxx>> wrote:
>>>>>>> >>>>>>>>>>>>> >                 > Lukasz,
>>>>>>> >>>>>>>>>>>>> >                 >
>>>>>>> >>>>>>>>>>>>> >                 > Sorry for long response. Our OSCI
>>>>>>> team will
>>>>>>> >>>>>>>>>>>>> > build a
>>>>>>> >>>>>>>>>>>>> >                 package. Dmitry B, could
>>>>>>> >>>>>>>>>>>>> >                 > you reply with the download link
>>>>>>> when it is
>>>>>>> >>>>>>>>>>>>> > available?
>>>>>>> >>>>>>>>>>>>> >                 Ticket OSCI-1016.
>>>>>>> >>>>>>>>>>>>> >                 >
>>>>>>> >>>>>>>>>>>>> >                 >
>>>>>>> >>>>>>>>>>>>> >                 > On Wed, Mar 12, 2014 at 2:18 PM,
>>>>>>> Lukasz
>>>>>>> >>>>>>>>>>>>> > Oles
>>>>>>> >>>>>>>>>>>>> >                 <loles@xxxxxxxxxxxx
>>>>>>> >>>>>>>>>>>>> > <mailto:loles@xxxxxxxxxxxx>> wrote:
>>>>>>> >>>>>>>>>>>>> >                 >>
>>>>>>> >>>>>>>>>>>>> >                 >> Sure, I will look into it. Can
>>>>>>> you give me
>>>>>>> >>>>>>>>>>>>> > a link to
>>>>>>> >>>>>>>>>>>>> >                 rabbitmq rpm which
>>>>>>> >>>>>>>>>>>>> >                 >> you used?
>>>>>>> >>>>>>>>>>>>> >                 >>
>>>>>>> >>>>>>>>>>>>> >                 >> Regards,
>>>>>>> >>>>>>>>>>>>> >                 >>
>>>>>>> >>>>>>>>>>>>> >                 >>
>>>>>>> >>>>>>>>>>>>> >                 >> On Wed, Mar 12, 2014 at 11:10 AM,
>>>>>>> Dmitry
>>>>>>> >>>>>>>>>>>>> > Pyzhov
>>>>>>> >>>>>>>>>>>>> >                 <dpyzhov@xxxxxxxxxxxx
>>>>>>> >>>>>>>>>>>>> > <mailto:dpyzhov@xxxxxxxxxxxx>>
>>>>>>> >>>>>>>>>>>>> >                 >> wrote:
>>>>>>> >>>>>>>>>>>>> >                 >>>
>>>>>>> >>>>>>>>>>>>> >                 >>> Lukasz,
>>>>>>> >>>>>>>>>>>>> >                 >>>
>>>>>>> >>>>>>>>>>>>> >                 >>> actually we have no idea what is
>>>>>>> wrong
>>>>>>> >>>>>>>>>>>>> > with fresh
>>>>>>> >>>>>>>>>>>>> >                 rabbitmq. For some
>>>>>>> >>>>>>>>>>>>> >                 >>> reason refresh of task status
>>>>>>> takes too
>>>>>>> >>>>>>>>>>>>> > much time.
>>>>>>> >>>>>>>>>>>>> >                 Dmitry tried to find the
>>>>>>> >>>>>>>>>>>>> >                 >>> root cause, but did not succeed.
>>>>>>> Could
>>>>>>> >>>>>>>>>>>>> > you
>>>>>>> >>>>>>>>>>>>> >                 investigate the issue?
>>>>>>> >>>>>>>>>>>>> >                 >>>
>>>>>>> >>>>>>>>>>>>> >                 >>>
>>>>>>> >>>>>>>>>>>>> >                 >>> On Tue, Mar 11, 2014 at 9:49 PM,
>>>>>>> Lukasz
>>>>>>> >>>>>>>>>>>>> > Oles
>>>>>>> >>>>>>>>>>>>> >                 <loles@xxxxxxxxxxxx
>>>>>>> >>>>>>>>>>>>> > <mailto:loles@xxxxxxxxxxxx>> wrote:
>>>>>>> >>>>>>>>>>>>> >                 >>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>> Dmitry,
>>>>>>> >>>>>>>>>>>>> >                 >>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>> rabbitmq update looks
>>>>>>> interesting, I can
>>>>>>> >>>>>>>>>>>>> > look into
>>>>>>> >>>>>>>>>>>>> >                 it. Do I need any
>>>>>>> >>>>>>>>>>>>> >                 >>>> additional information?
>>>>>>> >>>>>>>>>>>>> >                 >>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>> regards,
>>>>>>> >>>>>>>>>>>>> >                 >>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>> On Tue, Mar 11, 2014 at 2:05
>>>>>>> PM, Dmitry
>>>>>>> >>>>>>>>>>>>> > Pyzhov
>>>>>>> >>>>>>>>>>>>> >                 <dpyzhov@xxxxxxxxxxxx
>>>>>>> >>>>>>>>>>>>> > <mailto:dpyzhov@xxxxxxxxxxxx>>
>>>>>>> >>>>>>>>>>>>> >                 >>>> wrote:
>>>>>>> >>>>>>>>>>>>> >                 >>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>> Great!
>>>>>>> >>>>>>>>>>>>> >                 >>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>> Lukasz, could you help us with
>>>>>>> rabbitmq
>>>>>>> >>>>>>>>>>>>> > update? We
>>>>>>> >>>>>>>>>>>>> >                 faced an issue with
>>>>>>> >>>>>>>>>>>>> >                 >>>>> it:
>>>>>>> >>>>>>>>>>>>> > https://bugs.launchpad.net/fuel/+bug/1278336
>>>>>>> >>>>>>>>>>>>> >                 >>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>> Also, could you participate it
>>>>>>> design
>>>>>>> >>>>>>>>>>>>> > review:
>>>>>>> >>>>>>>>>>>>> >                 >>>>>
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> https://docs.google.com/document/d/1zqV58LZBLQ-0gllb_i3MyIKIMj-Qx8ELJohjcWs459s/edit?usp=sharing
>>>>>>> >>>>>>>>>>>>> >                 >>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>> On Mon, Mar 10, 2014 at 9:01
>>>>>>> PM, Mike
>>>>>>> >>>>>>>>>>>>> > Scherbakov
>>>>>>> >>>>>>>>>>>>> >                 >>>>> <mscherbakov@xxxxxxxxxxxx
>>>>>>> >>>>>>>>>>>>> >                 <mailto:mscherbakov@xxxxxxxxxxxx>>
>>>>>>> wrote:
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>> Lukasz,
>>>>>>> >>>>>>>>>>>>> >                 >>>>>> please take any bugs from
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> https://launchpad.net/fuel/+milestone/5.0
>>>>>>> >>>>>>>>>>>>> >                 >>>>>> which are not assigned to
>>>>>>> particular
>>>>>>> >>>>>>>>>>>>> > person. You
>>>>>>> >>>>>>>>>>>>> >                 are likely to be interested
>>>>>>> >>>>>>>>>>>>> >                 >>>>>> in those which are assigned to
>>>>>>> >>>>>>>>>>>>> > "fuel-python". Of
>>>>>>> >>>>>>>>>>>>> >                 course, it's preferred to
>>>>>>> >>>>>>>>>>>>> >                 >>>>>> work on Critical and High
>>>>>>> priority
>>>>>>> >>>>>>>>>>>>> > bugs in a
>>>>>>> >>>>>>>>>>>>> >                 first order.
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>> We are in a design phase for
>>>>>>> 5.0.
>>>>>>> >>>>>>>>>>>>> > Please take a
>>>>>>> >>>>>>>>>>>>> >                 look at
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> https://mirantis.jira.com/wiki/display/PRD/5.0+-+Mirantis+OpenStack+release+home+page
>>>>>>> .
>>>>>>> >>>>>>>>>>>>> >                 >>>>>> I'm discussing this still with
>>>>>>> >>>>>>>>>>>>> > management, and we
>>>>>>> >>>>>>>>>>>>> >                 will likely have only part
>>>>>>> >>>>>>>>>>>>> >                 >>>>>> of what is on the page. Your
>>>>>>> comments
>>>>>>> >>>>>>>>>>>>> > and input
>>>>>>> >>>>>>>>>>>>> >                 into design docs (which you
>>>>>>> >>>>>>>>>>>>> >                 >>>>>> can find following blueprint
>>>>>>> link,
>>>>>>> >>>>>>>>>>>>> > then "Read the
>>>>>>> >>>>>>>>>>>>> >                 full spec") is very
>>>>>>> >>>>>>>>>>>>> >                 >>>>>> welcome.
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>> Start looking over and try to
>>>>>>> identify
>>>>>>> >>>>>>>>>>>>> > spot which
>>>>>>> >>>>>>>>>>>>> >                 is in most interest
>>>>>>> >>>>>>>>>>>>> >                 >>>>>> of you. Dmitry/Evgeny will
>>>>>>> help to
>>>>>>> >>>>>>>>>>>>> > identify areas
>>>>>>> >>>>>>>>>>>>> >                 where help is mostly
>>>>>>> >>>>>>>>>>>>> >                 >>>>>> needed. Sorry for not
>>>>>>> responding to
>>>>>>> >>>>>>>>>>>>> > you in time.
>>>>>>> >>>>>>>>>>>>> >                 I'll get my team to fix
>>>>>>> >>>>>>>>>>>>> >                 >>>>>> this.
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>> FYI: Today is holiday in
>>>>>>> Russia &
>>>>>>> >>>>>>>>>>>>> > Ukraine
>>>>>>> >>>>>>>>>>>>> >                 >>>>>> Thanks,
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>> On Thu, Mar 6, 2014 at 12:37
>>>>>>> AM, Mike
>>>>>>> >>>>>>>>>>>>> > Scherbakov
>>>>>>> >>>>>>>>>>>>> >                 >>>>>> <mscherbakov@xxxxxxxxxxxx
>>>>>>> >>>>>>>>>>>>> >                 <mailto:mscherbakov@xxxxxxxxxxxx>>
>>>>>>> wrote:
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>> It's great. I would be happy
>>>>>>> to see
>>>>>>> >>>>>>>>>>>>> > Lukasz
>>>>>>> >>>>>>>>>>>>> >                 working on Fuel.
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>> Actually, Lukasz already
>>>>>>> doing great
>>>>>>> >>>>>>>>>>>>> > job helping
>>>>>>> >>>>>>>>>>>>> >                 us with Nailgun
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>> scalability issues
>>>>>>> resolution.
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>> Dmitry, please arrange
>>>>>>> meeting
>>>>>>> >>>>>>>>>>>>> > between our
>>>>>>> >>>>>>>>>>>>> >                 Python engineers and
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>> Lukasz, and identify areas
>>>>>>> where
>>>>>>> >>>>>>>>>>>>> > contribution of
>>>>>>> >>>>>>>>>>>>> >                 Lukasz will be the most
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>> effective. It should be
>>>>>>> aligned with
>>>>>>> >>>>>>>>>>>>> > the
>>>>>>> >>>>>>>>>>>>> >                 development of our engineers too.
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>> Thanks,
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>> On Tue, Mar 4, 2014 at 3:27
>>>>>>> PM, Piotr
>>>>>>> >>>>>>>>>>>>> > Siwczak
>>>>>>> >>>>>>>>>>>>> >                 <psiwczak@xxxxxxxxxxxx
>>>>>>> >>>>>>>>>>>>> > <mailto:psiwczak@xxxxxxxxxxxx>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>> wrote:
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>>> Mike,
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>>> For now I see Lukasz has
>>>>>>> finished
>>>>>>> >>>>>>>>>>>>> > his work for
>>>>>>> >>>>>>>>>>>>> >                 Softlayer/Express (at
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>>> least for the next few
>>>>>>> weeks)and can
>>>>>>> >>>>>>>>>>>>> > use his
>>>>>>> >>>>>>>>>>>>> >                 time to engage into Fuel
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>>> development. Please feel
>>>>>>> free to
>>>>>>> >>>>>>>>>>>>> > assign him to
>>>>>>> >>>>>>>>>>>>> >                 Fuel tasks.
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>>> -Piotr
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>> --
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>> Mike Scherbakov
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>> #mihgen
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>> --
>>>>>>> >>>>>>>>>>>>> >                 >>>>>> Mike Scherbakov
>>>>>>> >>>>>>>>>>>>> >                 >>>>>> #mihgen
>>>>>>> >>>>>>>>>>>>> >                 >>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>>
>>>>>>> >>>>>>>>>>>>> >                 >>>> --
>>>>>>> >>>>>>>>>>>>> >                 >>>> Łukasz Oleś
>>>>>>> >>>>>>>>>>>>> >                 >>>
>>>>>>> >>>>>>>>>>>>> >                 >>>
>>>>>>> >>>>>>>>>>>>> >                 >>
>>>>>>> >>>>>>>>>>>>> >                 >>
>>>>>>> >>>>>>>>>>>>> >                 >>
>>>>>>> >>>>>>>>>>>>> >                 >> --
>>>>>>> >>>>>>>>>>>>> >                 >> Łukasz Oleś
>>>>>>> >>>>>>>>>>>>> >                 >
>>>>>>> >>>>>>>>>>>>> >                 >
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >             --
>>>>>>> >>>>>>>>>>>>> >             Łukasz Oleś
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >     --
>>>>>>> >>>>>>>>>>>>> >     Łukasz Oleś
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> >
>>>>>>> >>>>>>>>>>>>> > --
>>>>>>> >>>>>>>>>>>>> > You received this message because you are subscribed
>>>>>>> to the
>>>>>>> >>>>>>>>>>>>> > Google
>>>>>>> >>>>>>>>>>>>> > Groups "fuel-core-team" group.
>>>>>>> >>>>>>>>>>>>> > To unsubscribe from this group and stop receiving
>>>>>>> emails from
>>>>>>> >>>>>>>>>>>>> > it, send
>>>>>>> >>>>>>>>>>>>> > an email to fuel-core-team+unsubscribe@xxxxxxxxxxxx
>>>>>>> >>>>>>>>>>>>> > <mailto:fuel-core-team+unsubscribe@xxxxxxxxxxxx>.
>>>>>>> >>>>>>>>>>>>> > For more options, visit
>>>>>>> >>>>>>>>>>>>> > https://groups.google.com/a/mirantis.com/d/optout.
>>>>>>> >>>>>>>>>>>>>
>>>>>>> >>>>>>>>>>>>> --
>>>>>>> >>>>>>>>>>>>> You received this message because you are subscribed
>>>>>>> to the
>>>>>>> >>>>>>>>>>>>> Google Groups "fuel-core-team" group.
>>>>>>> >>>>>>>>>>>>> To unsubscribe from this group and stop receiving
>>>>>>> emails from
>>>>>>> >>>>>>>>>>>>> it, send an email to
>>>>>>> fuel-core-team+unsubscribe@xxxxxxxxxxxx.
>>>>>>> >>>>>>>>>>>>> For more options, visit
>>>>>>> >>>>>>>>>>>>> https://groups.google.com/a/mirantis.com/d/optout.
>>>>>>> >>>>>>>>>>>>
>>>>>>> >>>>>>>>>>>>
>>>>>>> >>>>>>>>>>>>
>>>>>>> >>>>>>>>>>>>
>>>>>>> >>>>>>>>>>>> --
>>>>>>> >>>>>>>>>>>> Mike Scherbakov
>>>>>>> >>>>>>>>>>>> #mihgen
>>>>>>> >>>>>>>>>>>
>>>>>>> >>>>>>>>>>>
>>>>>>> >>>>>>>>>>> --
>>>>>>> >>>>>>>>>>> You received this message because you are subscribed to
>>>>>>> the
>>>>>>> >>>>>>>>>>> Google Groups "fuel-core-team" group.
>>>>>>> >>>>>>>>>>> To unsubscribe from this group and stop receiving emails
>>>>>>> from it,
>>>>>>> >>>>>>>>>>> send an email to fuel-core-team+unsubscribe@xxxxxxxxxxxx
>>>>>>> .
>>>>>>> >>>>>>>>>>> For more options, visit
>>>>>>> >>>>>>>>>>> https://groups.google.com/a/mirantis.com/d/optout.
>>>>>>> >>>>>>>>>>
>>>>>>> >>>>>>>>>>
>>>>>>> >>>>>>>>>>
>>>>>>> >>>>>>>>>>
>>>>>>> >>>>>>>>>> --
>>>>>>> >>>>>>>>>> Andrey Danin
>>>>>>> >>>>>>>>>> adanin@xxxxxxxxxxxx
>>>>>>> >>>>>>>>>> skype: gcon.monolake
>>>>>>> >>>>>>>>>
>>>>>>> >>>>>>>>>
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>>
>>>>>>> >>>>>>>> --
>>>>>>> >>>>>>>> Łukasz Oleś
>>>>>>> >>>>>>>
>>>>>>> >>>>>>>
>>>>>>> >>>>>>>
>>>>>>> >>>>>>>
>>>>>>> >>>>>>> --
>>>>>>> >>>>>>> Mike Scherbakov
>>>>>>> >>>>>>> #mihgen
>>>>>>> >>>>>>
>>>>>>> >>>>>>
>>>>>>> >>>>>
>>>>>>> >>>>>
>>>>>>> >>>>>
>>>>>>> >>>>> --
>>>>>>> >>>>> Łukasz Oleś
>>>>>>> >>>>
>>>>>>> >>>>
>>>>>>> >>>
>>>>>>> >>
>>>>>>> >
>>>>>>> > --
>>>>>>> > You received this message because you are subscribed to the Google
>>>>>>> Groups
>>>>>>> > "fuel-core-team" group.
>>>>>>> > To unsubscribe from this group and stop receiving emails from it,
>>>>>>> send an
>>>>>>> > email to fuel-core-team+unsubscribe@xxxxxxxxxxxx.
>>>>>>> > For more options, visit
>>>>>>> https://groups.google.com/a/mirantis.com/d/optout.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Dmitry Borodaenko
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Łukasz Oleś
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "fuel-core-team" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to fuel-core-team+unsubscribe@xxxxxxxxxxxx.
>>>>> For more options, visit
>>>>> https://groups.google.com/a/mirantis.com/d/optout.
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Yours Faithfully,
>>>> Vladimir Kuklin,
>>>> Fuel Library Tech Lead,
>>>> Mirantis, Inc.
>>>> +7 (495) 640-49-04
>>>> +7 (926) 702-39-68
>>>> Skype kuklinvv
>>>> 45bk3, Vorontsovskaya Str.
>>>> Moscow, Russia,
>>>> www.mirantis.com <http://www.mirantis.ru/>
>>>> www.mirantis.ru
>>>> vkuklin@xxxxxxxxxxxx
>>>>
>>>
>>>
>>>
>>> --
>>> Łukasz Oleś
>>>
>>
>>
>
>
> --
> Mike Scherbakov
> #mihgen
>

References