fuel-dev team mailing list archive

[Roman Alekseenkov <ralekseenkov@xxxxxxxxxxxx>]

Team,

Shall we settle on a conclusion? If we need more memory for a working HA,
let's accept this fact and move on.

I believe we should define a set of supported scenarios for the VirtualBox,
update our scripts, update release notes and also change quick start on
Mirantis portal. I filed https://bugs.launchpad.net/fuel/+bug/1361331 to
track that.

My suggestion is to introduce something along the lines of:

   1. config_4GB.sh - non-HA, 1 controller, 1 compute
   2. config_8GB.sh - non-HA, 1 controller, 3 compute nodes
   3. config_16GB.sh - HA, 3 controllers, 1 compute

If we want to go beyond that, there should be an answer in form of
deploying OVA/OVF into vCenter and utilizing more compute power from a set
of ESX hosts.

Thanks,
Roman


On Fri, Aug 22, 2014 at 1:09 AM, Meg McRoberts <mmcroberts@xxxxxxxxxxxx>
wrote:

> We should check with our training people to see what they need.  For
> example, any
> reasonable evaluation of MongoDB (used for Ceilometer) or Zabbix probably
> requires
> a lot more resources than people are going to have on their personal
> machine, but it
> would be nice if they could configure some sort of minimally-functional
> version of these
> on a smaller system just for training purposes.
>
>
> On Fri, Aug 22, 2014 at 12:12 AM, Sergii Golovatiuk <
> sgolovatiuk@xxxxxxxxxxxx> wrote:
>
>> Hi,
>>
>> I would say we have to update release notes. Even now it's barely
>> possible to unleash Fuel power on laptop with 8GB of RAM. With 16Gb you may
>> have up to 5 VMs simulating HA environment. If we limit VM RAM to 2.5 user
>> may have 5x2.5+1(master node)=13.5 GB which should be enough for demo
>> purposes.
>>
>>
>> --
>> Best regards,
>> Sergii Golovatiuk,
>> Skype #golserge
>> IRC #holser
>>
>>
>> On Fri, Aug 22, 2014 at 3:06 AM, Christopher Aedo <caedo@xxxxxxxxxxxx>
>> wrote:
>>
>>> In my opinion it's reasonable to limit expectations on what you can
>>> evaluate on virtualbox, and provide appropriate caveats.  The truth is
>>> that making a virtual machine do everything it needs to do to stay
>>> synced in real-time with two or more other virtual machines requires
>>> resources.  Evaluating Fuel as a deployment tool under virtualbox
>>> makes sense, but expecting to get a legitimate evaluation of working
>>> (and performant) HA under a severely restricted environment doesn't
>>> seem reasonable.
>>>
>>> Would a legitimate customer expect to be able to do a thorough
>>> evaluation of MOS without using anything more than a single machine
>>> with a moderate amount of ram?
>>>
>>> -Christopher
>>>
>>> On Thu, Aug 21, 2014 at 4:59 PM, David Easter <deaster@xxxxxxxxxxxx>
>>> wrote:
>>> > It has already became nearly impossible to run the default virtualbox
>>> > environment (with a cluster size of 3 nodes) on an 8GB system.  It
>>> would
>>> > become challenging if we begin to tax a 16GB system to do this now…
>>> >
>>> > Are there alternatives or specific tunings that can be done so that the
>>> > sizing for a virtualbox trial is different than what is needed for
>>> > production or physical server testing?
>>> >
>>> > Thanks,
>>> >
>>> > - David J. Easter
>>> >   Director of Product Management,   Mirantis, Inc.
>>> >
>>> > From: "ralekseenkov@xxxxxxxxxxxx" <ralekseenkov@xxxxxxxxxxxx>
>>> > Date: Thursday, August 21, 2014 at 4:29 PM
>>> > To: Sergii Golovatiuk <sgolovatiuk@xxxxxxxxxxxx>, David Easter
>>> > <deaster@xxxxxxxxxxxx>
>>> > Cc: Bogdan Dobrelya <bdobrelia@xxxxxxxxxxxx>, "
>>> fuel-dev@xxxxxxxxxxxxxxxxxxx"
>>> > <fuel-dev@xxxxxxxxxxxxxxxxxxx>
>>> > Subject: Re: [Fuel-dev] RAM issues with environments
>>> >
>>> > David,
>>> >
>>> > FYI - it will impact VirtualBox trials. 3 controllers x 3GB = 9 GB RAM
>>> only
>>> > for controllers if you run HA.
>>> >
>>> > Roman
>>> >
>>> >
>>> > On Thu, Aug 21, 2014 at 5:22 AM, Sergii Golovatiuk
>>> > <sgolovatiuk@xxxxxxxxxxxx> wrote:
>>> >>
>>> >> +1 for not using debug at CI gates (only BVT)
>>> >> +1 for adding atop to our builds as it helps to understand what was
>>> wrong
>>> >> at that particular time
>>> >> +1 for increasing RAM (though we'll try to tune rabbit and Galera).
>>> 3GB
>>> >> should be enough so we'll be able to run up to 2 environments on 32GB
>>> RAM
>>> >> servers.
>>> >>
>>> >> --
>>> >> Best regards,
>>> >> Sergii Golovatiuk,
>>> >> Skype #golserge
>>> >> IRC #holser
>>> >>
>>> >>
>>> >> On Thu, Aug 21, 2014 at 1:25 PM, Bogdan Dobrelya <
>>> bdobrelia@xxxxxxxxxxxx>
>>> >> wrote:
>>> >>>
>>> >>> On 08/21/2014 12:41 PM, Sergii Golovatiuk wrote:
>>> >>> > Hi,
>>> >>> >
>>> >>> > Digging the issue with Galera, I found that our environments have
>>> very
>>> >>> > high RAM utilization which leads to the problem during environment
>>> >>> > deployment. For instance "HA deployment + neutron/GRE" requires
>>> almost
>>> >>> > 2.6-2.7 GB during deployment
>>> >>> > (corosync+mysql+puppet++rabbit+neutron+ovs+openstack services). I
>>> found
>>> >>> > high swap in/swap out usage during deployment with very high load
>>> >>> > average. This creates many sporadic issues with some services. They
>>> >>> > time
>>> >>> > out in random place making our debugging very hard. I would like to
>>> >>> > review our policy for CI environment and increase RAM (at least
>>> for bvt
>>> >>> > tests) to 3 GB.
>>> >>> >
>>> >>> > --
>>> >>> > Best regards,
>>> >>> > Sergii Golovatiuk,
>>> >>> > Skype #golserge
>>> >>> > IRC #holser
>>> >>> >
>>> >>> >
>>> >>>
>>> >>> I believe we should do at least the following for our CI jobs and bvt
>>> >>> tests:
>>> >>> 1) Deployment shortcuts: stop deployment abruptly, if any deployment
>>> >>> blocker has been met, such as something exceeded given # of retries.
>>> >>> That could be done in puppet by overriding 'tries' behavior in exec
>>> >>> provider, or at orchestration layer as well.
>>> >>> 2) Load management: collect and automatically analyze atop stats
>>> (swap
>>> >>> rates, load average, io waiters) from jenkins slaves and vm nodes
>>> while
>>> >>> running the jobs, and stop or freeze some jobs, if some
>>> >>> performance-stopper criteria has been met as well.
>>> >>> 3) Do not use debug level logging for CI gates, use it only for bvt
>>> >>> tests.
>>> >>>
>>> >>> --
>>> >>> Best regards,
>>> >>> Bogdan Dobrelya,
>>> >>> Skype #bogdando_at_yahoo.com
>>> >>> Irc #bogdando
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Mailing list: https://launchpad.net/~fuel-dev
>>> >> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>> >> Unsubscribe : https://launchpad.net/~fuel-dev
>>> >> More help   : https://help.launchpad.net/ListHelp
>>> >>
>>> >
>>> >
>>> > --
>>> > Mailing list: https://launchpad.net/~fuel-dev
>>> > Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>> > Unsubscribe : https://launchpad.net/~fuel-dev
>>> > More help   : https://help.launchpad.net/ListHelp
>>> >
>>>
>>
>>
>> --
>> Mailing list: https://launchpad.net/~fuel-dev
>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~fuel-dev
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
> --
> Mailing list: https://launchpad.net/~fuel-dev
> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~fuel-dev
> More help   : https://help.launchpad.net/ListHelp
>
>