← Back to team overview

fuel-dev team mailing list archive

Re: RAM issues with environments

 

Hi,

I would say we have to update release notes. Even now it's barely possible
to unleash Fuel power on laptop with 8GB of RAM. With 16Gb you may have up
to 5 VMs simulating HA environment. If we limit VM RAM to 2.5 user may have
5x2.5+1(master node)=13.5 GB which should be enough for demo purposes.


--
Best regards,
Sergii Golovatiuk,
Skype #golserge
IRC #holser


On Fri, Aug 22, 2014 at 3:06 AM, Christopher Aedo <caedo@xxxxxxxxxxxx>
wrote:

> In my opinion it's reasonable to limit expectations on what you can
> evaluate on virtualbox, and provide appropriate caveats.  The truth is
> that making a virtual machine do everything it needs to do to stay
> synced in real-time with two or more other virtual machines requires
> resources.  Evaluating Fuel as a deployment tool under virtualbox
> makes sense, but expecting to get a legitimate evaluation of working
> (and performant) HA under a severely restricted environment doesn't
> seem reasonable.
>
> Would a legitimate customer expect to be able to do a thorough
> evaluation of MOS without using anything more than a single machine
> with a moderate amount of ram?
>
> -Christopher
>
> On Thu, Aug 21, 2014 at 4:59 PM, David Easter <deaster@xxxxxxxxxxxx>
> wrote:
> > It has already became nearly impossible to run the default virtualbox
> > environment (with a cluster size of 3 nodes) on an 8GB system.  It would
> > become challenging if we begin to tax a 16GB system to do this now…
> >
> > Are there alternatives or specific tunings that can be done so that the
> > sizing for a virtualbox trial is different than what is needed for
> > production or physical server testing?
> >
> > Thanks,
> >
> > - David J. Easter
> >   Director of Product Management,   Mirantis, Inc.
> >
> > From: "ralekseenkov@xxxxxxxxxxxx" <ralekseenkov@xxxxxxxxxxxx>
> > Date: Thursday, August 21, 2014 at 4:29 PM
> > To: Sergii Golovatiuk <sgolovatiuk@xxxxxxxxxxxx>, David Easter
> > <deaster@xxxxxxxxxxxx>
> > Cc: Bogdan Dobrelya <bdobrelia@xxxxxxxxxxxx>, "
> fuel-dev@xxxxxxxxxxxxxxxxxxx"
> > <fuel-dev@xxxxxxxxxxxxxxxxxxx>
> > Subject: Re: [Fuel-dev] RAM issues with environments
> >
> > David,
> >
> > FYI - it will impact VirtualBox trials. 3 controllers x 3GB = 9 GB RAM
> only
> > for controllers if you run HA.
> >
> > Roman
> >
> >
> > On Thu, Aug 21, 2014 at 5:22 AM, Sergii Golovatiuk
> > <sgolovatiuk@xxxxxxxxxxxx> wrote:
> >>
> >> +1 for not using debug at CI gates (only BVT)
> >> +1 for adding atop to our builds as it helps to understand what was
> wrong
> >> at that particular time
> >> +1 for increasing RAM (though we'll try to tune rabbit and Galera). 3GB
> >> should be enough so we'll be able to run up to 2 environments on 32GB
> RAM
> >> servers.
> >>
> >> --
> >> Best regards,
> >> Sergii Golovatiuk,
> >> Skype #golserge
> >> IRC #holser
> >>
> >>
> >> On Thu, Aug 21, 2014 at 1:25 PM, Bogdan Dobrelya <
> bdobrelia@xxxxxxxxxxxx>
> >> wrote:
> >>>
> >>> On 08/21/2014 12:41 PM, Sergii Golovatiuk wrote:
> >>> > Hi,
> >>> >
> >>> > Digging the issue with Galera, I found that our environments have
> very
> >>> > high RAM utilization which leads to the problem during environment
> >>> > deployment. For instance "HA deployment + neutron/GRE" requires
> almost
> >>> > 2.6-2.7 GB during deployment
> >>> > (corosync+mysql+puppet++rabbit+neutron+ovs+openstack services). I
> found
> >>> > high swap in/swap out usage during deployment with very high load
> >>> > average. This creates many sporadic issues with some services. They
> >>> > time
> >>> > out in random place making our debugging very hard. I would like to
> >>> > review our policy for CI environment and increase RAM (at least for
> bvt
> >>> > tests) to 3 GB.
> >>> >
> >>> > --
> >>> > Best regards,
> >>> > Sergii Golovatiuk,
> >>> > Skype #golserge
> >>> > IRC #holser
> >>> >
> >>> >
> >>>
> >>> I believe we should do at least the following for our CI jobs and bvt
> >>> tests:
> >>> 1) Deployment shortcuts: stop deployment abruptly, if any deployment
> >>> blocker has been met, such as something exceeded given # of retries.
> >>> That could be done in puppet by overriding 'tries' behavior in exec
> >>> provider, or at orchestration layer as well.
> >>> 2) Load management: collect and automatically analyze atop stats (swap
> >>> rates, load average, io waiters) from jenkins slaves and vm nodes while
> >>> running the jobs, and stop or freeze some jobs, if some
> >>> performance-stopper criteria has been met as well.
> >>> 3) Do not use debug level logging for CI gates, use it only for bvt
> >>> tests.
> >>>
> >>> --
> >>> Best regards,
> >>> Bogdan Dobrelya,
> >>> Skype #bogdando_at_yahoo.com
> >>> Irc #bogdando
> >>
> >>
> >>
> >> --
> >> Mailing list: https://launchpad.net/~fuel-dev
> >> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
> >> Unsubscribe : https://launchpad.net/~fuel-dev
> >> More help   : https://help.launchpad.net/ListHelp
> >>
> >
> >
> > --
> > Mailing list: https://launchpad.net/~fuel-dev
> > Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
> > Unsubscribe : https://launchpad.net/~fuel-dev
> > More help   : https://help.launchpad.net/ListHelp
> >
>

Follow ups

References