fuel-dev team mailing list archive

[David Easter <deaster@xxxxxxxxxxxx>]

Right now, Fuel deploys Ceilometer to use the OpenStack mySQL DB for
Ceilometer data, although we’re expecting to utilize MongoDB in the 4.1
Fuel version.  This is, as you mention, primarily to address the
performance issues with a shared mySQL DB.  That said, I don’t know that
it really changes the situation.  It’s not the OSTF info that’s being
stored, it’s the actual results of Ceilometer running against a
short-lived VM.  Those will still go into the Ceilometer DB, I believe, so
they’d have to be removed from there.

If, as Julian mentioned, there's a time-to-live mechanism in Ceilometer to
delete old samples then I would think we could just go with that.  When
OSTF runs the action, set the TTL against that sample.

Does that address the issue?

Thanks,

- David J. Easter
  Product Line Manager,  Mirantis



On 1/20/14, 4:48 AM, "Bogdan Dobrelya" <bdobrelia@xxxxxxxxxxxx> wrote:

>On 01/20/2014 01:29 PM, Dmitry Iakunchikov wrote:
>> David,
>> 
>> You're completely right,
>> 
>> The main problem is that ceilometer has possibility to create samples,
>> but not to delete. Because of that there is no possibility to remove
>> OSTF created data.
>1) What is current DB backend for Ceilometer? Can we use the separate
>database for OSTF and just drop while doing teardown?
>2) IIRC from Ceilometer PoC performance results, the main DB with Galera
>must never be used as backend for ceilometer because of performance
>issues?
>> 
>> Actually another way is to use time_to_live, but as you sad "As an
>> operator, I¹d expect that my data is retained even for items that have
>> been removed"(c)
>> 
>> 
>> 2014/1/17 David Easter <deaster@xxxxxxxxxxxx
>><mailto:deaster@xxxxxxxxxxxx>>
>> 
>>     I¹d like to make sure I understand the question.  Is this the
>>scenario?
>> 
>>       * A user installs OpenStack
>>       * The user runs the OpenStack Health Check (OSTF) against
>>         Ceilometer
>>       * The Health Check creates a VM against which ceilometer can
>>         collect data
>>       * Ceilometer collects the data from this VM for an amount of time
>>         and stores the data in mySQL
>>       * The Health Check then ends the test, removing the VM
>>       * The data collected about this sample VM is retained in mySQL and
>>         is not removed.
>> 
>>     Is this basically correct?
>> 
>>     If so, I¹d ask if Ceilometer removes data from VM¹s or nodes that
>>     have been deleted from OpenStack during normal operation or if the
>>     data is retained in the run-time scenarios as well?  If so, wouldn¹t
>>     this be a general requirement to remove data about entities that no
>>     longer exist in the environment vs. an issue specific to Health
>>     Check (OSTF)?
>> 
>>     As an operator, I¹d expect that my data is retained even for items
>>     that have been removed, but I agree that there should be a way for
>>     an operator to make a decision to remove stale data ­ either based
>>     on time or as a manually executed operation.  Removing data
>>     automatically right away could lead to a loss of historical
>>     information that could be used for longer term analysis and billing.
>> 
>>     Or am I misinterpreting the situation and Ceilometer already allows
>>     for deletion of data ­ and the question is just whether we should
>>     remove the data collected during the test?  If that is the only
>>     question, then yes ­ we should remove the data after the test is
>>done.
>> 
>>     Thanks,
>> 
>>     -Dave Easter
>> 
>>     From: Dmitry Iakunchikov <diakunchikov@xxxxxxxxxxxx
>>     <mailto:diakunchikov@xxxxxxxxxxxx>>
>>     Date: Friday, January 17, 2014 at 5:10 AM
>>     To: Nadya Privalova <nprivalova@xxxxxxxxxxxx
>>     <mailto:nprivalova@xxxxxxxxxxxx>>, "OpenStack Development Mailing
>>     List (not for usage questions)" <openstack-dev@xxxxxxxxxxxxxxxxxxx
>>     <mailto:openstack-dev@xxxxxxxxxxxxxxxxxxx>>, Dmitry Iakunchikov
>>     <diakunchikov@xxxxxxxxxxxx <mailto:diakunchikov@xxxxxxxxxxxx>>, Mike
>>     Scherbakov <mscherbakov@xxxxxxxxxxxx
>>     <mailto:mscherbakov@xxxxxxxxxxxx>>, Vladimir Kuklin
>>     <vkuklin@xxxxxxxxxxxx <mailto:vkuklin@xxxxxxxxxxxx>>,
>>     "fuel-dev@xxxxxxxxxxxxxxxxxxx <mailto:fuel-dev@xxxxxxxxxxxxxxxxxxx>"
>>     <fuel-dev@xxxxxxxxxxxxxxxxxxx <mailto:fuel-dev@xxxxxxxxxxxxxxxxxxx>>
>>     Subject: Re: [Fuel-dev] [openstack-dev] [OSTF][Ceilometer]
>>     ceilometer meters and samples delete
>> 
>>     For now in Fuel we keep samples forever
>> 
>>     In case if we will use time_to_live, how long we should keep this
>>data?
>> 
>> 
>>     2014/1/17 Julien Danjou <julien@xxxxxxxxxxx
>><mailto:julien@xxxxxxxxxxx>>
>> 
>>         On Fri, Jan 17 2014, Nadya Privalova wrote:
>> 
>>         > I would ask in another way.
>>         > Ceilometer has a mechanism to add a sample through POST. So
>>it looks
>not
>>         > consistent not to allow user to delete a sample.
>>         > IMHO, insertion and deletion through REST looks a little bit
>>hacky:
>user
>>         > always has an ability to fake data collected from OpenStack
>services. But
>>         > maybe I don't see any valuable usecases.
>>         > Anyway, it seems reasonable to have both add_sample and
>delete_sample in
>>         > API or not to have neither.
>> 
>>         From the user PoV, that totally makes sense, agreed.
>> 
>>         --
>>         Julien Danjou
>>         # Free Software hacker # independent consultant
>>         # http://julien.danjou.info
>> 
>> 
>> 
>> 
>>     -- 
>>     With Best Regards
>>     QA engineer Dmitry Iakunchikov
>>     -- Mailing list: https://launchpad.net/~fuel-dev Post to :
>>     fuel-dev@xxxxxxxxxxxxxxxxxxx <mailto:fuel-dev@xxxxxxxxxxxxxxxxxxx>
>>     Unsubscribe : https://launchpad.net/~fuel-dev More help :
>>     https://help.launchpad.net/ListHelp
>> 
>> 
>> 
>> 
>> -- 
>> With Best Regards
>> QA engineer Dmitry Iakunchikov
>> 
>> 
>
>
>-- 
>Best regards,
>Bogdan Dobrelya,
>Researcher TechLead, Mirantis, Inc.
>+38 (066) 051 07 53
>Skype bogdando_at_yahoo.com
>Irc #bogdando
>38, Lenina ave.
>Kharkov, Ukraine
>www.mirantis.com
>www.mirantis.ru
>bdobrelia@xxxxxxxxxxxx
>