fuel-dev team mailing list archive

[Nadya Privalova <nprivalova@xxxxxxxxxxxx>]

Dmitry,

Creating a patch is ok only for samples I guess. Because if you want to
remove resource it causes removing all meters and samples with this
resource_id. I'm afraid that feature may be rejected by community. And this
patch should not be only in client but in main code too.
The second option looks synthetic for me, but anyway it's up to you. As
well as the third.

P.S. I haven't ever try ttl mechanism, so test it before usage (make sure
it is implemented in backend that you want to test). Dropping db and
creating a new one should not be a problem for Ceilometer, I tried to do it
100+ times.

Thanks,
Nadya


On Wed, Jan 22, 2014 at 2:45 PM, Dmitry Iakunchikov <
diakunchikov@xxxxxxxxxxxx> wrote:

> Nadia,
>
> As I wrote in first letter, A am asking also about meters.
> Flow for OSTF tests is(Like David wrote):
>
>    - A user installs Mirantis OpenStack
>    - The user runs the Mirantis OpenStack Health Check (OSTF) against
>    Ceilometer
>    - The Health Check creates a VM against which ceilometer can collect
>    data
>    - Ceilometer collects the data from this VM for an amount of time and
>    stores the data in mySQL
>    - The Health Check then ends the test, removing the VM
>    - The data collected about this VM is retained in mySQL and is not
>    removed.
>
> And there is no possibility to delete this data(meters, samples,
> resources), and it is absolutely correct that data is kept in Ceilometer db
> after vm is deleted, but as operator I need possibility to  delete this data
>
> In my understanding, If we use time_to_live, we could not use TTL for
> specific sample, because this parameter sets in config file
> We can not drop database, because it could create some new troubles( Onse
> I tried to drob database, and after all steps ceilometer wouldn't started)
>
> And I see 3 ways:
>
>
>    - Create patch set to the ceilometer_client with API call
>    - Live created samples and meters, but create to them some not
>    original names
>    - Wipe database, but need to approve from Mike or Vladimir
>
>
>
>
> 2014/1/22 Nadya Privalova <nprivalova@xxxxxxxxxxxx>
>
>> Hi guys,
>>
>> I think that this discussion is mostly OSTF-related so I remove
>> openstack-dev.
>> Sorry, guys, I don't know how OSTF should work and still don't understand
>> your flow.
>> As for me, it is absolutely correct that data is kept in Ceilometer db
>> after vm is deleted, as David wrote.
>> Do you want that Ceilometer db will be empty after each test? Why are you
>> asking only about samples? There are resources and meters that are not
>> deleted either.
>> If you want to delete data anyway why you cannot drop db?
>>
>> Thanks,
>> Nadya
>>
>>
>>
>> On Tue, Jan 21, 2014 at 11:01 PM, David Easter <deaster@xxxxxxxxxxxx>wrote:
>>
>>> Right now, Fuel deploys Ceilometer to use the OpenStack mySQL DB for
>>> Ceilometer data, although we’re expecting to utilize MongoDB in the 4.1
>>> Fuel version.  This is, as you mention, primarily to address the
>>> performance issues with a shared mySQL DB.  That said, I don’t know that
>>> it really changes the situation.  It’s not the OSTF info that’s being
>>> stored, it’s the actual results of Ceilometer running against a
>>> short-lived VM.  Those will still go into the Ceilometer DB, I believe,
>>> so
>>> they’d have to be removed from there.
>>>
>>> If, as Julian mentioned, there's a time-to-live mechanism in Ceilometer
>>> to
>>> delete old samples then I would think we could just go with that.  When
>>> OSTF runs the action, set the TTL against that sample.
>>>
>>> Does that address the issue?
>>>
>>> Thanks,
>>>
>>> - David J. Easter
>>>   Product Line Manager,  Mirantis
>>>
>>>
>>>
>>> On 1/20/14, 4:48 AM, "Bogdan Dobrelya" <bdobrelia@xxxxxxxxxxxx> wrote:
>>>
>>> >On 01/20/2014 01:29 PM, Dmitry Iakunchikov wrote:
>>> >> David,
>>> >>
>>> >> You're completely right,
>>> >>
>>> >> The main problem is that ceilometer has possibility to create samples,
>>> >> but not to delete. Because of that there is no possibility to remove
>>> >> OSTF created data.
>>> >1) What is current DB backend for Ceilometer? Can we use the separate
>>> >database for OSTF and just drop while doing teardown?
>>> >2) IIRC from Ceilometer PoC performance results, the main DB with Galera
>>> >must never be used as backend for ceilometer because of performance
>>> >issues?
>>> >>
>>> >> Actually another way is to use time_to_live, but as you sad "As an
>>> >> operator, I¹d expect that my data is retained even for items that have
>>> >> been removed"(c)
>>> >>
>>> >>
>>> >> 2014/1/17 David Easter <deaster@xxxxxxxxxxxx
>>> >><mailto:deaster@xxxxxxxxxxxx>>
>>> >>
>>> >>     I¹d like to make sure I understand the question.  Is this the
>>> >>scenario?
>>> >>
>>> >>       * A user installs OpenStack
>>> >>       * The user runs the OpenStack Health Check (OSTF) against
>>> >>         Ceilometer
>>> >>       * The Health Check creates a VM against which ceilometer can
>>> >>         collect data
>>> >>       * Ceilometer collects the data from this VM for an amount of
>>> time
>>> >>         and stores the data in mySQL
>>> >>       * The Health Check then ends the test, removing the VM
>>> >>       * The data collected about this sample VM is retained in mySQL
>>> and
>>> >>         is not removed.
>>> >>
>>> >>     Is this basically correct?
>>> >>
>>> >>     If so, I¹d ask if Ceilometer removes data from VM¹s or nodes that
>>> >>     have been deleted from OpenStack during normal operation or if the
>>> >>     data is retained in the run-time scenarios as well?  If so,
>>> wouldn¹t
>>> >>     this be a general requirement to remove data about entities that
>>> no
>>> >>     longer exist in the environment vs. an issue specific to Health
>>> >>     Check (OSTF)?
>>> >>
>>> >>     As an operator, I¹d expect that my data is retained even for items
>>> >>     that have been removed, but I agree that there should be a way for
>>> >>     an operator to make a decision to remove stale data ­ either based
>>> >>     on time or as a manually executed operation.  Removing data
>>> >>     automatically right away could lead to a loss of historical
>>> >>     information that could be used for longer term analysis and
>>> billing.
>>> >>
>>> >>     Or am I misinterpreting the situation and Ceilometer already
>>> allows
>>> >>     for deletion of data ­ and the question is just whether we should
>>> >>     remove the data collected during the test?  If that is the only
>>> >>     question, then yes ­ we should remove the data after the test is
>>> >>done.
>>> >>
>>> >>     Thanks,
>>> >>
>>> >>     -Dave Easter
>>> >>
>>> >>     From: Dmitry Iakunchikov <diakunchikov@xxxxxxxxxxxx
>>> >>     <mailto:diakunchikov@xxxxxxxxxxxx>>
>>> >>     Date: Friday, January 17, 2014 at 5:10 AM
>>> >>     To: Nadya Privalova <nprivalova@xxxxxxxxxxxx
>>> >>     <mailto:nprivalova@xxxxxxxxxxxx>>, "OpenStack Development Mailing
>>> >>     List (not for usage questions)" <
>>> openstack-dev@xxxxxxxxxxxxxxxxxxx
>>> >>     <mailto:openstack-dev@xxxxxxxxxxxxxxxxxxx>>, Dmitry Iakunchikov
>>> >>     <diakunchikov@xxxxxxxxxxxx <mailto:diakunchikov@xxxxxxxxxxxx>>,
>>> Mike
>>> >>     Scherbakov <mscherbakov@xxxxxxxxxxxx
>>> >>     <mailto:mscherbakov@xxxxxxxxxxxx>>, Vladimir Kuklin
>>> >>     <vkuklin@xxxxxxxxxxxx <mailto:vkuklin@xxxxxxxxxxxx>>,
>>> >>     "fuel-dev@xxxxxxxxxxxxxxxxxxx <mailto:
>>> fuel-dev@xxxxxxxxxxxxxxxxxxx>"
>>> >>     <fuel-dev@xxxxxxxxxxxxxxxxxxx <mailto:
>>> fuel-dev@xxxxxxxxxxxxxxxxxxx>>
>>> >>     Subject: Re: [Fuel-dev] [openstack-dev] [OSTF][Ceilometer]
>>> >>     ceilometer meters and samples delete
>>> >>
>>> >>     For now in Fuel we keep samples forever
>>> >>
>>> >>     In case if we will use time_to_live, how long we should keep this
>>> >>data?
>>> >>
>>> >>
>>> >>     2014/1/17 Julien Danjou <julien@xxxxxxxxxxx
>>> >><mailto:julien@xxxxxxxxxxx>>
>>> >>
>>> >>         On Fri, Jan 17 2014, Nadya Privalova wrote:
>>> >>
>>> >>         > I would ask in another way.
>>> >>         > Ceilometer has a mechanism to add a sample through POST. So
>>> >>it looks
>>> >not
>>> >>         > consistent not to allow user to delete a sample.
>>> >>         > IMHO, insertion and deletion through REST looks a little bit
>>> >>hacky:
>>> >user
>>> >>         > always has an ability to fake data collected from OpenStack
>>> >services. But
>>> >>         > maybe I don't see any valuable usecases.
>>> >>         > Anyway, it seems reasonable to have both add_sample and
>>> >delete_sample in
>>> >>         > API or not to have neither.
>>> >>
>>> >>         From the user PoV, that totally makes sense, agreed.
>>> >>
>>> >>         --
>>> >>         Julien Danjou
>>> >>         # Free Software hacker # independent consultant
>>> >>         # http://julien.danjou.info
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>     --
>>> >>     With Best Regards
>>> >>     QA engineer Dmitry Iakunchikov
>>> >>     -- Mailing list: https://launchpad.net/~fuel-dev Post to :
>>> >>     fuel-dev@xxxxxxxxxxxxxxxxxxx <mailto:fuel-dev@xxxxxxxxxxxxxxxxxxx
>>> >
>>> >>     Unsubscribe : https://launchpad.net/~fuel-dev More help :
>>> >>     https://help.launchpad.net/ListHelp
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> With Best Regards
>>> >> QA engineer Dmitry Iakunchikov
>>> >>
>>> >>
>>> >
>>> >
>>> >--
>>> >Best regards,
>>> >Bogdan Dobrelya,
>>> >Researcher TechLead, Mirantis, Inc.
>>> >+38 (066) 051 07 53
>>> >Skype bogdando_at_yahoo.com
>>> >Irc #bogdando
>>> >38, Lenina ave.
>>> >Kharkov, Ukraine
>>> >www.mirantis.com
>>> >www.mirantis.ru
>>> >bdobrelia@xxxxxxxxxxxx
>>> >
>>>
>>>
>>>
>>> --
>>> Mailing list: https://launchpad.net/~fuel-dev
>>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~fuel-dev
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>
>>
>
>
> --
> With Best Regards
> QA engineer Dmitry Iakunchikov
>