graphite-dev team mailing list archive

[Dieter P <question223956@xxxxxxxxxxxxxxxxxxxxx>]

Question #223956 on Graphite changed:
https://answers.launchpad.net/graphite/+question/223956

Dieter P posted a new comment:
> For us data geeks, "as.we.are.used.to" is perfectly fine, but if you want to share the information with less technical users,
> having an optional name override is very useful. Or if you abbreviate a metric, e.g. "if.tx.eth0", it would be nice to display on
> the graph "Interface Transmits: Ethernet 0". I'm thinking of the broader audience. For situations where no display name is
> provided, everything just defaults to the node name.

ok so, in accordance with the later section about tag-based naming and
context-based name computation, we could just provide a mapping for
every tag to a "friendly" version: friendlies = {'if': 'interface',
'tx': 'transmits', 'eth0': 'Ethernet 0'}, this way we can still leverage
the name computation, but only with friendly names. (note: this probably
sounds harder then it really is, i'm doing this stuff with G-E and it's
pretty easy to generate names in basically any format you want)

> It's very useful for documenting things like: Where the data came from, who's in charge of the data source, what the data
> means, etc. When you have millions of unique timeseries and many users working with the system, this kind of data can be
> invaluable and I think Graphite users could benefit from having it as well. But no one is forced to use it.

OK, but that doesn't translate into any particular datastructures
AFAICT.  One could, for example, have a pure tag-based metrics database
(without the extra metadata), and a separate database for additional
metadata by merely referencing the tag keys/values.

> The "name" field is only used in the general UID meta object for OpenTSDB and reflects either a METRIC, a TAGK or a TAGV
> name so it's valid in this situation but you're absolutely right that it's a terrible tag name, e.g. "name=something".
> OpenTSDB's schema is more efficient when you have fewer tags, (...)

ok so basically it seems you're using it only as a hack because things get slow if you have too many tags ;)
I still stand by my "split everything into tags, and avoid a 'name' tag/attribute" stance. we just need a decent database that can deal with tags properly

> (...) the "units" field. It's something that could be bolted on to Graphite directly so that folks could upgrade and
> fill out the meta without having to rename their existing data files.

IMHO all tags are an intrinsic part of the identity of a metric.  if my code is reporting a metric in bits, but then i change it to report in bytes, then i just want to update the tag in the same place i'm reporting from; so that any metric submitted always has the correct unit that applies to that metric:
* less work (no need to update the unit somewhere else)
* no possibility that the wrong unit is used when looking at a different timerange then what the unit applies for (or a timerange with the unit switch in the middle)
* no ambiguity in tools that work with the data *before* it hits the storage system. for example statsd has a backend that feeds a websocket in addition to graphite, so you can view data as it gets generated on the originating host.
the fact that you get a different metric if you change the unit is not a problem, if you query for "tx eth0 server123 in bits" both metrics would match and both would show up (and one of them gets automatically scaled), the UI is transparant to this.

BTW to avoid any confusion, i'm not an official graphite dev, i just
contributed code and worked on a bunch of related projects.  the
opinions expressed are mine, not of the graphite devs (though i will try
to convince/discuss (with) them, based on what i learned with G-E :-)

-- 
You received this question notification because you are a member of
graphite-dev, which is an answer contact for Graphite.