yahoo-eng-team team mailing list archive

[Augustina Ragwitz <1665810@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

This is kind of a halfway between Nova and Infra, but I'm adding it here
for tracking purposes. Basically we have a tool for bug reporting in
Nova that should be hosted in an Infra resource so that we're not
dependent on whoever is running the bugs team to host it. Additionally
other teams might find it useful.

To that end, I spoke with Infra about it and they expressed interest in
having it integrated with their current tools if we wanted to host it.
So this bug is basically figure out how to integrate the nova bugs
dashboard with what's in Infra.

Here's the discussion log
(http://eavesdrop.openstack.org/meetings/infra/2016/infra.2016-12-13-19.03.log.txt):

19:40:41 <fungi> #topic possible hosting for a nova bugs dashboard (auggy)
19:40:47 <fungi> #link http://45.55.105.55:8082/bugs-dashboard.html current nova bugs dashboard
19:41:01 <auggy> yes! we chatted in #openstack-infra about this
19:41:16 <auggy> i just wanted to check on what you all need from me in order to evaluate this request
19:41:22 <fungi> #link http://lists.openstack.org/pipermail/openstack-dev/2016-December/108872.html Useful metrics?
19:41:28 <fungi> (related ml thread)
19:41:58 <auggy> basically we have this dashboard we use that markus wrote and hosted, and we just want it centralized so it's not depending on a single person
19:42:28 <auggy> so i thought i'd ask if it made sense to have you all host it, but if you have other suggestions then I'm open to that too
19:42:54 <fungi> #link https://github.com/markuszoeller/openstack/blob/master/scripts/launchpad/bugs_dashboard.py Creates a HTML file which can be used as a dashboard for cleanup tasks of the bug management.
19:42:57 <auggy> It's a python script that runs via cron on the hour that makes some read only queries to launchpand and then renders the results to html
19:43:05 <auggy> thx fungi !
19:43:29 <auggy> it's also potentially usable by other openstack projects but i'm not sure how custom our bug queries are in that code
19:44:11 <fungi> we have this...
19:44:16 <JayF> auggy: just as a note, Ironic has http://ironic-divius.rhcloud.com -- I don't know where the source is that generates that though. It looks like yours covers our use case + maybe more.
19:44:18 <fungi> #link http://git.openstack.org/cgit/openstack-infra/puppet-bugdaystats/tree/manifests/site.pp
19:44:28 <fungi> could likely call it from an adjacent cron
19:44:53 <jeblair> i think the infrastructure is run by and for all the openstack projects, so these should absolutely be run centrally.  any project should feel free to propose changes to infra systems to add things like this.  :)
19:44:56 <clarkb> my preference would be to incorporate wahtever we do into bugday, we can evolve it if we need to, but seems like every project has their own one of these thigns and if we had to host a different one for each project well thats not really scalable
19:45:12 <jeblair> clarkb: collaboration, if possible would be nice
19:45:19 <clarkb> (neutron had similar things at one point that did get incorporated into bugday fwiw)
19:45:26 <jeblair> clarkb: this seems a little different than bugday though
19:45:38 <fungi> #link https://git.openstack.org/cgit/openstack-infra/bugdaystats
19:45:42 <jeblair> clarkb: are you thinking of the neutron reviewday thing?
19:45:51 <clarkb> jeblair: ya
19:46:07 <fungi> yeah, neutron's thing is a gerrit dashboard url generator run as part of reviewday
19:46:19 <clarkb> ya using bug info as an input iirc
19:46:45 <fungi> well, sure, reviewday itself is also doing lp queries to line reviews up with open bugs, blueprints, et cetera
19:46:57 <auggy> so if you want me to look at bugday or another tool you already have and see about integrating this into that then i can do that and then come back
19:48:10 <fungi> bugday has graphs of bug status changes over time but is currently short-duration and high-frequency updates
19:48:24 <persia> Is the scale issue about execution resources, or maintenance over time?
19:48:29 <clarkb> right, I guess my point is I think its ok to fix deficiencies in bugday so that it works for the larger set of use cases
19:48:37 <clarkb> like neutron has done
19:48:41 <clarkb> and potentially ironic and nova could do
19:48:58 <fungi> persia: more about target audience/use case i think
19:48:58 <jeblair> yeah, any integration with existing tools would be great as collectively we can spend time on making fewer tools better rather than all writing the same tool over
19:49:33 <jeblair> however, i'd rather see us running 5 redundant tools centrally than 5 redundant tools in random hosting places
19:49:46 <persia> So, one tool with per-project views?
19:50:00 <fungi> i think sdague also makes a good point that we have a lot of tooling built around graphite now, and bugday is relatively simple and could stand to be redone in something more inline with our present toolset too
19:50:16 <jeblair> yeah, bugday -> graphite would be pretty easy
19:51:05 <fungi> so maybe this script could seed a new bugstat project of some kind that can repace the bugday use cases as well with some graphs once someone wants to implement that part of it
19:51:59 <auggy> yeah i could work with markus to set it up in an openstack repo under infra or wherever you all think is best
19:52:05 <fungi> with graphite/grafana we can easily accommodate the old bugday use case (what did our squash impact look like in a 24 hour period) along with longer trending bug managers want to see
19:52:34 <persia> With nova workflow, but for selectable project?
19:52:52 <auggy> well that's the work that would have to be done on it
19:53:09 <fungi> it wouldn't be the first time nova's workflow for something got adopted by a ton of other projects ;)
19:53:30 <auggy> i haven't looked to see how specific the lp queries are but potentially one could specify them seperately to match their bugs
19:53:48 <fungi> so i'm not going to push for adding more to bugday unless this is the start of a bugday v2 reimplementation anyway
19:54:33 <auggy> eg, what queries do you want to go with which tabs kind of thing
19:54:33 <auggy> kk how about i work with markus to set this up as an openstack-infra repo project thingy?
19:54:33 <auggy> and then do some work to make it more project universal?
19:54:33 <fungi> but running this from a cron in the bugdaystats::site class and adding a tab on status.o.o for it or whatever seems fine to me
19:55:19 <fungi> auggy: i think that sounds like a fine plan
19:55:29 <fungi> anyone disagree?
19:56:02 <jeblair> sgtm
19:56:16 <pabelanger> ++
19:56:23 <auggy> thanks :)
19:56:24 <fungi> #agreed The current Nova bug dashboard authors are welcome to import their work into a new Infra repo and then work on making it generalized for other projects' use.
19:56:56 <fungi> once that step is done, we can get more into the weeds on how to automate deployment and where to link/display it

** Affects: nova
     Importance: Undecided
     Assignee: Maciej Szankin (mszankin)
         Status: Confirmed

** Changed in: nova
       Status: New => Confirmed

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1665810

Title:
  Nova Bugs Dashboard should be hosted by Infra

Status in OpenStack Compute (nova):
  Confirmed

Bug description:
  This is kind of a halfway between Nova and Infra, but I'm adding it
  here for tracking purposes. Basically we have a tool for bug reporting
  in Nova that should be hosted in an Infra resource so that we're not
  dependent on whoever is running the bugs team to host it. Additionally
  other teams might find it useful.

  To that end, I spoke with Infra about it and they expressed interest
  in having it integrated with their current tools if we wanted to host
  it. So this bug is basically figure out how to integrate the nova bugs
  dashboard with what's in Infra.

  Here's the discussion log
  (http://eavesdrop.openstack.org/meetings/infra/2016/infra.2016-12-13-19.03.log.txt):

  19:40:41 <fungi> #topic possible hosting for a nova bugs dashboard (auggy)
  19:40:47 <fungi> #link http://45.55.105.55:8082/bugs-dashboard.html current nova bugs dashboard
  19:41:01 <auggy> yes! we chatted in #openstack-infra about this
  19:41:16 <auggy> i just wanted to check on what you all need from me in order to evaluate this request
  19:41:22 <fungi> #link http://lists.openstack.org/pipermail/openstack-dev/2016-December/108872.html Useful metrics?
  19:41:28 <fungi> (related ml thread)
  19:41:58 <auggy> basically we have this dashboard we use that markus wrote and hosted, and we just want it centralized so it's not depending on a single person
  19:42:28 <auggy> so i thought i'd ask if it made sense to have you all host it, but if you have other suggestions then I'm open to that too
  19:42:54 <fungi> #link https://github.com/markuszoeller/openstack/blob/master/scripts/launchpad/bugs_dashboard.py Creates a HTML file which can be used as a dashboard for cleanup tasks of the bug management.
  19:42:57 <auggy> It's a python script that runs via cron on the hour that makes some read only queries to launchpand and then renders the results to html
  19:43:05 <auggy> thx fungi !
  19:43:29 <auggy> it's also potentially usable by other openstack projects but i'm not sure how custom our bug queries are in that code
  19:44:11 <fungi> we have this...
  19:44:16 <JayF> auggy: just as a note, Ironic has http://ironic-divius.rhcloud.com -- I don't know where the source is that generates that though. It looks like yours covers our use case + maybe more.
  19:44:18 <fungi> #link http://git.openstack.org/cgit/openstack-infra/puppet-bugdaystats/tree/manifests/site.pp
  19:44:28 <fungi> could likely call it from an adjacent cron
  19:44:53 <jeblair> i think the infrastructure is run by and for all the openstack projects, so these should absolutely be run centrally.  any project should feel free to propose changes to infra systems to add things like this.  :)
  19:44:56 <clarkb> my preference would be to incorporate wahtever we do into bugday, we can evolve it if we need to, but seems like every project has their own one of these thigns and if we had to host a different one for each project well thats not really scalable
  19:45:12 <jeblair> clarkb: collaboration, if possible would be nice
  19:45:19 <clarkb> (neutron had similar things at one point that did get incorporated into bugday fwiw)
  19:45:26 <jeblair> clarkb: this seems a little different than bugday though
  19:45:38 <fungi> #link https://git.openstack.org/cgit/openstack-infra/bugdaystats
  19:45:42 <jeblair> clarkb: are you thinking of the neutron reviewday thing?
  19:45:51 <clarkb> jeblair: ya
  19:46:07 <fungi> yeah, neutron's thing is a gerrit dashboard url generator run as part of reviewday
  19:46:19 <clarkb> ya using bug info as an input iirc
  19:46:45 <fungi> well, sure, reviewday itself is also doing lp queries to line reviews up with open bugs, blueprints, et cetera
  19:46:57 <auggy> so if you want me to look at bugday or another tool you already have and see about integrating this into that then i can do that and then come back
  19:48:10 <fungi> bugday has graphs of bug status changes over time but is currently short-duration and high-frequency updates
  19:48:24 <persia> Is the scale issue about execution resources, or maintenance over time?
  19:48:29 <clarkb> right, I guess my point is I think its ok to fix deficiencies in bugday so that it works for the larger set of use cases
  19:48:37 <clarkb> like neutron has done
  19:48:41 <clarkb> and potentially ironic and nova could do
  19:48:58 <fungi> persia: more about target audience/use case i think
  19:48:58 <jeblair> yeah, any integration with existing tools would be great as collectively we can spend time on making fewer tools better rather than all writing the same tool over
  19:49:33 <jeblair> however, i'd rather see us running 5 redundant tools centrally than 5 redundant tools in random hosting places
  19:49:46 <persia> So, one tool with per-project views?
  19:50:00 <fungi> i think sdague also makes a good point that we have a lot of tooling built around graphite now, and bugday is relatively simple and could stand to be redone in something more inline with our present toolset too
  19:50:16 <jeblair> yeah, bugday -> graphite would be pretty easy
  19:51:05 <fungi> so maybe this script could seed a new bugstat project of some kind that can repace the bugday use cases as well with some graphs once someone wants to implement that part of it
  19:51:59 <auggy> yeah i could work with markus to set it up in an openstack repo under infra or wherever you all think is best
  19:52:05 <fungi> with graphite/grafana we can easily accommodate the old bugday use case (what did our squash impact look like in a 24 hour period) along with longer trending bug managers want to see
  19:52:34 <persia> With nova workflow, but for selectable project?
  19:52:52 <auggy> well that's the work that would have to be done on it
  19:53:09 <fungi> it wouldn't be the first time nova's workflow for something got adopted by a ton of other projects ;)
  19:53:30 <auggy> i haven't looked to see how specific the lp queries are but potentially one could specify them seperately to match their bugs
  19:53:48 <fungi> so i'm not going to push for adding more to bugday unless this is the start of a bugday v2 reimplementation anyway
  19:54:33 <auggy> eg, what queries do you want to go with which tabs kind of thing
  19:54:33 <auggy> kk how about i work with markus to set this up as an openstack-infra repo project thingy?
  19:54:33 <auggy> and then do some work to make it more project universal?
  19:54:33 <fungi> but running this from a cron in the bugdaystats::site class and adding a tab on status.o.o for it or whatever seems fine to me
  19:55:19 <fungi> auggy: i think that sounds like a fine plan
  19:55:29 <fungi> anyone disagree?
  19:56:02 <jeblair> sgtm
  19:56:16 <pabelanger> ++
  19:56:23 <auggy> thanks :)
  19:56:24 <fungi> #agreed The current Nova bug dashboard authors are welcome to import their work into a new Infra repo and then work on making it generalized for other projects' use.
  19:56:56 <fungi> once that step is done, we can get more into the weeds on how to automate deployment and where to link/display it

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1665810/+subscriptions