fuel-dev team mailing list archive

[Vladimir Kozhukalov <vkozhukalov@xxxxxxxxxxxx>]

Sergey,

Great initiative. If you have thoughts about how we can improve Fuel
"diagnostic snapshot" feature, let's discuss it in our maling list
fuel-dev@xxxxxxxxxxxxxxxxxxx or on our weekly IRC meeting
https://wiki.openstack.org/wiki/Meetings/Fuel.

Adding some commands to save their output is simple. One can add whatever
they need into /etc/nailgun/settings.yaml file. Upgrading this tool on the
fly is much more complicated improvement. Besides, our current solution has
some problems with the ability to gather diagnostic data from a number of
nodes in parallel. At the moment it is just one thread which runs one
command on one remote node at each moment.

We also thought a bit about how to filter out sensitive data from
diagnostic snapshot, but we've not managed to find a suitable solution. At
the moment all passwords are stored in a database as a plain text. So we
tried to find all passwords wherever they appear and substitute them with a
meaningless string. Because passwords maybe something like "1" or "admin",
it is not so simple to filter them out without breaking other useful data.

We also discussed the initiative which is about create shell wrapper for
arbitrary command which would be able to save stdout and stderr into files
or publish them on a paste service.

Anyway, please describe as much clearly as you can how you feel you can
integrate parts of your troubleshooting tool into Fuel diagnostic snapshot
tool.


Vladimir Kozhukalov


On Thu, Feb 27, 2014 at 4:43 PM, Matthew Mosesohn <mmosesohn@xxxxxxxxxxxx>wrote:

> Any log/snapshot submission system (like anonymous FTP upload) should
> accept anonymous uploads (but not anonymous read/list) for user ease
> of use.
>
> I believe anything extra beyond our current logging implementation
> should involve manual command workarounds for data gathering. For
> example, debugging glance-api with logs redirected for 5 minutes:
> service openstack-glance-api stop
> mkdir /root/glance-capture-logs
> timeout 5m glance-api -v -d --log-file
> /root/glance-capture-logs/glance-api.log
> service openstack-glance-api start
>
> Then collector will pull all files out of /root, for example..
>
>
>
> On Thu, Feb 27, 2014 at 4:36 PM, Pavel Vaylov <pvaylov@xxxxxxxxxxxx>
> wrote:
> > 1. Absolutely need a way to upgrade/fix log collection process, not only
> > snapshot tool. For example this
> https://bugs.launchpad.net/fuel/+bug/1284867
> > and probably new issues with logging.
> >
> > 2. Sergey, as I informed we have a solution for it, Andrey Ryabinin can
> > confirm (we bought subscription to some file sharing service). But the
> > solution is require to many manual steps from the Operation engineer.
> >
> > 3. Will test the tool on my Lab.
> >
> >
> >
> >
> > On Thu, Feb 27, 2014 at 2:19 PM, Sergey Yudin <syudin@xxxxxxxxxxxx>
> wrote:
> >>
> >> Hi, guys.
> >> Being part of TET team, i noticed that most difficult and long part of
> >> troubleshotting is getting debug information from the customer, and i
> want
> >> improve this.
> >>
> >> I wanna discuss fuel snapshot tool, and related blueprint on launchpad.
> >>
> https://blueprints.launchpad.net/fuel/+spec/fuel-snapshot-additional-info
> >>
> >>
> >> From my point of view the tools not only has lack of information, as
> >> mentioned in the blueprint, but also has lack of some base features
> which
> >> will make it usefull.
> >>
> >> 1. We need ability to upgrade this tool on fly, so if customer will face
> >> some issue support team will be able to send updated snapshot tool, or
> even
> >> fix exist tool for getting required information, and not rely on tool
> which
> >> was written years ago.
> >>
> >> 2. We need to think of storage for snapshots, which may be huge, so will
> >> not fit onto email, and also may contain private information, so will
> not
> >> fit for public storage.
> >>
> >> 3. There is tool written by me for gathering information from customer
> >> installation,
> >>
> >>
> https://docs.google.com/a/mirantis.com/document/d/1KyVuJcKy6lln4rI1d2kEZS63NU1ykswz0ib4MoM5bog/edit#
> >> https://github.com/tsipa/troubleshooting
> >>  maybe we should use it. I'm agree to participate in integration it into
> >> fuel snapshot tool.
> >>
> >> Maybe i missed something and guys from support team has something more
> to
> >> say.
> >
> >
> >
> >
> > --
> > Best regards
> >
> > Pavel Vaylov
> > Operations Engineer
> > Mirantis Service Desk
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "fuel-core-team" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to fuel-core-team+unsubscribe@xxxxxxxxxxxx.
> > For more options, visit
> > https://groups.google.com/a/mirantis.com/groups/opt_out.
>
> --
> You received this message because you are subscribed to the Google Groups
> "fuel-core-team" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to fuel-core-team+unsubscribe@xxxxxxxxxxxx.
> For more options, visit
> https://groups.google.com/a/mirantis.com/groups/opt_out.
>