fuel-dev team mailing list archive

[Vladimir Kuklin <vkuklin@xxxxxxxxxxxx>]

Let's push our OSCI team to do it.


On Wed, Feb 5, 2014 at 4:23 PM, Bogdan Dobrelya <bdobrelia@xxxxxxxxxxxx>wrote:

> - openstack-dev
> + fuel-core team
>
> Hello. In order to make possible to implement logging improvements
> (
> https://blueprints.launchpad.net/fuel/+spec/refactor-logging-puppet-openstack-services
> )
> in Fuel manifests for puppet-openstack modules, I'd like to suggest us:
> 1) do not wait for Oslo patch for
> https://bugs.launchpad.net/nova/+bug/904307 will be included for all
> Openstack projects in the later I releases (if it will...). Why? There
> is a chance some Openstack projects (deployed by Fuel as well) will not
> sync this patch from Oslo in I, or even J, despite it had been accepted
> in Oslu-incubatuor, thus - we could not provide then new logging
> 'use_syslog_rfc_format' config option for such services.
> (Just for note, this option is heavily needed to reflect app/service
> names in Fuel UI logging section.)
>
> 2) improve our hardened Mirantis Openstack packages by including this
> patch from Oslo-incubator by our own initiative, for any release we
> want. I believe OSCI team can do this (see How incubation works in Oslo
> https://wiki.openstack.org/wiki/Oslo#Incubation).
>
> What do you think?
>
> -------- Original Message --------
> Subject: Fw: [Openstack-dev] [Fuel] [Oslo] Add APP-NAME (RFC5424) for
> Oslo syslog logging
> Date: Mon, 03 Feb 2014 16:58:40 +0200
> From: Bogdan Dobrelya <bdobrelia@xxxxxxxxxxxx>
> Organization: Mirantis
> To: openstack-dev@xxxxxxxxxxxxxxxxxxx, fuel-dev@xxxxxxxxxxxxxxxxxxx
>
> On 12/20/2013 08:21 PM, Bogdan Dobrelya wrote:
> > *Preamble*
> > Hi stackers, I was trying to implement correct APP-NAME tags for remote
> > logging in Fuel for Openstack, and faced the
> > https://bugs.launchpad.net/nova/+bug/904307 issue. There are no logging
> > options in Python 2.6/2.7 to address this APP-NAME in logging formats or
> > configs (log_format, log_config(_append)).
> >
> > Just look at the log file names, and you will understand me:
> > cinder-cinder.api.extensions.log
> > cinder-cinder.db.sqlalchemy.session.log
> > cinder-cinder.log
> > cinder-cinder.openstack.common.rpc.common.log
> > cinder-eventlet.wsgi.server.log
> > cinder-keystoneclient.middleware.auth_token.log
> > glance-eventlet.wsgi.server.log
> > glance-glance.api.middleware.cache.log
> > glance-glance.api.middleware.cache_manage.log
> > glance-glance.image_cache.log
> > glance-keystoneclient.middleware.auth_token.log
> > keystone-root.log
> > nova-keystoneclient.middleware.auth_token.log
> > nova-nova.api.openstack.compute.extensions.log
> > nova-nova.api.openstack.extensions.log
> > nova-nova.ec2.wsgi.server.log
> > nova-nova.log
> > nova-nova.metadata.wsgi.server.log
> > nova-nova.network.driver.log
> > nova-nova.osapi_compute.wsgi.server.log
> > nova-nova.S3.log
> > quantum-eventlet.wsgi.server.log
> > quantum-keystoneclient.middleware.auth_token.log
> > quantum-quantum.api.extensions.log
> > quantum-quantum.manager.log
> > quantum-quantum.openstack.common.rpc.amqp.log
> > quantum-quantum.plugins.openvswitch.ovs_quantum_plugin.log
> >
> > But I actually want to see something like this:
> > cinder-api.log
> > cinder-volume.log
> > glance-api.log
> > glance-manage.log
> > glance-registry.log
> > keystone-all.log
> > nova-api.log
> > nova-conductor.log
> > nova-consoleauth.log
> > nova-objectstore.log
> > nova-scheduler.log
> > ...and so on.
> >
> > Another words, logging should honor RFC3164 & RFC5424, here are some
> quotes:
> > "The MSG part has two fields known as the TAG field and the CONTENT
> > field. The value in the TAG field will be the name of the program or
> > process that generated the message. The CONTENT contains the details of
> > the message..."
> > "The APP-NAME field SHOULD identify the device or application that
> > originated the message..."
> >
> > I see two solutions for this issue.
> >
> > *Solution 1*
> > The one of possible solutions is to use new key for log_format (i.e.
> > %(binary_name)s) to extract application/service name for log records.
> > The implementation could be like patch #4:
> > https://review.openstack.org/#/c/63094/4
> > And the log_format could be like this:
> > log_format=%(asctime)s %(binary_name)s %(levelname)s: %(name)s:
> %(message)s
> >
> > The patch is applicable to other Openstack services, which did not moved
> > to Oslo yet.
> > I tested it with nova services, and all services can start with
> > log_format using %(binary_name)s, but nova-api. Looks like
> > /keystoneclient/middleware/auth_token.py is unhappy with this patch, see
> > the trace http://paste.openstack.org/show/55519/
> >
> > *Solution 2*
> > The other and only option I can suggest, is to backport 'ident' from
> > python 3.3, see http://hg.python.org/cpython/rev/6baa90fa2b6d
> > The implementation could be like this:
> > https://review.openstack.org/#/c/63094
> > To ensure we will have APP-NAME in message we can set use_syslog = true
> > and check the results.
> > If we're using log_config_append, the formatters and handlers could be
> > like this:
> > [formatter_normal]
> > format = %(levelname)s: %(message)s
> > [handler_production]
> > class = openstack.common.log.RFCSysLogHandler
> > level = INFO
> > formatter = normal
> > args = ('/dev/log', handlers.SysLogHandler.LOG_LOCAL6)
> >
> > The patch is also applicable to other Openstack services, which did not
> > moved to Oslo yet.
> > For syslog logging, the application/service/process name (aka APP-NAME,
> > see RFC5424) would be added before the MSG part, right after it has been
> > formatted, and there is no need to use any special log_format settings
> > as well.
> >
> > *Conclusion*
> > I vote for implement solution 2 for Oslo logging, and for those
> > Openstack services, which don't use Oslo for logging yet. That would not
> > require any changes outside of the Openstack modules, thus looks like a
> > good compromise for backporting 'ident' feature for APP-NAME tags from
> > Python 3.3. What do you think?
> >
>
> Hi, stackers.
> FYI, the patch https://bugs.launchpad.net/oslo/+bug/904307 had been
> merged into Oslo-incubator to implement Option 2 (see above).
>
> According to the docs provided, to honor RFC5424 APP-NAME for syslog
> messages, the existing syslog format is DEPRECATED during I and will be
> removed in J.
>
> New use_syslog_rfc_format option was introduced:
> (Optional) use syslog rfc5424 format for logging. If enabled, will add
> APP-NAME (RFC5424) before the MSG part of the syslog message.  The old
> format without APP-NAME is deprecated in I, and will be removed in J.
>
> 1) Please consider to sync the patch to any affected Openstack projects
> (i.e. all projects with use_syslog option in their configs) to ensure
> the application/service name will be present in the syslog messages, if
> use_syslog_rfc_format = true.
>
> 2) Please consider to add this info to the Syslog section of the
> https://wiki.openstack.org/wiki/LoggingStandards
>
> > --
> > Best regards,
> > Bogdan Dobrelya,
> > Researcher TechLead, Mirantis, Inc.
> > +38 (066) 051 07 53
> > Skype bogdando_at_yahoo.com
> > Irc #bogdando
> > 38, Lenina ave.
> > Kharkov, Ukraine
> > www.mirantis.com
> > www.mirantis.ru
> > bdobrelia@xxxxxxxxxxxx
> >
>
>
> --
> Best regards,
> Bogdan Dobrelya,
> Researcher TechLead, Mirantis, Inc.
> +38 (066) 051 07 53
> Skype bogdando_at_yahoo.com
> Irc #bogdando
> 38, Lenina ave.
> Kharkov, Ukraine
> www.mirantis.com
> www.mirantis.ru
> bdobrelia@xxxxxxxxxxxx
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "fuel-core-team" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to fuel-core-team+unsubscribe@xxxxxxxxxxxx.
> For more options, visit
> https://groups.google.com/a/mirantis.com/groups/opt_out.
>



-- 
Yours Faithfully,
Vladimir Kuklin,
Senior Deployment Engineer,
Mirantis, Inc.
+7 (495) 640-49-04
+7 (926) 702-39-68
Skype kuklinvv
45bk3, Vorontsovskaya Str.
Moscow, Russia,
www.mirantis.com <http://www.mirantis.ru/>
www.mirantis.ru
vkuklin@xxxxxxxxxxxx