fuel-dev team mailing list archive

[Bogdan Dobrelya <bdobrelia@xxxxxxxxxxxx>]

*Preamble*
Hi stackers, I was trying to implement correct APP-NAME tags for remote 
logging in Fuel for Openstack, and faced the 
https://bugs.launchpad.net/nova/+bug/904307 issue. There are no logging 
options in Python 2.6/2.7 to address this APP-NAME in logging formats or 
configs (log_format, log_config(_append)).

Just look at the log file names, and you will understand me:
cinder-cinder.api.extensions.log
cinder-cinder.db.sqlalchemy.session.log
cinder-cinder.log
cinder-cinder.openstack.common.rpc.common.log
cinder-eventlet.wsgi.server.log
cinder-keystoneclient.middleware.auth_token.log
glance-eventlet.wsgi.server.log
glance-glance.api.middleware.cache.log
glance-glance.api.middleware.cache_manage.log
glance-glance.image_cache.log
glance-keystoneclient.middleware.auth_token.log
keystone-root.log
nova-keystoneclient.middleware.auth_token.log
nova-nova.api.openstack.compute.extensions.log
nova-nova.api.openstack.extensions.log
nova-nova.ec2.wsgi.server.log
nova-nova.log
nova-nova.metadata.wsgi.server.log
nova-nova.network.driver.log
nova-nova.osapi_compute.wsgi.server.log
nova-nova.S3.log
quantum-eventlet.wsgi.server.log
quantum-keystoneclient.middleware.auth_token.log
quantum-quantum.api.extensions.log
quantum-quantum.manager.log
quantum-quantum.openstack.common.rpc.amqp.log
quantum-quantum.plugins.openvswitch.ovs_quantum_plugin.log

But I actually want to see something like this:
cinder-api.log
cinder-volume.log
glance-api.log
glance-manage.log
glance-registry.log
keystone-all.log
nova-api.log
nova-conductor.log
nova-consoleauth.log
nova-objectstore.log
nova-scheduler.log
...and so on.

Another words, logging should honor RFC3164 & RFC5424, here are some quotes:
"The MSG part has two fields known as the TAG field and the CONTENT
field. The value in the TAG field will be the name of the program or 
process that generated the message. The CONTENT contains the details of 
the message..."
"The APP-NAME field SHOULD identify the device or application that
originated the message..."

I see two solutions for this issue.

*Solution 1*
The one of possible solutions is to use new key for log_format (i.e. 
%(binary_name)s) to extract application/service name for log records.
The implementation could be like patch #4: 
https://review.openstack.org/#/c/63094/4
And the log_format could be like this:
log_format=%(asctime)s %(binary_name)s %(levelname)s: %(name)s: %(message)s

The patch is applicable to other Openstack services, which did not moved 
to Oslo yet.
I tested it with nova services, and all services can start with 
log_format using %(binary_name)s, but nova-api. Looks like 
/keystoneclient/middleware/auth_token.py is unhappy with this patch, see 
the trace http://paste.openstack.org/show/55519/

*Solution 2*
The other and only option I can suggest, is to backport 'ident' from 
python 3.3, see http://hg.python.org/cpython/rev/6baa90fa2b6d
The implementation could be like this: 
https://review.openstack.org/#/c/63094
To ensure we will have APP-NAME in message we can set use_syslog = true 
and check the results.
If we're using log_config_append, the formatters and handlers could be 
like this:
[formatter_normal]
format = %(levelname)s: %(message)s
[handler_production]
class = openstack.common.log.RFCSysLogHandler
level = INFO
formatter = normal
args = ('/dev/log', handlers.SysLogHandler.LOG_LOCAL6)

The patch is also applicable to other Openstack services, which did not 
moved to Oslo yet.
For syslog logging, the application/service/process name (aka APP-NAME, 
see RFC5424) would be added before the MSG part, right after it has been 
formatted, and there is no need to use any special log_format settings 
as well.

*Conclusion*
I vote for implement solution 2 for Oslo logging, and for those 
Openstack services, which don't use Oslo for logging yet. That would not 
require any changes outside of the Openstack modules, thus looks like a 
good compromise for backporting 'ident' feature for APP-NAME tags from 
Python 3.3. What do you think?

P.S. Sorry for spamming, fuel-dev, have a nice weekend :-)
--
Best regards,
Bogdan Dobrelya,
Researcher TechLead, Mirantis, Inc.
+38 (066) 051 07 53
Skype bogdando_at_yahoo.com
Irc #bogdando
38, Lenina ave.
Kharkov, Ukraine
www.mirantis.com
www.mirantis.ru
bdobrelia@xxxxxxxxxxxx