openstack team mailing list archive

[Chu Duc Minh <chu.ducminh@xxxxxxxxx>]

On Thu, Jul 25, 2013 at 7:30 PM, Jay Pipes <jaypipes@xxxxxxxxx> wrote:

> You will need to provide more details than "old" vs. "new" OpenStack.
>
> For example...
>
> 1) What is your network model in the old vs. new
>
Old: nova-network + FlatDHCP
New: Quantum + OpenvSwitch + network-namespace + metadata-ns-proxy

2) What version of OpenStack is the old
>
Old: Essex
New: Grizzly

3) Is Keystone used in old and new? If so, what drivers are used in
> Keystone?
>
Yes, both use Keystone with SQL backend

4) Do you have errors in any of your log files (usually an indication that
> something like a timeout or failure on

RPC which may affect performance)
>
No, i didn't found it.


> 5) Are you using nova-conductor in the new?
>
I have nova-conductor process run on Controller node, but seem nova-compute
don't use this.


> 6) What database backend are you using?
>
MySQL.


> 7) Do a time keystone user-list on both old and new
>
# keystone user-list
Old:
Run 1st time:
real    *0m2.494s*
user    0m0.112s
sys     0m0.040s
2nd time:
real    *0m0.279s*
user    0m0.112s
sys     0m0.020s


New:
Run 1st time:
real    *0m1.687s*
user    0m0.176s
sys     0m0.012s
2nd time:
real    *0m0.213s*
user    0m0.160s
sys     0m0.040s


8) Pastebin your conf files, with passwords removed
>
My config files are quite large.
I can sum up that all services & API are HA-ed using HAproxy, RabbitMQ
Cluster, Database replication, v.v...

This morning, i just do some tuning memcached for Dashboard service (HA-ed
using HAProxy), then my Dashboard become faster significantly.

The questions remain are:
1. Why metadata query so slow? Possible problems? ( "curl
http://169.254.169.254/openstack"; took 2.5-5 seconds to complete,
"ec2metadata" took 14-17 seconds to complete - i tested many times.)
2. Why API calls using *-client reduce time from 2nd time on my old Essex
deployment, but not on my new Grizzly deployment? Maybe I need some "cache"
settings?

Thank you very much!


> The more information you give, the better folks can help you.
>
> Best,
> -jay
>
>
> On 07/25/2013 07:14 AM, Chu Duc Minh wrote:
>
>> Check some more API (I run these command below from Controller node):
>> # time quantum subnet-list
>> (...have 4 subnet)
>> real    0m0.676s
>> user    0m0.196s
>> sys     0m0.020s
>>
>> # time quantum router-list
>> (...have 1 router)
>> real    0m0.496s
>> user    0m0.164s
>> sys     0m0.052s
>>
>> # time nova list --all_tenants=1
>> (...have 5 instances)
>> real    0m1.290s
>> user    0m0.308s
>> sys     0m0.040s
>>
>> Compare with my old OpenStack deployment on weaker servers, it took 1/3
>> in times.
>>
>>
>>
>> On Thu, Jul 25, 2013 at 5:43 PM, Peter Cheung <mcheung63@xxxxxxxxxxx
>> <mailto:mcheung63@xxxxxxxxxxx>**> wrote:
>>
>>     I am having a problem about calling API speed is up and down,
>>     something need 0.1s, something it needs 3s
>>
>>     Thanks
>>     from Peter
>>
>>
>>     ------------------------------**------------------------------**
>> ------------
>>
>>     Date: Thu, 25 Jul 2013 17:41:11 +0700
>>     From: chu.ducminh@xxxxxxxxx <mailto:chu.ducminh@xxxxxxxxx>
>>     To: openstack@xxxxxxxxxxxxxxxxxxx
>>     <mailto:openstack@lists.**launchpad.net<openstack@xxxxxxxxxxxxxxxxxxx>
>> >;
>>     openstack@xxxxxxxxxxxxxxxxxxx <mailto:openstack@lists.**openstack.org<openstack@xxxxxxxxxxxxxxxxxxx>
>> >
>>
>>     Subject: [Openstack] Call to API very slow [Grizzly]
>>
>>
>>     All operations in my Openstack dashboard very slow (compare to my
>>     old Openstack deployment)
>>     Then i do some check on an instance:
>>
>>     $ time curl http://169.254.169.254/**openstack<http://169.254.169.254/openstack>
>>     2012-08-10
>>     2013-04-04
>>     latest
>>     real    0m*5.605s*
>>
>>     user    0m0.004s
>>     sys    0m0.004s
>>
>>     5 seconds for a simple API query !??
>>
>>
>>     in quantum-ns-metadata-proxyxxxx.**log, i saw:
>>     2013-07-25 *17:17:09 *  DEBUG
>>
>>     [quantum.agent.metadata.**namespace_proxy] Request: GET /openstack
>>     HTTP/1.0
>>     Accept: */*
>>     Content-Type: text/plain
>>     Host: 169.254.169.254
>>     User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0
>>     OpenSSL/1.0.1 zlib/1.2.3.4 <http://1.2.3.4> libidn/1.23 librtmp/2.3
>>     2013-07-25 *17:17:14*    DEBUG
>>
>>     [quantum.agent.metadata.**namespace_proxy] {'date': 'Thu, 25 Jul 2013
>>     10:17:14 GMT', 'status': '200', 'content-length': '28',
>>     'content-type': 'text/html; charset=UTF-8', 'content-location':
>>     u'http://169.254.169.254/**openstack<http://169.254.169.254/openstack>
>> '}
>>     2013-07-25 17:17:14    DEBUG
>>     [quantum.agent.metadata.**namespace_proxy] 2012-08-10
>>     2013-04-04
>>     latest
>>
>>     I take a look at metadata-agent.log, and saw almost request/response
>>     finished @*17:17:09
>>     *
>>     But the last finished *@**17:17:14
>>     *2013-07-25 *17:17:14*    DEBUG [quantum.agent.metadata.agent]
>>
>>     {'date': 'Thu, 25 Jul 2013 10:17:14 GMT', 'status': '200',
>>     'content-length': '28', 'content-type': 'text/html; charset=UTF-8',
>>     'content-location': u'http://172.30.1.14:8775/**openstack<http://172.30.1.14:8775/openstack>
>> '}
>>     *
>>     *
>>
>>     I enabled slow query log on MySql, but can't find any slow query.
>>
>>     Do you know possible problems in this situation?
>>     Thank you very much!
>>
>>
>>     ______________________________**_________________ Mailing list:
>>     https://launchpad.net/~**openstack <https://launchpad.net/~openstack>Post to :
>>     openstack@xxxxxxxxxxxxxxxxxxx <mailto:openstack@lists.**launchpad.net<openstack@xxxxxxxxxxxxxxxxxxx>
>> >
>>
>>     Unsubscribe : https://launchpad.net/~**openstack<https://launchpad.net/~openstack>More help :
>>     https://help.launchpad.net/**ListHelp<https://help.launchpad.net/ListHelp>
>>
>>
>>
>>
>> ______________________________**_________________
>> Mailing list: https://launchpad.net/~**openstack<https://launchpad.net/~openstack>
>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~**openstack<https://launchpad.net/~openstack>
>> More help   : https://help.launchpad.net/**ListHelp<https://help.launchpad.net/ListHelp>
>>
>>
>
> ______________________________**_________________
> Mailing list: https://launchpad.net/~**openstack<https://launchpad.net/~openstack>
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~**openstack<https://launchpad.net/~openstack>
> More help   : https://help.launchpad.net/**ListHelp<https://help.launchpad.net/ListHelp>
>