yahoo-eng-team team mailing list archive

[Chenjun Shen <1723142@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

We use openstack-ansbile project (branch 14.1.0, so basically newton) to
deploy our openstack cloud.

How to reproduce problem:
In our case, we have a big project, which has 400+ servers, and 26 networks.
When we click project/network/network topology page, it times out.

In the code, we found a scalability problem. Please see these lines:
https://github.com/openstack/horizon/blob/master/openstack_dashboard/dashboards/project/network_topology/views.py#L251-L259

When loading network topology page (in _get_servers function), it
iterates servers variable, and for each running server(instance), will
make a i_console.get_console() call, and it's only sequentially.

Imagine you have 1000 running servers in the project, each get_console()
takes 200 milliseconds. So in the end in order to get all consoles, it
will cost you 1000 x 200 (milliseconds) = 200 seconds. In openstack-
ansible project, the default timeout for haproxy is only 50. So in the
end it already times out for the user.

Something even worse, in the network topology diagram, it seems to have
an auto refresh enabled. So every 20-30(probably), it fires all the
requests to backend again.

I might have some suggestions.
Either we make this console link in network topology configurable,
or we fire the requests concurrently to backend?

Thanks,
Chenjun Shen

** Affects: horizon
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Dashboard (Horizon).
https://bugs.launchpad.net/bugs/1723142

Title:
  network topology page doesn't scale

Status in OpenStack Dashboard (Horizon):
  New

Bug description:
  We use openstack-ansbile project (branch 14.1.0, so basically newton)
  to deploy our openstack cloud.

  How to reproduce problem:
  In our case, we have a big project, which has 400+ servers, and 26 networks.
  When we click project/network/network topology page, it times out.

  In the code, we found a scalability problem. Please see these lines:
  https://github.com/openstack/horizon/blob/master/openstack_dashboard/dashboards/project/network_topology/views.py#L251-L259

  When loading network topology page (in _get_servers function), it
  iterates servers variable, and for each running server(instance), will
  make a i_console.get_console() call, and it's only sequentially.

  Imagine you have 1000 running servers in the project, each
  get_console() takes 200 milliseconds. So in the end in order to get
  all consoles, it will cost you 1000 x 200 (milliseconds) = 200
  seconds. In openstack-ansible project, the default timeout for haproxy
  is only 50. So in the end it already times out for the user.

  Something even worse, in the network topology diagram, it seems to
  have an auto refresh enabled. So every 20-30(probably), it fires all
  the requests to backend again.

  I might have some suggestions.
  Either we make this console link in network topology configurable,
  or we fire the requests concurrently to backend?

  Thanks,
  Chenjun Shen

To manage notifications about this bug go to:
https://bugs.launchpad.net/horizon/+bug/1723142/+subscriptions