yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1837908] [NEW] Dashboard Hangs while trying to create and associate floating IP to instance at the same time

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Kyle Dean <1837908@xxxxxxxxxxxxxxxxxx>
Date: Thu, 25 Jul 2019 14:19:58 -0000
Reply-to: Bug 1837908 <1837908@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

OS: Ubuntu 18.04
Horizon Installation: Manual via apt
Package repo: ubuntu cloud archive
Openstack release: Stein
Architecture: 3 node, controller, network, compute.

Services ########
controller: keystone, nova, glance, neutron-server/ml2-plugin, designate, horizon, heat 
Network: neutron-l3-agent neutron-metadata-agent neutron-dhcp-agent neutron-linuxbridge-agent Compute: nova-compute, neutron-linuxbridge-agent
All services behind HA proxy with TLS.


Description: 
I have tested this exact bug twice, and have the exact same issue on both installations, one production, one test.

The UI does not allow the creation and association of a floating IP at
the same time.

However succeds using the CLI
create a floating IP manually using CLI and attaching to instance in one command succeed....

openstack floating ip create --dns-domain=stack.lon.example.com. --dns-
name=testy101 --fixed-ip-address=192.1.2.51 --port 59934e2e-6c6f-4c8f-
b0cf-933bcf3497c0 91217bab-6250-4ff5-ae61-0accd79a5d41

ability to ping host afterwards verified.

when using the UI the following was carried out.

Reproducible steps
log into UI and select ( associate floating IP on instance ), then select ( plus+ ) button, then select ( Allocate IP ) this is the part where a new floating IP is created and attached to the instance I had this working in ocata, but now it goes into a continuous loop with the working twirler never stopping. 

Refreshing the page, the floating IP is confirmed as being created but
never actually attached.

no errors in logs


tail -f -n1000 /var/log/apache2/horizon/error.log
[Thu Jul 25 13:47:44.477285 2019] [wsgi:error] [pid 11933:tid 140288635377408] [remote 172.30.0.2:50814] DEBUG:stevedore.extension:found extension EntryPoint.parse('http = oslo_policy._external:HttpCheck')
[Thu Jul 25 13:47:44.477908 2019] [wsgi:error] [pid 11933:tid 140288635377408] [remote 172.30.0.2:50814] DEBUG:stevedore.extension:found extension EntryPoint.parse('https = oslo_policy._external:HttpsCheck')
[Thu Jul 25 13:47:58.151732 2019] [wsgi:error] [pid 11932:tid 140288710911744] [remote 172.30.0.2:51324] DEBUG:stevedore.extension:found extension EntryPoint.parse('http = oslo_policy._external:HttpCheck')
[Thu Jul 25 13:47:58.152439 2019] [wsgi:error] [pid 11932:tid 140288710911744] [remote 172.30.0.2:51324] DEBUG:stevedore.extension:found extension EntryPoint.parse('https = oslo_policy._external:HttpsCheck')


tail -f -n1000 /var/log/apache2/horizon/access.log
172.30.0.2 - - [25/Jul/2019:14:47:22 +0100] "GET /horizon/project/instances/ HTTP/1.1" 200 8564 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
172.30.0.2 - - [25/Jul/2019:14:47:45 +0100] "GET /horizon/static/dashboard/css/361cca58bb99.css HTTP/1.1" 200 4729 "https://openstack.lon.example.com/horizon/project/instances/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
172.30.0.2 - - [25/Jul/2019:14:47:45 +0100] "GET /horizon/i18n/js/horizon+openstack_dashboard/ HTTP/1.1" 200 3612 "https://openstack.lon.example.com/horizon/project/instances/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
172.30.0.2 - - [25/Jul/2019:14:47:45 +0100] "GET /horizon/static/dashboard/js/b2bb2963e6de.js HTTP/1.1" 200 37926 "https://openstack.lon.example.com/horizon/project/instances/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
172.30.0.2 - - [25/Jul/2019:14:47:45 +0100] "GET /horizon/static/dashboard/css/7b50ccce00d0.css HTTP/1.1" 200 60951 "https://openstack.lon.example.com/horizon/project/instances/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
172.30.0.2 - - [25/Jul/2019:14:47:45 +0100] "GET /horizon/static/dashboard/js/787a5a315d99.js HTTP/1.1" 200 122901 "https://openstack.lon.example.com/horizon/project/instances/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
172.30.0.2 - - [25/Jul/2019:14:47:45 +0100] "GET /horizon/static/dashboard/js/c927fd827a6d.js HTTP/1.1" 200 429891 "https://openstack.lon.example.com/horizon/project/instances/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
172.30.0.2 - - [25/Jul/2019:14:47:45 +0100] "GET /horizon/static/dashboard/img/logo.svg HTTP/1.1" 200 5972 "https://openstack.lon.example.com/horizon/project/instances/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
172.30.0.2 - - [25/Jul/2019:14:47:45 +0100] "GET /horizon/static/horizon/lib/font_awesome/fonts/fontawesome-webfont.woff2?v=4.7.0 HTTP/1.1" 200 77387 "https://openstack.lon.example.com/horizon/static/dashboard/css/7b50ccce00d0.css"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
172.30.0.2 - - [25/Jul/2019:14:47:45 +0100] "GET /horizon/static/dashboard/img/apple-touch-icon.png HTTP/1.1" 200 1171 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
172.30.0.2 - - [25/Jul/2019:14:47:45 +0100] "GET /horizon/static/dashboard/img/favicon.ico HTTP/1.1" 200 10440 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
172.30.0.2 - - [25/Jul/2019:14:47:46 +0100] "GET /horizon/header/ HTTP/1.1" 200 574 "https://openstack.lon.example.com/horizon/project/instances/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
172.30.0.2 - - [25/Jul/2019:14:47:55 +0100] "GET /horizon/project/floating_ips/associate/?instance_id=fd08fac5-497b-4d69-8443-d0cc3ea82410&next=%2Fhorizon%2Fproject%2Finstances%2F HTTP/1.1" 200 2148 "https://openstack.lon.example.com/horizon/project/instances/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
172.30.0.2 - - [25/Jul/2019:14:48:05 +0100] "GET /horizon/project/floating_ips/allocate/ HTTP/1.1" 200 1987 "https://openstack.lon.example.com/horizon/project/instances/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
172.30.0.2 - - [25/Jul/2019:14:48:07 +0100] "POST /horizon/project/floating_ips/allocate/ HTTP/1.1" 200 662 "https://openstack.lon.example.com/horizon/project/instances/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"


http://paste.openstack.org/show/754854/ < local_settings.py

http://paste.openstack.org/show/754856/ < apache2/conf-enabled
/openstack-dashboard.conf


Steps to reproduce: 

1. install openstack and supporting services on a 3 node setup with VXLAN and provider network 
2. install dashboard
3. go to UI
4. create instance assigned to selfservice network
5. associate floating IP to instance by clicking on instance dropdown
6. select plus button
7. select Allocate IP

** Affects: horizon
     Importance: Undecided
         Status: New

** Attachment added: "Photo of issue"
   https://bugs.launchpad.net/bugs/1837908/+attachment/5279219/+files/horizon%20bug.png

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Dashboard (Horizon).
https://bugs.launchpad.net/bugs/1837908

Title:
  Dashboard Hangs while trying to create and associate floating IP to
  instance at the same time

Status in OpenStack Dashboard (Horizon):
  New

Bug description:
  OS: Ubuntu 18.04
  Horizon Installation: Manual via apt
  Package repo: ubuntu cloud archive
  Openstack release: Stein
  Architecture: 3 node, controller, network, compute.

  Services ########
  controller: keystone, nova, glance, neutron-server/ml2-plugin, designate, horizon, heat 
  Network: neutron-l3-agent neutron-metadata-agent neutron-dhcp-agent neutron-linuxbridge-agent Compute: nova-compute, neutron-linuxbridge-agent
  All services behind HA proxy with TLS.

  
  Description: 
  I have tested this exact bug twice, and have the exact same issue on both installations, one production, one test.

  The UI does not allow the creation and association of a floating IP at
  the same time.

  However succeds using the CLI
  create a floating IP manually using CLI and attaching to instance in one command succeed....

  openstack floating ip create --dns-domain=stack.lon.example.com.
  --dns-name=testy101 --fixed-ip-address=192.1.2.51 --port 59934e2e-
  6c6f-4c8f-b0cf-933bcf3497c0 91217bab-6250-4ff5-ae61-0accd79a5d41

  ability to ping host afterwards verified.

  when using the UI the following was carried out.

  Reproducible steps
  log into UI and select ( associate floating IP on instance ), then select ( plus+ ) button, then select ( Allocate IP ) this is the part where a new floating IP is created and attached to the instance I had this working in ocata, but now it goes into a continuous loop with the working twirler never stopping. 

  Refreshing the page, the floating IP is confirmed as being created but
  never actually attached.

  no errors in logs

  
  tail -f -n1000 /var/log/apache2/horizon/error.log
  [Thu Jul 25 13:47:44.477285 2019] [wsgi:error] [pid 11933:tid 140288635377408] [remote 172.30.0.2:50814] DEBUG:stevedore.extension:found extension EntryPoint.parse('http = oslo_policy._external:HttpCheck')
  [Thu Jul 25 13:47:44.477908 2019] [wsgi:error] [pid 11933:tid 140288635377408] [remote 172.30.0.2:50814] DEBUG:stevedore.extension:found extension EntryPoint.parse('https = oslo_policy._external:HttpsCheck')
  [Thu Jul 25 13:47:58.151732 2019] [wsgi:error] [pid 11932:tid 140288710911744] [remote 172.30.0.2:51324] DEBUG:stevedore.extension:found extension EntryPoint.parse('http = oslo_policy._external:HttpCheck')
  [Thu Jul 25 13:47:58.152439 2019] [wsgi:error] [pid 11932:tid 140288710911744] [remote 172.30.0.2:51324] DEBUG:stevedore.extension:found extension EntryPoint.parse('https = oslo_policy._external:HttpsCheck')


  tail -f -n1000 /var/log/apache2/horizon/access.log
  172.30.0.2 - - [25/Jul/2019:14:47:22 +0100] "GET /horizon/project/instances/ HTTP/1.1" 200 8564 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
  172.30.0.2 - - [25/Jul/2019:14:47:45 +0100] "GET /horizon/static/dashboard/css/361cca58bb99.css HTTP/1.1" 200 4729 "https://openstack.lon.example.com/horizon/project/instances/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
  172.30.0.2 - - [25/Jul/2019:14:47:45 +0100] "GET /horizon/i18n/js/horizon+openstack_dashboard/ HTTP/1.1" 200 3612 "https://openstack.lon.example.com/horizon/project/instances/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
  172.30.0.2 - - [25/Jul/2019:14:47:45 +0100] "GET /horizon/static/dashboard/js/b2bb2963e6de.js HTTP/1.1" 200 37926 "https://openstack.lon.example.com/horizon/project/instances/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
  172.30.0.2 - - [25/Jul/2019:14:47:45 +0100] "GET /horizon/static/dashboard/css/7b50ccce00d0.css HTTP/1.1" 200 60951 "https://openstack.lon.example.com/horizon/project/instances/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
  172.30.0.2 - - [25/Jul/2019:14:47:45 +0100] "GET /horizon/static/dashboard/js/787a5a315d99.js HTTP/1.1" 200 122901 "https://openstack.lon.example.com/horizon/project/instances/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
  172.30.0.2 - - [25/Jul/2019:14:47:45 +0100] "GET /horizon/static/dashboard/js/c927fd827a6d.js HTTP/1.1" 200 429891 "https://openstack.lon.example.com/horizon/project/instances/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
  172.30.0.2 - - [25/Jul/2019:14:47:45 +0100] "GET /horizon/static/dashboard/img/logo.svg HTTP/1.1" 200 5972 "https://openstack.lon.example.com/horizon/project/instances/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
  172.30.0.2 - - [25/Jul/2019:14:47:45 +0100] "GET /horizon/static/horizon/lib/font_awesome/fonts/fontawesome-webfont.woff2?v=4.7.0 HTTP/1.1" 200 77387 "https://openstack.lon.example.com/horizon/static/dashboard/css/7b50ccce00d0.css"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
  172.30.0.2 - - [25/Jul/2019:14:47:45 +0100] "GET /horizon/static/dashboard/img/apple-touch-icon.png HTTP/1.1" 200 1171 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
  172.30.0.2 - - [25/Jul/2019:14:47:45 +0100] "GET /horizon/static/dashboard/img/favicon.ico HTTP/1.1" 200 10440 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
  172.30.0.2 - - [25/Jul/2019:14:47:46 +0100] "GET /horizon/header/ HTTP/1.1" 200 574 "https://openstack.lon.example.com/horizon/project/instances/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
  172.30.0.2 - - [25/Jul/2019:14:47:55 +0100] "GET /horizon/project/floating_ips/associate/?instance_id=fd08fac5-497b-4d69-8443-d0cc3ea82410&next=%2Fhorizon%2Fproject%2Finstances%2F HTTP/1.1" 200 2148 "https://openstack.lon.example.com/horizon/project/instances/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
  172.30.0.2 - - [25/Jul/2019:14:48:05 +0100] "GET /horizon/project/floating_ips/allocate/ HTTP/1.1" 200 1987 "https://openstack.lon.example.com/horizon/project/instances/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"
  172.30.0.2 - - [25/Jul/2019:14:48:07 +0100] "POST /horizon/project/floating_ips/allocate/ HTTP/1.1" 200 662 "https://openstack.lon.example.com/horizon/project/instances/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0"

  
  http://paste.openstack.org/show/754854/ < local_settings.py

  http://paste.openstack.org/show/754856/ < apache2/conf-enabled
  /openstack-dashboard.conf

  
  Steps to reproduce: 

  1. install openstack and supporting services on a 3 node setup with VXLAN and provider network 
  2. install dashboard
  3. go to UI
  4. create instance assigned to selfservice network
  5. associate floating IP to instance by clicking on instance dropdown
  6. select plus button
  7. select Allocate IP

To manage notifications about this bug go to:
https://bugs.launchpad.net/horizon/+bug/1837908/+subscriptions
Follow ups

[Bug 1837908] Re: Dashboard Hangs while trying to create and associate floating IP to instance at the same time
From: Akihiro Motoki, 2021-03-03