yahoo-eng-team team mailing list archive

Thread
Date
[Bug 2114734] [NEW] nova-ceph-multistore job fails to start ceph mgr

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Balazs Gibizer <2114734@xxxxxxxxxxxxxxxxxx>
Date: Mon, 16 Jun 2025 12:35:45 -0000
Reply-to: Bug 2114734 <2114734@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
Public bug reported:

https://zuul.opendev.org/t/openstack/build/c778c2aa083b486ab3111d39888aa8a1/log/job-
output.txt#11480

2025-06-13 14:44:09.389832 | controller | Restarting the monitor...
11458	2025-06-13 14:44:10.735862 | controller | Wrote config to /etc/ceph/ceph.conf
11459	2025-06-13 14:44:10.740251 | controller | Wrote keyring to /etc/ceph/ceph.client.admin.keyring
11460	2025-06-13 14:44:10.740311 | controller | Creating mgr...
11461	2025-06-13 14:44:10.741239 | controller | Verifying port 0.0.0.0:9283 ...
11462	2025-06-13 14:44:10.741282 | controller | Verifying port 0.0.0.0:8765 ...
11463	2025-06-13 14:44:12.646379 | controller | Waiting for mgr to start...
11464	2025-06-13 14:44:12.646444 | controller | Waiting for mgr...
11465	2025-06-13 14:44:14.205574 | controller | mgr not available, waiting (1/15)...
11466	2025-06-13 14:44:17.774101 | controller | mgr not available, waiting (2/15)...
11467	2025-06-13 14:44:21.348443 | controller | mgr not available, waiting (3/15)...
11468	2025-06-13 14:44:25.026204 | controller | mgr not available, waiting (4/15)...
11469	2025-06-13 14:44:28.729969 | controller | mgr not available, waiting (5/15)...
11470	2025-06-13 14:44:32.232272 | controller | mgr not available, waiting (6/15)...
11471	2025-06-13 14:44:36.009568 | controller | mgr not available, waiting (7/15)...
11472	2025-06-13 14:44:39.986524 | controller | mgr not available, waiting (8/15)...
11473	2025-06-13 14:44:43.663126 | controller | mgr not available, waiting (9/15)...
11474	2025-06-13 14:44:47.338021 | controller | mgr not available, waiting (10/15)...
11475	2025-06-13 14:44:50.983672 | controller | mgr not available, waiting (11/15)...
11476	2025-06-13 14:44:54.896368 | controller | mgr not available, waiting (12/15)...
11477	2025-06-13 14:44:58.778592 | controller | mgr not available, waiting (13/15)...
11478	2025-06-13 14:45:02.837945 | controller | mgr not available, waiting (14/15)...
11479	2025-06-13 14:45:06.914148 | controller | mgr not available, waiting (15/15)...
11480	2025-06-13 14:45:10.562853 | controller | Error: mgr not available after 15 tries

❯ logsearch log --job-group nova-devstack --result FAILURE --days 7
'Error: mgr not available after 15 tries'

Builds with matching logs 2/56:
+----------------------------------+---------------------+----------------+----------+-----------------------------------+--------+----------------------+
| uuid                             | finished            | project        | pipeline | review                            | branch | job                  |
+----------------------------------+---------------------+----------------+----------+-----------------------------------+--------+----------------------+
| c778c2aa083b486ab3111d39888aa8a1 | 2025-06-13T14:47:40 | openstack/nova | check    | https://review.opendev.org/948086 | master | nova-ceph-multistore |
| 5c8c59b424884098875f75d064ba9e52 | 2025-06-10T21:05:49 | openstack/nova | gate     | https://review.opendev.org/937245 | master | nova-ceph-multistore |
+----------------------------------+---------------------+----------------+----------+-----------------------------------+--------+----------------------+

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2114734

Title:
  nova-ceph-multistore job fails to start ceph mgr

Status in OpenStack Compute (nova):
  New

Bug description:
  https://zuul.opendev.org/t/openstack/build/c778c2aa083b486ab3111d39888aa8a1/log/job-
  output.txt#11480

  2025-06-13 14:44:09.389832 | controller | Restarting the monitor...
  11458	2025-06-13 14:44:10.735862 | controller | Wrote config to /etc/ceph/ceph.conf
  11459	2025-06-13 14:44:10.740251 | controller | Wrote keyring to /etc/ceph/ceph.client.admin.keyring
  11460	2025-06-13 14:44:10.740311 | controller | Creating mgr...
  11461	2025-06-13 14:44:10.741239 | controller | Verifying port 0.0.0.0:9283 ...
  11462	2025-06-13 14:44:10.741282 | controller | Verifying port 0.0.0.0:8765 ...
  11463	2025-06-13 14:44:12.646379 | controller | Waiting for mgr to start...
  11464	2025-06-13 14:44:12.646444 | controller | Waiting for mgr...
  11465	2025-06-13 14:44:14.205574 | controller | mgr not available, waiting (1/15)...
  11466	2025-06-13 14:44:17.774101 | controller | mgr not available, waiting (2/15)...
  11467	2025-06-13 14:44:21.348443 | controller | mgr not available, waiting (3/15)...
  11468	2025-06-13 14:44:25.026204 | controller | mgr not available, waiting (4/15)...
  11469	2025-06-13 14:44:28.729969 | controller | mgr not available, waiting (5/15)...
  11470	2025-06-13 14:44:32.232272 | controller | mgr not available, waiting (6/15)...
  11471	2025-06-13 14:44:36.009568 | controller | mgr not available, waiting (7/15)...
  11472	2025-06-13 14:44:39.986524 | controller | mgr not available, waiting (8/15)...
  11473	2025-06-13 14:44:43.663126 | controller | mgr not available, waiting (9/15)...
  11474	2025-06-13 14:44:47.338021 | controller | mgr not available, waiting (10/15)...
  11475	2025-06-13 14:44:50.983672 | controller | mgr not available, waiting (11/15)...
  11476	2025-06-13 14:44:54.896368 | controller | mgr not available, waiting (12/15)...
  11477	2025-06-13 14:44:58.778592 | controller | mgr not available, waiting (13/15)...
  11478	2025-06-13 14:45:02.837945 | controller | mgr not available, waiting (14/15)...
  11479	2025-06-13 14:45:06.914148 | controller | mgr not available, waiting (15/15)...
  11480	2025-06-13 14:45:10.562853 | controller | Error: mgr not available after 15 tries

  ❯ logsearch log --job-group nova-devstack --result FAILURE --days 7
  'Error: mgr not available after 15 tries'

  Builds with matching logs 2/56:
  +----------------------------------+---------------------+----------------+----------+-----------------------------------+--------+----------------------+
  | uuid                             | finished            | project        | pipeline | review                            | branch | job                  |
  +----------------------------------+---------------------+----------------+----------+-----------------------------------+--------+----------------------+
  | c778c2aa083b486ab3111d39888aa8a1 | 2025-06-13T14:47:40 | openstack/nova | check    | https://review.opendev.org/948086 | master | nova-ceph-multistore |
  | 5c8c59b424884098875f75d064ba9e52 | 2025-06-10T21:05:49 | openstack/nova | gate     | https://review.opendev.org/937245 | master | nova-ceph-multistore |
  +----------------------------------+---------------------+----------------+----------+-----------------------------------+--------+----------------------+

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2114734/+subscriptions