sts-sponsors team mailing list archive

Thread
Date
[Bug 1870087] Re: Old broker lockfile blocks landscape-client starts

To: sts-sponsors@xxxxxxxxxxxxxxxxxxx
From: Simon Poirier <1870087@xxxxxxxxxxxxxxxxxx>
Date: Tue, 14 Apr 2020 19:55:11 -0000
Reply-to: Bug 1870087 <1870087@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
** Also affects: landscape-client (Ubuntu)
   Importance: Undecided
       Status: New

** Also affects: landscape-client (Ubuntu Focal)
   Importance: Undecided
       Status: New

** Changed in: landscape-client (Ubuntu Focal)
       Status: New => In Progress

** Changed in: landscape-client (Ubuntu Focal)
     Assignee: (unassigned) => Simon Poirier (simpoir)

-- 
You received this bug notification because you are a member of STS
Sponsors, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1870087

Title:
  Old broker lockfile blocks landscape-client starts

Status in Landscape Client:
  Fix Committed
Status in landscape-client package in Ubuntu:
  In Progress
Status in landscape-client source package in Focal:
  In Progress

Bug description:
  I have a machine which was failing to connect to the landscape
  service. In syslog I found this traceback:

  Apr  1 03:27:53 maas-1 landscape-client[1538354]: Traceback (most recent call last):
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:   File "/usr/lib/python3/dist-packages/twisted/python/lockfile.py", line 160, in lock
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:     symlink(str(os.getpid()), self.name)
  Apr  1 03:27:53 maas-1 landscape-client[1538354]: FileExistsError: [Errno 17] File exists: '1538397' -> b'/var/lib/landscape/client/sockets/broker.sock.lock'
  Apr  1 03:27:53 maas-1 landscape-client[1538354]: During handling of the above exception, another exception occurred:
  Apr  1 03:27:53 maas-1 landscape-client[1538354]: Traceback (most recent call last):
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:   File "/usr/bin/landscape-broker", line 8, in <module>
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:     run(sys.argv)
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:   File "/usr/lib/python3/dist-packages/landscape/client/broker/service.py", line 93, in run
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:     run_landscape_service(BrokerConfiguration, BrokerService, args)
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:   File "/usr/lib/python3/dist-packages/landscape/client/service.py", line 115, in run_landscape_service
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:     startApplication(application, False)
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:   File "/usr/lib/python3/dist-packages/twisted/application/app.py", line 690, in startApplication
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:     service.IService(application).startService()
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:   File "/usr/lib/python3/dist-packages/twisted/application/service.py", line 288, in startService
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:     service.startService()
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:   File "/usr/lib/python3/dist-packages/landscape/client/broker/service.py", line 79, in startService
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:     self.publisher.start()
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:   File "/usr/lib/python3/dist-packages/landscape/client/amp.py", line 45, in start
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:     self._port = self._reactor.listen_unix(socket_path, factory)
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:   File "/usr/lib/python3/dist-packages/landscape/lib/reactor.py", line 228, in listen_unix
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:     return self._reactor.listenUNIX(socket, factory, wantPID=True)
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:   File "/usr/lib/python3/dist-packages/twisted/internet/posixbase.py", line 397, in listenUNIX
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:     p.startListening()
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:   File "/usr/lib/python3/dist-packages/twisted/internet/unix.py", line 372, in startListening
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:     if not self.lockFile.lock():
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:   File "/usr/lib/python3/dist-packages/twisted/python/lockfile.py", line 185, in lock
  Apr  1 03:27:53 maas-1 landscape-client[1538354]:     kill(int(pid), 0)
  Apr  1 03:27:53 maas-1 landscape-client[1538354]: PermissionError: [Errno 1] Operation not permitted

  In the sockets directory I saw:

  $ sudo ls /var/lib/landscape/client/sockets/ -la
  total 8
  drwxr-x--- 2 landscape root      4096 Apr  1 03:27 .
  drwxr-xr-x 7 landscape root      4096 Apr  1 03:27 ..
  srw-rw-rw- 1 landscape landscape    0 Mar 12 01:41 broker.sock
  lrwxrwxrwx 1 landscape landscape    3 Mar 12 01:41 broker.sock.lock -> 905

  Removing those two files allowed the landscape client to start as
  normal.

  Looks like we need some lockfile cleanup code on start.

To manage notifications about this bug go to:
https://bugs.launchpad.net/landscape-client/+bug/1870087/+subscriptions