canonical-ubuntu-qa team mailing list archive

Thread
Date
[Bug 2097687] [NEW] ubuntu_zfs_stress sut-deploy fails with File_Lock_Failed

To: canonical-ubuntu-qa@xxxxxxxxxxxxxxxxxxx
From: Tim Whisonant <2097687@xxxxxxxxxxxxxxxxxx>
Date: Sat, 08 Feb 2025 23:20:29 -0000
Reply-to: Bug 2097687 <2097687@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
Public bug reported:

Found on j:gcp-6.8 6.8.0-1024.26~22.04.1 during 2025.01.13 on
n1-highcpu-32 during provisioning.

1	 	Started by user unknown or anonymous
2	 	Running as SYSTEM
3	 	Building remotely on <redacted> ...
4	 	[WS-CLEANUP] Deleting project workspace...
5	 	[WS-CLEANUP] Deferred wipeout is used...
6	 	[jammy-linux-gcp-6.8-gcp-amd64-6.8.0-n1-highcpu-32-ubuntu_zfs_stress] $ /bin/bash /tmp/jenkins17456929552925101237.sh
7	 	-------------------------------------------------------------------------------------------------------
8	 	D E P L O Y
9	 	-------------------------------------------------------------------------------------------------------
10	 	            Json File Exists
11	 	            Json File Loaded OK
12	 	            Re-Generating Json File: 
13	 	            Checking for imagejson Directory
14	 	            Identifying Lock File
15	 	Traceback (most recent call last):
16	 	  File "/home/google/ckct/lib/google.py", line 430, in get_image
17	 	    j.check_date(jsonfile)
18	 	  File "/home/google/ckct/lib/image_cache.py", line 101, in check_date
19	 	    raise StaleJsonError('Json File is greater than 24 hours old')
20	 	lib.image_cache.StaleJsonError: Json File is greater than 24 hours old
21	 	
22	 	During handling of the above exception, another exception occurred:
23	 	
24	 	Traceback (most recent call last):
25	 	  File "/home/google/ckct/lib/utils.py", line 66, in lock
26	 	    fcntl.flock(self.lockf, fcntl.LOCK_EX | fcntl.LOCK_NB)
27	 	BlockingIOError: [Errno 11] Resource temporarily unavailable
28	 	
29	 	During handling of the above exception, another exception occurred:
30	 	
31	 	Traceback (most recent call last):
32	 	  File "/home/google/ckct/sut-deploy", line 440, in 
33	 	    retval = main(args)
34	 	             ^^^^^^^^^^
35	 	  File "/home/google/ckct/lib/log.py", line 63, in retfunc
36	 	    retval = func(*args, **kwargs)
37	 	             ^^^^^^^^^^^^^^^^^^^^^
38	 	  File "/home/google/ckct/sut-deploy", line 155, in main
39	 	    retval = cloud.deploy(
40	 	             ^^^^^^^^^^^^^
41	 	  File "/home/google/ckct/lib/log.py", line 53, in retfunc
42	 	    retval = func(*args, **kwargs)
43	 	             ^^^^^^^^^^^^^^^^^^^^^
44	 	  File "/home/google/ckct/lib/google.py", line 700, in deploy
45	 	    image = s.get_image(series, region, arch, bool(sev_type))
46	 	            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
47	 	  File "/home/google/ckct/lib/google.py", line 436, in get_image
48	 	    j.generate(jsonfile)
49	 	  File "/home/google/ckct/lib/image_cache.py", line 161, in generate
50	 	    lock_file.lock()
51	 	  File "/home/google/ckct/lib/utils.py", line 77, in lock
52	 	    raise File_Lock_Failed(e)
53	 	lib.utils.File_Lock_Failed: [Errno 11] Resource temporarily unavailable
54	 	Unable to obtain Lock, file in use
55	 	ERROR: (1) ERROR: (gcloud.compute.instances.get-serial-port-output) Could not fetch serial port output: The resource 'projects/ubuntu-devel/zones/europe-west1-b/instances/j-lgcp68-gcp-6-8-n1hcpu32-u-zfs-stress' was not found
56	 	Unable to obtain console log, with error:
57	 	Unable to get gcloud console log
58	 	INFO - [31m    File ubuntu_zfs_stress-console.log for sut-scan does not exist, Check Path. (It's expected if console log was not obtainable)[0m
59	 	Traceback (most recent call last):
60	 	  File "/home/google/ckct/sut-release", line 137, in 
61	 	    retval = app.main(args)
62	 	             ^^^^^^^^^^^^^^
63	 	  File "/home/google/ckct/sut-release", line 48, in main
64	 	    cloud.destroy(region, args.name)
65	 	  File "/home/google/ckct/lib/log.py", line 53, in retfunc
66	 	    retval = func(*args, **kwargs)
67	 	             ^^^^^^^^^^^^^^^^^^^^^
68	 	  File "/home/google/ckct/lib/google.py", line 779, in destroy
69	 	    instances[instance_name]["zone"],
70	 	    ~~~~~~~~~^^^^^^^^^^^^^^^
71	 	KeyError: 'j-lgcp68-gcp-6-8-n1hcpu32-u-zfs-stress'
72	 	Build step 'Execute shell' marked build as failure
73	 	Recording test results
74	 	[Checks API] No suitable checks publisher found.
75	 	Archiving artifacts
76	 	Triggering a new build of <redacted> ...
77	 	Finished: FAILURE

** Affects: ubuntu-kernel-tests
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Canonical
Platform QA Team, which is subscribed to ubuntu-kernel-tests.
https://bugs.launchpad.net/bugs/2097687

Title:
  ubuntu_zfs_stress sut-deploy fails with File_Lock_Failed

Status in ubuntu-kernel-tests:
  New

Bug description:
  Found on j:gcp-6.8 6.8.0-1024.26~22.04.1 during 2025.01.13 on
  n1-highcpu-32 during provisioning.

  1	 	Started by user unknown or anonymous
  2	 	Running as SYSTEM
  3	 	Building remotely on <redacted> ...
  4	 	[WS-CLEANUP] Deleting project workspace...
  5	 	[WS-CLEANUP] Deferred wipeout is used...
  6	 	[jammy-linux-gcp-6.8-gcp-amd64-6.8.0-n1-highcpu-32-ubuntu_zfs_stress] $ /bin/bash /tmp/jenkins17456929552925101237.sh
  7	 	-------------------------------------------------------------------------------------------------------
  8	 	D E P L O Y
  9	 	-------------------------------------------------------------------------------------------------------
  10	 	            Json File Exists
  11	 	            Json File Loaded OK
  12	 	            Re-Generating Json File: 
  13	 	            Checking for imagejson Directory
  14	 	            Identifying Lock File
  15	 	Traceback (most recent call last):
  16	 	  File "/home/google/ckct/lib/google.py", line 430, in get_image
  17	 	    j.check_date(jsonfile)
  18	 	  File "/home/google/ckct/lib/image_cache.py", line 101, in check_date
  19	 	    raise StaleJsonError('Json File is greater than 24 hours old')
  20	 	lib.image_cache.StaleJsonError: Json File is greater than 24 hours old
  21	 	
  22	 	During handling of the above exception, another exception occurred:
  23	 	
  24	 	Traceback (most recent call last):
  25	 	  File "/home/google/ckct/lib/utils.py", line 66, in lock
  26	 	    fcntl.flock(self.lockf, fcntl.LOCK_EX | fcntl.LOCK_NB)
  27	 	BlockingIOError: [Errno 11] Resource temporarily unavailable
  28	 	
  29	 	During handling of the above exception, another exception occurred:
  30	 	
  31	 	Traceback (most recent call last):
  32	 	  File "/home/google/ckct/sut-deploy", line 440, in 
  33	 	    retval = main(args)
  34	 	             ^^^^^^^^^^
  35	 	  File "/home/google/ckct/lib/log.py", line 63, in retfunc
  36	 	    retval = func(*args, **kwargs)
  37	 	             ^^^^^^^^^^^^^^^^^^^^^
  38	 	  File "/home/google/ckct/sut-deploy", line 155, in main
  39	 	    retval = cloud.deploy(
  40	 	             ^^^^^^^^^^^^^
  41	 	  File "/home/google/ckct/lib/log.py", line 53, in retfunc
  42	 	    retval = func(*args, **kwargs)
  43	 	             ^^^^^^^^^^^^^^^^^^^^^
  44	 	  File "/home/google/ckct/lib/google.py", line 700, in deploy
  45	 	    image = s.get_image(series, region, arch, bool(sev_type))
  46	 	            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  47	 	  File "/home/google/ckct/lib/google.py", line 436, in get_image
  48	 	    j.generate(jsonfile)
  49	 	  File "/home/google/ckct/lib/image_cache.py", line 161, in generate
  50	 	    lock_file.lock()
  51	 	  File "/home/google/ckct/lib/utils.py", line 77, in lock
  52	 	    raise File_Lock_Failed(e)
  53	 	lib.utils.File_Lock_Failed: [Errno 11] Resource temporarily unavailable
  54	 	Unable to obtain Lock, file in use
  55	 	ERROR: (1) ERROR: (gcloud.compute.instances.get-serial-port-output) Could not fetch serial port output: The resource 'projects/ubuntu-devel/zones/europe-west1-b/instances/j-lgcp68-gcp-6-8-n1hcpu32-u-zfs-stress' was not found
  56	 	Unable to obtain console log, with error:
  57	 	Unable to get gcloud console log
  58	 	INFO - [31m    File ubuntu_zfs_stress-console.log for sut-scan does not exist, Check Path. (It's expected if console log was not obtainable)[0m
  59	 	Traceback (most recent call last):
  60	 	  File "/home/google/ckct/sut-release", line 137, in 
  61	 	    retval = app.main(args)
  62	 	             ^^^^^^^^^^^^^^
  63	 	  File "/home/google/ckct/sut-release", line 48, in main
  64	 	    cloud.destroy(region, args.name)
  65	 	  File "/home/google/ckct/lib/log.py", line 53, in retfunc
  66	 	    retval = func(*args, **kwargs)
  67	 	             ^^^^^^^^^^^^^^^^^^^^^
  68	 	  File "/home/google/ckct/lib/google.py", line 779, in destroy
  69	 	    instances[instance_name]["zone"],
  70	 	    ~~~~~~~~~^^^^^^^^^^^^^^^
  71	 	KeyError: 'j-lgcp68-gcp-6-8-n1hcpu32-u-zfs-stress'
  72	 	Build step 'Execute shell' marked build as failure
  73	 	Recording test results
  74	 	[Checks API] No suitable checks publisher found.
  75	 	Archiving artifacts
  76	 	Triggering a new build of <redacted> ...
  77	 	Finished: FAILURE

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-kernel-tests/+bug/2097687/+subscriptions