group.of.nepali.translators team mailing list archive
-
group.of.nepali.translators team
-
Mailing list archive
-
Message #48005
[Bug 2060769] Re: pro-client decodes utf-8 strictly, which causes problems in some locales
This bug was fixed in the package ubuntu-advantage-tools - 33.2~18.04
---------------
ubuntu-advantage-tools (33.2~18.04) bionic; urgency=medium
* Backport 33.2 to bionic (LP: #2069237)
ubuntu-advantage-tools (33.2) oracular; urgency=medium
* d/apparmor: add apt-news access to package information on the system
(LP: #2072489) (GH: #3193)
ubuntu-advantage-tools (33.1) oracular; urgency=medium
* New upstream release 33.1: (LP: #2060769)
- system:
+ always pass C.UTF8 as the language when calling a subprocess
+ ignore utf-8 decode errors on subprocess output
ubuntu-advantage-tools (33) oracular; urgency=medium
* d/apparmor: adjust the esm_cache apparmor profile to allow reading of dpkg
data directory (LP: #2067810) (GH: #3137)
* New upstream release 33 (LP: #2069237)
- apt: use Python bindings instead of apt CLI to query for installed
packages (LP: #2060769) (LP: #2068744)
- beta: drop support for beta services
- contracts: add support for contracts which target a specific series
- fips: change enable functionality to ensure all packages with a FIPS
candidate are upgraded to the FIPS version (GH: #2667)
- fix:
+ add the current_status field to the plan api return object
+ change recommended attach method to magic attach (GH: #3040)
- livepatch: prefer the term 'coverage' instead of 'support' in messaging
(GH: #3063)
- realtime:
+ auto-select the raspi variant when appropriate
+ inform the user when auto-selecting a variant
-- Lucas Moura <lucas.moura@xxxxxxxxxxxxx> Thu, 18 Jul 2024 11:20:12
-0400
** Changed in: ubuntu-advantage-tools (Ubuntu Xenial)
Status: Fix Committed => Fix Released
--
You received this bug notification because you are a member of नेपाली
भाषा समायोजकहरुको समूह, which is subscribed to Xenial.
Matching subscriptions: Ubuntu 16.04 Bugs
https://bugs.launchpad.net/bugs/2060769
Title:
pro-client decodes utf-8 strictly, which causes problems in some
locales
Status in ubuntu-advantage-tools package in Ubuntu:
Fix Released
Status in ubuntu-advantage-tools source package in Xenial:
Fix Released
Status in ubuntu-advantage-tools source package in Bionic:
Fix Released
Status in ubuntu-advantage-tools source package in Focal:
Fix Released
Status in ubuntu-advantage-tools source package in Jammy:
Fix Released
Status in ubuntu-advantage-tools source package in Noble:
Fix Released
Bug description:
[ Impact ]
If for any reason the user's system has any non-utf8 character as part
of any subprocess command executed, the Pro Client will fail to parse
the return output and break.
Independent of why this happens, the Pro Client should not be relying
on CLI output when there are Python bindings available to perform the
same task - and python3-apt provides ways to list installed packages.
The fix to this bug is to ignore utf-8 errors in the occasions where
it needs to subprocess. Even other software than APT may include non-
utf8 output by chance. To avoid corner case scenarios, setting LANG=C
when executing any subprocess should mitigate the presence of non-utf8
anyway.
[ Test Plan ]
Subprocessing is part of the regular operation of the Pro Client in
many scenarios covered by the integration / acceptance tests.
For this particular issue:
- Launch a container for the release to be tested
- Install a non-utf-8 locale and set it as the default. e.g.
- sudo locale-gen fr_FR
- sudo dpkg-reconfigure locales # select fr_FR
- # log out and log in
- pro status
- see it fail
- install the package from -proposed
- see it does not fail anymore
[ Where problems could occur ]
Ignoring non-utf8 characters when decoding will not prevent the Client
from reading all the actual utf-8 present in the output. The pieces we
parse from the commands we execute are not involving any possible non-
utf8 text. If we would have assessed this wrong, this could cause all
sorts of regressions.
To be extra careful, we are setting LANG=C.UTF-8 - so we have way
better expectation management on the outputs from any external command
we run on a system.
[ Original description ]
["2024-04-10T08:28:17.952", "DEBUG", "ubuntupro.system", "load_file", 517, "Reading file: /etc/os-release", {}]
["2024-04-10T08:28:17.953", "DEBUG", "ubuntupro.system", "load_file", 517, "Reading file: /proc/version_signature", {}]
["2024-04-10T08:28:18.575", "ERROR", "ubuntupro.cli", "wrapper", 1768, "Unhandled exception, please file a bug", {"exc_info": "Traceback (most recent call
last):\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 1705, in wrapper\n return func(*args, **kwargs)\n File \"/usr/lib/py
thon3/dist-packages/uaclient/cli/__init__.py\", line 1838, in main\n return_value = args.action(args, cfg=cfg, extra_args=extra_args)\n File \"/usr/li
b/python3/dist-packages/uaclient/cli/__init__.py\", line 1475, in action_status\n status, ret = actions.status(\n File \"/usr/lib/python3/dist-package
s/uaclient/actions.py\", line 173, in status\n status = ua_status.status(cfg=cfg, show_all=show_all)\n File \"/usr/lib/python3/dist-packages/uaclient/
status.py\", line 418, in status\n response = _unattached_status(cfg)\n File \"/usr/lib/python3/dist-packages/uaclient/status.py\", line 283, in _unat
tached_status\n resources = get_available_resources(cfg)\n File \"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 680, in get_available_re
sources\n resources = client.available_resources()\n File \"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 114, in available_resources\n
activity_info = self._get_activity_info()\n File \"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 413, in _get_activity_info\n \"deskt
op\": system.is_desktop(),\n File \"/usr/lib/python3/dist-packages/uaclient/system.py\", line 397, in is_desktop\n for package in apt.get_installed_pa
ckages():\n File \"/usr/lib/python3/dist-packages/uaclient/apt.py\", line 727, in get_installed_packages\n out, _ = system.subp([\"apt\", \"list\", \"
--installed\"])\n File \"/usr/lib/python3/dist-packages/uaclient/system.py\", line 703, in subp\n out, err = _subp(\n File \"/usr/lib/python3/dist-pa
ckages/uaclient/system.py\", line 649, in _subp\n out_result = out.decode(\"utf-8\") if out else \"\"\nUnicodeDecodeError: 'utf-8' codec can't decode b
yte 0xe1 in position 109: invalid continuation byte"}]
["2024-04-10T08:30:25.404", "DEBUG", "ubuntupro.config", "parse_config", 625, "Using client configuration file at /etc/ubuntu-advantage/uaclient.conf", {}
]
["2024-04-10T08:30:25.408", "DEBUG", "ubuntupro.system", "load_file", 517, "Reading file: /etc/ubuntu-advantage/uaclient.conf", {}]
["2024-04-10T08:30:25.409", "DEBUG", "ubuntupro.files.files", "read", 60, "Tried to load /var/lib/ubuntu-advantage/user-config.json but file does not exis
t", {}]
["2024-04-10T08:30:25.421", "DEBUG", "ubuntupro.http", "configure_web_proxy", 146, "Setting no_proxy: 169.254.169.254,[fd00:ec2::254],metadata", {}]
["2024-04-10T08:30:25.422", "DEBUG", "ubuntupro.http", "configure_web_proxy", 154, "Setting global proxy dict", {}]
["2024-04-10T08:30:25.422", "DEBUG", "ubuntupro.cli", "main", 1825, "Executed with sys.argv: ['/usr/bin/pro', 'attach', '<REDACTED>']", {}]
["2024-04-10T08:30:25.422", "DEBUG", "ubuntupro.files.files", "read", 60, "Tried to load /var/lib/ubuntu-advantage/private/machine-token.json but file doe
s not exist", {}]
["2024-04-10T08:30:25.422", "DEBUG", "ubuntupro.lock", "__enter__", 77, "spin lock starting for pro attach", {}]
["2024-04-10T08:30:25.423", "DEBUG", "ubuntupro.system", "write_file", 556, "Writing file /run/ubuntu-advantage/notices/60-operation_in_progress atomicall
y via tempfile /run/ubuntu-advantage/notices/tmparyyoirn", {}]
["2024-04-10T08:30:25.423", "DEBUG", "ubuntupro.system", "write_file", 556, "Writing file /var/lib/ubuntu-advantage/lock atomically via tempfile /var/lib/
ubuntu-advantage/tmpn1yxl91o", {}]
["2024-04-10T08:30:25.424", "DEBUG", "ubuntupro.system", "write_file", 556, "Writing file /run/ubuntu-advantage/notices/60-operation_in_progress atomicall
y via tempfile /run/ubuntu-advantage/notices/tmpwqyq85v1", {}]
["2024-04-10T08:30:25.424", "DEBUG", "ubuntupro.files.files", "read", 60, "Tried to load /var/lib/ubuntu-advantage/private/machine-token.json but file doe
s not exist", {}]
["2024-04-10T08:30:25.424", "DEBUG", "ubuntupro.system", "load_file", 517, "Reading file: /etc/machine-id", {}]
["2024-04-10T08:30:25.424", "DEBUG", "ubuntupro.system", "load_file", 517, "Reading file: /etc/os-release", {}]
["2024-04-10T08:30:25.425", "DEBUG", "ubuntupro.system", "load_file", 517, "Reading file: /proc/version_signature", {}]
["2024-04-10T08:30:25.970", "DEBUG", "ubuntupro.system", "ensure_file_absent", 576, "Removed file: /run/ubuntu-advantage/notices/60-operation_in_progress"
, {}]
["2024-04-10T08:30:25.970", "DEBUG", "ubuntupro.system", "ensure_file_absent", 576, "Removed file: /var/lib/ubuntu-advantage/lock", {}]
["2024-04-10T08:30:25.971", "ERROR", "ubuntupro.cli", "wrapper", 1768, "Unhandled exception, please file a bug", {"exc_info": "Traceback (most recent call
last):\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 1705, in wrapper\n return func(*args, **kwargs)\n File \"/usr/lib/py
thon3/dist-packages/uaclient/cli/__init__.py\", line 1838, in main\n return_value = args.action(args, cfg=cfg, extra_args=extra_args)\n File \"/usr/li
b/python3/dist-packages/uaclient/cli/__init__.py\", line 257, in new_f\n return f(args, cfg=cfg, **kwargs)\n File \"/usr/lib/python3/dist-packages/uac
lient/cli/__init__.py\", line 202, in new_f\n return f(*args, **kwargs)\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 186,
in new_f\n retval = f(*args, cfg=cfg, **kwargs)\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 1305, in action_attach\n a
ctions.attach_with_token(cfg, token=token, allow_enable=allow_enable)\n File \"/usr/lib/python3/dist-packages/uaclient/actions.py\", line 65, in attach_w
ith_token\n new_machine_token = contract_client.add_contract_machine(\n File \"/usr/lib/python3/dist-packages/uaclient/util.py\", line 92, in decorato
r\n return f(*args, **kwargs)\n File \"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 92, in add_contract_machine\n activity_info = se
lf._get_activity_info()\n File \"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 413, in _get_activity_info\n \"desktop\": system.is_deskt
op(),\n File \"/usr/lib/python3/dist-packages/uaclient/system.py\", line 397, in is_desktop\n for package in apt.get_installed_packages():\n File \"/
usr/lib/python3/dist-packages/uaclient/apt.py\", line 727, in get_installed_packages\n out, _ = system.subp([\"apt\", \"list\", \"--installed\"])\n Fi
le \"/usr/lib/python3/dist-packages/uaclient/system.py\", line 703, in subp\n out, err = _subp(\n File \"/usr/lib/python3/dist-packages/uaclient/syste
m.py\", line 649, in _subp\n out_result = out.decode(\"utf-8\") if out else \"\"\nUnicodeDecodeError: 'utf-8' codec can't decode byte 0xe1 in position
109: invalid continuation byte"}]
["2024-04-10T08:38:29.763", "DEBUG", "ubuntupro.config", "parse_config", 625, "Using client configuration file at /etc/ubuntu-advantage/uaclient.conf", {}
]
["2024-04-10T08:38:29.767", "DEBUG", "ubuntupro.system", "load_file", 517, "Reading file: /etc/ubuntu-advantage/uaclient.conf", {}]
["2024-04-10T08:38:29.768", "DEBUG", "ubuntupro.files.files", "read", 60, "Tried to load /var/lib/ubuntu-advantage/user-config.json but file does not exis
t", {}]
["2024-04-10T08:38:29.780", "DEBUG", "ubuntupro.http", "configure_web_proxy", 146, "Setting no_proxy: 169.254.169.254,[fd00:ec2::254],metadata", {}]
["2024-04-10T08:38:29.780", "DEBUG", "ubuntupro.http", "configure_web_proxy", 154, "Setting global proxy dict", {}]
["2024-04-10T08:38:29.780", "DEBUG", "ubuntupro.cli", "main", 1825, "Executed with sys.argv: ['/usr/bin/pro', 'attach', '<REDACTED>']", {}]
["2024-04-10T08:38:29.781", "DEBUG", "ubuntupro.files.files", "read", 60, "Tried to load /var/lib/ubuntu-advantage/private/machine-token.json but file doe
s not exist", {}]
["2024-04-10T08:38:29.781", "DEBUG", "ubuntupro.lock", "__enter__", 77, "spin lock starting for pro attach", {}]
["2024-04-10T08:38:29.781", "DEBUG", "ubuntupro.system", "write_file", 556, "Writing file /run/ubuntu-advantage/notices/60-operation_in_progress atomicall
y via tempfile /run/ubuntu-advantage/notices/tmplx6ehdag", {}]
["2024-04-10T08:38:29.782", "DEBUG", "ubuntupro.system", "write_file", 556, "Writing file /var/lib/ubuntu-advantage/lock atomically via tempfile /var/lib/
ubuntu-advantage/tmp6ml_ltfj", {}]
["2024-04-10T08:38:29.782", "DEBUG", "ubuntupro.system", "write_file", 556, "Writing file /run/ubuntu-advantage/notices/60-operation_in_progress atomicall
y via tempfile /run/ubuntu-advantage/notices/tmpvxf_b1v0", {}]
["2024-04-10T08:38:29.783", "DEBUG", "ubuntupro.files.files", "read", 60, "Tried to load /var/lib/ubuntu-advantage/private/machine-token.json but file doe
s not exist", {}]
["2024-04-10T08:38:29.783", "DEBUG", "ubuntupro.system", "load_file", 517, "Reading file: /etc/machine-id", {}]
["2024-04-10T08:38:29.783", "DEBUG", "ubuntupro.system", "load_file", 517, "Reading file: /etc/os-release", {}]
["2024-04-10T08:38:29.784", "DEBUG", "ubuntupro.system", "load_file", 517, "Reading file: /proc/version_signature", {}]
["2024-04-10T08:38:30.366", "DEBUG", "ubuntupro.system", "ensure_file_absent", 576, "Removed file: /run/ubuntu-advantage/notices/60-operation_in_progress"
, {}]
["2024-04-10T08:38:30.367", "DEBUG", "ubuntupro.system", "ensure_file_absent", 576, "Removed file: /var/lib/ubuntu-advantage/lock", {}]
["2024-04-10T08:38:30.367", "ERROR", "ubuntupro.cli", "wrapper", 1768, "Unhandled exception, please file a bug", {"exc_info": "Traceback (most recent call
last):\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 1705, in wrapper\n return func(*args, **kwargs)\n File \"/usr/lib/py
thon3/dist-packages/uaclient/cli/__init__.py\", line 1838, in main\n return_value = args.action(args, cfg=cfg, extra_args=extra_args)\n File \"/usr/li
b/python3/dist-packages/uaclient/cli/__init__.py\", line 257, in new_f\n return f(args, cfg=cfg, **kwargs)\n File \"/usr/lib/python3/dist-packages/uac
lient/cli/__init__.py\", line 202, in new_f\n return f(*args, **kwargs)\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 186,
in new_f\n retval = f(*args, cfg=cfg, **kwargs)\n File \"/usr/lib/python3/dist-packages/uaclient/cli/__init__.py\", line 1305, in action_attach\n a
ctions.attach_with_token(cfg, token=token, allow_enable=allow_enable)\n File \"/usr/lib/python3/dist-packages/uaclient/actions.py\", line 65, in attach_w
ith_token\n new_machine_token = contract_client.add_contract_machine(\n File \"/usr/lib/python3/dist-packages/uaclient/util.py\", line 92, in decorato
r\n return f(*args, **kwargs)\n File \"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 92, in add_contract_machine\n activity_info = se
lf._get_activity_info()\n File \"/usr/lib/python3/dist-packages/uaclient/contract.py\", line 413, in _get_activity_info\n \"desktop\": system.is_deskt
op(),\n File \"/usr/lib/python3/dist-packages/uaclient/system.py\", line 397, in is_desktop\n for package in apt.get_installed_packages():\n File \"/
usr/lib/python3/dist-packages/uaclient/apt.py\", line 727, in get_installed_packages\n out, _ = system.subp([\"apt\", \"list\", \"--installed\"])\n Fi
le \"/usr/lib/python3/dist-packages/uaclient/system.py\", line 703, in subp\n out, err = _subp(\n File \"/usr/lib/python3/dist-packages/uaclient/syste
m.py\", line 649, in _subp\n out_result = out.decode(\"utf-8\") if out else \"\"\nUnicodeDecodeError: 'utf-8' codec can't decode byte 0xe1 in position
109: invalid continuation byte"}]
ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: ubuntu-advantage-tools 31.2~20.04
ProcVersionSignature: Ubuntu 5.15.0-101.111~20.04.1-generic 5.15.143
Uname: Linux 5.15.0-101-generic x86_64
NonfreeKernelModules: falcon_lsm_serviceable falcon_nf_netcontain falcon_kal falcon_lsm_pinned_16407 falcon_lsm_pinned_16206
ApportVersion: 2.20.11-0ubuntu27.27
Architecture: amd64
CasperMD5CheckResult: skip
Date: Wed Apr 10 08:28:11 2024
PackageArchitecture: all
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=es_ES
SHELL=/bin/bash
SourcePackage: ubuntu-advantage-tools
UpgradeStatus: No upgrade log present (probably fresh install)
apparmor_logs.txt:
cloud-id.txt-error: Invalid command specified 'cloud-id'.
livepatch-status.txt-error: Invalid command specified '/snap/bin/canonical-livepatch status'.
ua-status.json-error: Failed running command 'pro status --format json' [exit(1)]. Message:
uaclient.conf:
contract_url: https://contracts.canonical.com
log_level: debug
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ubuntu-advantage-tools/+bug/2060769/+subscriptions
