yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #67525
[Bug 1088611] Re: using random hostnames to detect dns proxies allows for false positives
This bug was fixed in the package cloud-init -
0.7.9-233-ge586fe35-0ubuntu1~16.04.1
---------------
cloud-init (0.7.9-233-ge586fe35-0ubuntu1~16.04.1) xenial-proposed; urgency=medium
* debian/cloud-init.templates: enable Scaleway cloud.
* debian/cloud-init.templates: enable Aliyun cloud.
* drop the following cherry picks, now incorporated in snapshot.
+ debian/patches/cpick-5fb49bac-azure-identify-platform...
+ debian/patches/cpick-003c6678-net-remove-systemd-link...
+ debian/patches/cpick-1cd4323b-azure-remove-accidental...
+ debian/patches/cpick-ebc9ecbc-Azure-Add-network-config...
+ debian/patches/cpick-11121fe4-systemd-make-cloud-final...
* debian/patches/stable-release-no-jsonschema-dep.patch:
add patch to remove optional dependency on jsonschema.
* New upstream snapshot.
- cloudinit.net: add initialize_network_device function and tests
[Chad Smith]
- makefile: fix ci-deps-ubuntu target [Chad Smith]
- tests: adjust locale integration test to parse default locale.
- tests: remove 'yakkety' from releases as it is EOL.
- centos: do not package systemd-fsck drop-in.
- systemd: make systemd-fsck run after cloud-init.service (LP: #1691489)
- tests: Add initial tests for EC2 and improve a docstring.
- locale: Do not re-run locale-gen if provided locale is system default.
- archlinux: fix set hostname usage of write_file. [Joshua Powers]
- sysconfig: support subnet type of 'manual'.
- tools/run-centos: make running with no argument show help.
- Drop rand_str() usage in DNS redirection detection
[Bob Aman] (LP: #1088611)
- sysconfig: use MACADDR on bonds/bridges to configure mac_address
[Ryan Harper]
- net: eni route rendering missed ipv6 default route config
[Ryan Harper] (LP: #1701097)
- sysconfig: enable mtu set per subnet, including ipv6 mtu
[Ryan Harper]
- sysconfig: handle manual type subnets [Ryan Harper]
- sysconfig: fix ipv6 gateway routes [Ryan Harper]
- sysconfig: fix rendering of bond, bridge and vlan types.
[Ryan Harper]
- Templatize systemd unit files for cross distro deltas. [Ryan Harper]
- sysconfig: ipv6 and default gateway fixes. [Ryan Harper]
- net: fix renaming of nics to support mac addresses written in upper
case. (LP: #1705147)
- tests: fixes for issues uncovered when moving to python 3.6.
- sysconfig: include GATEWAY value if set in subnet
[Ryan Harper]
- Scaleway: add datasource with user and vendor data for Scaleway.
[Julien Castets]
- Support comments in content read by load_shell_content.
- cloudinitlocal fail to run during boot [Hongjiang Zhang]
- doc: fix disk setup example table_type options [Sandor Zeestraten]
- tools: Fix exception handling. [Joonas Kylmälä]
- tests: fix usage of mock in GCE test.
- test_gce: Fix invalid mock of platform_reports_gce to return False
[Chad Smith]
- test: fix incorrect keyid for apt repository. [Joshua Powers]
- tests: Update version of pylxd [Joshua Powers]
- write_files: Remove log from helper function signatures.
[Andrew Jorgensen]
- doc: document the cmdline options to NoCloud [Brian Candler]
- read_dmi_data: always return None when inside a container. (LP: #1701325)
- requirements.txt: remove trailing white space.
- Tests: Simplify the check on ssh-import-id [Joshua Powers]
- tests: update ntp tests after sntp added [Joshua Powers]
- FreeBSD: Make freebsd a variant, fix unittests and
tools/build-on-freebsd.
- FreeBSD: fix test failure
- FreeBSD: replace ifdown/ifup with "ifconfig down" and "ifconfig up".
[Hongjiang Zhang]
- FreeBSD: fix cdrom mounting failure if /mnt/cdrom/secure did not exist.
[Hongjiang Zhang]
- main: Don't use templater to format the welcome message
[Andrew Jorgensen]
- docs: Automatically generate module docs form schema if present.
[Chad Smith]
- debian: fix path comment in /etc/hosts template. [Jens Sandmann]
- suse: add hostname and fully qualified domain to template.
[Jens Sandmann]
- write_file(s): Print permissions as octal, not decimal [Andrew Jorgensen]
- ci deps: Add --test-distro to read-dependencies to install all deps
[Chad Smith]
- tools/run-centos: cleanups and move to using read-dependencies
- pkg build ci: Add make ci-deps-<distro> target to install pkgs
[Chad Smith]
- selinux: Allow restorecon to be non-fatal. [Ryan Harper]
- net: Allow netinfo subprocesses to return 0 or 1 due to selinux.
[Ryan Harper]
- net: Allow for NetworkManager configuration [Ryan McCabe]
- Use distro release version to determine if we use systemd in redhat spec
[Ryan Harper]
- net: normalize data in network_state object
- Integration Testing: tox env, pyxld 2.2.3, and revamp framework
[Wesley Wiedenmeier]
- Chef: Update omnibus url to chef.io, minor doc changes. [JJ Asghar]
- tools: add centos scripts to build and test [Joshua Powers]
- Drop cheetah python module as it is not needed by trunk [Ryan Harper]
- rhel/centos spec cleanups.
- cloud.cfg: move to a template. setup.py changes along the way.
- Makefile: add deb-src and srpm targets. use PYVER more places.
- makefile: fix python 2/3 detection in the Makefile [Chad Smith]
- snap: Removing snapcraft plug line [Joshua Powers]
- RHEL/CentOS: Fix default routes for IPv4/IPv6 configuration.
[Andreas Karis]
- test: Fix pyflakes complaint of unused import. [Joshua Powers]
- NoCloud: support seed of nocloud from smbios information
[Vladimir Pouzanov] (LP: #1691772)
- net: when selecting a network device, use natural sort order
[Marc-Aurèle Brothier]
- fix typos and remove whitespace in various docs [Stephan Telling]
- systemd: Fix typo in comment in cloud-init.target. [Chen-Han Hsiao]
- Tests: Skip jsonschema related unit tests when dependency is absent.
[Chad Smith]
- tools/net-convert.py: support old cloudinit versions by using kwargs.
- ntp: Add schema definition and passive schema validation.
[Chad Smith] (LP: #1692916)
- Fix eni rendering for bridge params that require repeated key for
values. [Ryan Harper] (LP: #1706752)
- AliYun: Enable platform identification and enable by default.
[Junjie Wang] (LP: #1638931)
-- Scott Moser <smoser@xxxxxxxxxx> Mon, 31 Jul 2017 16:36:16 -0400
** Changed in: cloud-init (Ubuntu Xenial)
Status: Fix Committed => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1088611
Title:
using random hostnames to detect dns proxies allows for false
positives
Status in cloud-init:
Fix Committed
Status in cloud-init package in Ubuntu:
Fix Released
Status in cloud-init source package in Xenial:
Fix Released
Status in cloud-init source package in Zesty:
Fix Released
Bug description:
=== Begin SRU Template ===
[Impact]
Prior to this fix, cloud-init attempts to detect dns redirection by doing
dns queries for a random hostname and two invalid hostnames. Then, if
the result returned for the input value was the same as the response for
the invalid query cloud-init would assume that result was also invalid.
The change was to replace the random string with
__cloud_init_expected_not_found__
This is a valid hostname and resolution will use the 'search' path in
resolv.conf where the other invalid domain names would not.
[Test Case]
The test case for this consists of excercising the the 'is_resolvable_url'
method in cloudinit.util and watching dns queries. To do this, see the
following steps:
a.) start an lxc container
$ release=xenial
$ name=$release-1088611
$ lxc launch ubuntu-daily:$release $name
b.) start a dnsmasq server
$ ./run-dnsmasq lxdbr0
...
=== listening on 10.75.205.2/24 ===
# run-dnsmasq is attached and at
# https://git.launchpad.net/~smoser/cloud-init/+git/sru-info/tree/bugs/lp-1088611/run-dnsmasq
c.) point /etc/resolv.conf at your server ip
$ lxc exec $name -- sh -c 'exec >/etc/resolv.conf;
echo nameserver 10.75.205.2; echo search foo;'
d.) perform query via is_resolvable_url watch dnsmasq output, expect
to see the random query.
$ lxc exec $name -- python3 -c 'import sys;
from cloudinit.util import is_resolvable_url;
print(is_resolvable_url(sys.argv[1]))' http://ubuntu.com
e.) upgrade to -proposed version
f.) perform query via is_resolvable_url, expect to *not* see random query.
[Regression Potential]
Immediate regression seems unlikely. Effectively the change in cloud-init
code path was simply to change a dns lookup attempt from rand() to a defined
string.
We chose a random string initially to make it difficult for a dns server to
circumvent cloud-init's attempt to identify dns redirection. The regression
path really then seems to involve a dns redirection service specifically
provding a response for '__cloud_init_expected_not_found__' that differs
from does-not-exist.example.com. Cloud-init could then be tricked into
believing that a apt mirror was valid where it previously would have
identified the dns redirection. The failure would be seen as errors
in package installation or 'apt-get update'.
[Other Info]
Upstream commit at
https://git.launchpad.net/cloud-init/commit/?id=42a7b34a12
Original upstream commit at
https://git.launchpad.net/cloud-init/commit/?id=1bb67be5bd
=== End SRU Template ===
The fix that's been applied for bug #974509 checks for the presence of
a redirector by looking of three hostnames, and treating as invalid
any results pointing to a matching address:
- does-not-exist.example.com.
- example.invalid.
- a random, unqualified 32-character alphanumeric hostname.
The last of these carries a small but non-zero risk of colliding with
a real hostname, and there's a small but non-zero risk that this host
points to the same address as something we care about. If possible,
it would be better to not include this random-host lookup in the
algorithm, as somewhere, some day, chances are there will eventually
be a collision, causing an incomprehensible and unreproducible failure
for a user.
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1088611/+subscriptions
