enterprise-support team mailing list archive

[Scott Moser <smoser@xxxxxxxxxx>]

Public bug reported:

Many people run squid (squid-deb-proxy, or maas-proxy) to provide ubuntu
archive mirror caching and proxying.  MAAS sets this up by default for
users with the 'maas-proxy' package.

On or about Friday February 19, this setup began to fail for many people.
Users would see 'apt-get update' returning 503 errors.  For me, I saw 503 on security.ubuntu.com addresses.

The reason for the failure was that the squid proxy began using ipv6
addresses for instead of ipv4.  The squid proxy host did not have ipv6
connectivity and thus would fail.

The fix/workaround is to add the following to your squid config:
  # http://www.squid-cache.org/Doc/config/dns_v4_first/
  dns_v4_first on

The appropriate squid config file depends on what is running squid.
  maas-proxy: /usr/share/maas/maas-proxy.conf
  squid-deb-proxy: /etc/init/squid-deb-proxy.conf

I'm not sure how this previously worked, nor what change was made.
One change that was made in this time frame was a glibc update (2.19-0ubuntu6.6 to 2.19-0ubuntu6.7) for security (CVE-2013-7423 CVE-2014-9402 CVE-2015-1472 CVE-2015-1473).  But it doesn't seem to make sense that that would change squid3 to start looking for AAAA records when it did not previously.
i can verify that as late as
  Thu Feb 18 06:36:07 EST 2016
i was seeing entries in my squid logs with
  1455713142.896    335 10.7.2.103 TCP_REFRESH_UNMODIFIED/200 82620 GET http://security.ubuntu.com/ubuntu/dists/xenial-security/InRelease - HIER_DIRECT/91.189.88.149 -
but now i get
  1455879482.210      1 10.7.2.103 TCP_REFRESH_FAIL/200 635 GET http://security.ubuntu.com/ubuntu/dists/precise-security/main/i18n/Index - HIER_DIRECT/2001:67c:1562::14 -

** Affects: maas
     Importance: Undecided
         Status: New

** Affects: squid (Ubuntu)
     Importance: Undecided
         Status: Confirmed

** Affects: squid-deb-proxy (Ubuntu)
     Importance: Undecided
         Status: Confirmed

** Also affects: squid (Ubuntu)
   Importance: Undecided
       Status: New

** Also affects: squid-deb-proxy (Ubuntu)
   Importance: Undecided
       Status: New

** Description changed:

  Many people run squid (squid-deb-proxy, or maas-proxy) to provide ubuntu
  archive mirror caching and proxying.  MAAS sets this up by default for
  users with the 'maas-proxy' package.
  
  On or about Friday February 19, this setup began to fail for many people.
  Users would see 'apt-get update' returning 503 errors.  For me, I saw 503 on security.ubuntu.com addresses.
  
  The reason for the failure was that the squid proxy began using ipv6
  addresses for instead of ipv4.  The squid proxy host did not have ipv6
  connectivity and thus would fail.
  
  The fix/workaround is to add the following to your squid config:
-   # http://www.squid-cache.org/Doc/config/dns_v4_first/
-   dns_v4_first on
+   # http://www.squid-cache.org/Doc/config/dns_v4_first/
+   dns_v4_first on
  
  The appropriate squid config file depends on what is running squid.
-   maas-proxy: /usr/share/maas/maas-proxy.conf
-   squid-deb-proxy: /etc/init/squid-deb-proxy.conf
+   maas-proxy: /usr/share/maas/maas-proxy.conf
+   squid-deb-proxy: /etc/init/squid-deb-proxy.conf
  
  I'm not sure how this previously worked, nor what change was made.
  One change that was made in this time frame was a glibc update (2.19-0ubuntu6.6 to 2.19-0ubuntu6.7) for security (CVE-2013-7423 CVE-2014-9402 CVE-2015-1472 CVE-2015-1473).  But it doesn't seem to make sense that that would change squid3 to start looking for AAAA records when it did not previously.
- 
-  
- but if you run squid on a host that has ipv6 , but no ipv6 connectivity
- --> Jonathanb (textual@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx) has joined #canonical
- <-- Grazina has quit (Client exited)
- <smoser> it used to work and end up getting ipv4 addresses
- <smoser> something changed today or yesterday
- <-- bkhan_AWAY has quit (Ping timeout: 121 seconds)
- <smoser> and now that is getting ipv6 addresses.
- <-- xnox has quit (Quit: ZNC - http://znc.in)
- <smoser> meaning if you have no ipv6 route to security.ubuntu.com, you get 503 on 'apt-get update'
- <smoser> i can verify that as late as
- <smoser>  human time: Thu Feb 18 06:36:07 EST 2016
- <-- verterok has quit (Quit: Coyote finally caught me)
- --- john-mcaleely|away is now known as john-mcaleely
- <smoser> i was seeing entries in my squid logs with
- <-- alan_g has quit (Quit: Ex-Chat)
- <smoser>  human time: Thu Feb 18 06:36:07 EST 2016
- <-- mup has quit (Connection closed)
- <smoser> 1455713142.896    335 10.7.2.103 TCP_REFRESH_UNMODIFIED/200 82620 GET http://security.ubuntu.com/ubuntu/dists/xenial-security/InRelease - HIER_DIRECT/91.189.88.149 -
- <smoser> but now i get
- <smoser> 1455879482.210      1 10.7.2.103 TCP_REFRESH_FAIL/200 635 GET http://security.ubuntu.com/ubuntu/dists/precise-security/main/i18n/Index - HIER_DIRECT/2001:67c:1562::14 -
+ i can verify that as late as
+   Thu Feb 18 06:36:07 EST 2016
+ i was seeing entries in my squid logs with
+   1455713142.896    335 10.7.2.103 TCP_REFRESH_UNMODIFIED/200 82620 GET http://security.ubuntu.com/ubuntu/dists/xenial-security/InRelease - HIER_DIRECT/91.189.88.149 -
+ but now i get
+   1455879482.210      1 10.7.2.103 TCP_REFRESH_FAIL/200 635 GET http://security.ubuntu.com/ubuntu/dists/precise-security/main/i18n/Index - HIER_DIRECT/2001:67c:1562::14 -

-- 
You received this bug notification because you are a member of Ubuntu
Server/Client Support Team, which is subscribed to squid in Ubuntu.
Matching subscriptions: Ubuntu Server/Client Support Team
https://bugs.launchpad.net/bugs/1547640

Title:
  proxy tries ipv6 and gets 503 when no ipv6 routes

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1547640/+subscriptions