← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1757054] [NEW] OpenStack datasource does not deal with 404 Not Found correctly

 

Public bug reported:

OpenStack datasource does not deal with 404 Not Found correctly.If we
boot a vm without userdata in OpenStack environment, CloudInit retry to
get usedata when 404 Not Found returned. The GuestOS start up times will
increase.I think it's not needed and a waste of time.

Cloud-Init LOG:

2018-03-19 11:09:16,391 - url_helper.py[DEBUG]: [0/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration
2018-03-19 11:09:16,414 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
2018-03-19 11:09:17,415 - url_helper.py[DEBUG]: [1/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration
2018-03-19 11:09:17,445 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
2018-03-19 11:09:18,446 - url_helper.py[DEBUG]: [2/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration
2018-03-19 11:09:18,468 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
2018-03-19 11:09:19,470 - url_helper.py[DEBUG]: [3/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration
2018-03-19 11:09:19,535 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
2018-03-19 11:09:20,536 - url_helper.py[DEBUG]: [4/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration
2018-03-19 11:09:20,556 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
2018-03-19 11:09:21,557 - url_helper.py[DEBUG]: [5/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration
2018-03-19 11:09:21,584 - openstack.py[DEBUG]: Failed reading optional path http://169.254.169.254:80/openstack/2015-10-15/user_data due to: 404 Client Error: Not Found

Go through the code, I found the following logic which i think it's not correct.
cloudinit/sources/helpers/openstack.py
def should_retry_cb(_request_args, cause):
    try:
        code = int(cause.code)
        if code >= 400:
            return False
    except (TypeError, ValueError):
        # Older versions of requests didn't have a code.
        pass
    return True

response = url_helper.readurl(path,
               retries=self.retries,
               ssl_details=self.ssl_details,
               timeout=self.timeout,
                exception_cb=should_retry_cb)

1.When 404 returned, do not need to retry.
2.exception_cb=should_retry_cb need to be changed to exception_cb=should_skip_retry_cb.  May be a Bug here:)

** Affects: cloud-init
     Importance: Undecided
         Status: New

** Description changed:

  OpenStack datasource does not deal with 404 Not Found correctly.If we
  boot a vm without userdata in OpenStack environment, CloudInit retry to
  get usedata when 404 Not Found returned. The GuestOS start up times will
  increase.I think it's not needed and a waste of time.
- 
  
  Cloud-Init LOG:
  
  2018-03-19 11:09:16,391 - url_helper.py[DEBUG]: [0/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration
  2018-03-19 11:09:16,414 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
  2018-03-19 11:09:17,415 - url_helper.py[DEBUG]: [1/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration
  2018-03-19 11:09:17,445 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
  2018-03-19 11:09:18,446 - url_helper.py[DEBUG]: [2/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration
  2018-03-19 11:09:18,468 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
  2018-03-19 11:09:19,470 - url_helper.py[DEBUG]: [3/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration
  2018-03-19 11:09:19,535 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
  2018-03-19 11:09:20,536 - url_helper.py[DEBUG]: [4/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration
  2018-03-19 11:09:20,556 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
  2018-03-19 11:09:21,557 - url_helper.py[DEBUG]: [5/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration
  2018-03-19 11:09:21,584 - openstack.py[DEBUG]: Failed reading optional path http://169.254.169.254:80/openstack/2015-10-15/user_data due to: 404 Client Error: Not Found
  
- 
  Go through the code, I found the following logic which i think it's not correct.
  cloudinit/sources/helpers/openstack.py
-         def should_retry_cb(_request_args, cause):
-             try:
-                 code = int(cause.code)
-                 if code >= 400:
-                     return False
-             except (TypeError, ValueError):
-                 # Older versions of requests didn't have a code.
-                 pass
-             return True
+ def should_retry_cb(_request_args, cause):
+     try:
+         code = int(cause.code)
+         if code >= 400:
+             return False
+     except (TypeError, ValueError):
+         # Older versions of requests didn't have a code.
+         pass
+     return True
  
-         response = url_helper.readurl(path,
-                                       retries=self.retries,
-                                       ssl_details=self.ssl_details,
-                                       timeout=self.timeout,
-                                       exception_cb=should_retry_cb)
+ response = url_helper.readurl(path,
+                retries=self.retries,
+                ssl_details=self.ssl_details,
+                timeout=self.timeout,
+                 exception_cb=should_retry_cb)
+ 
  1.When 404 returned, do not need to retry.
  2.exception_cb=should_retry_cb need to be changed to exception_cb=should_skip_retry_cb.  May be a Bug here:)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1757054

Title:
  OpenStack datasource does not deal with 404 Not Found correctly

Status in cloud-init:
  New

Bug description:
  OpenStack datasource does not deal with 404 Not Found correctly.If we
  boot a vm without userdata in OpenStack environment, CloudInit retry
  to get usedata when 404 Not Found returned. The GuestOS start up times
  will increase.I think it's not needed and a waste of time.

  Cloud-Init LOG:

  2018-03-19 11:09:16,391 - url_helper.py[DEBUG]: [0/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration
  2018-03-19 11:09:16,414 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
  2018-03-19 11:09:17,415 - url_helper.py[DEBUG]: [1/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration
  2018-03-19 11:09:17,445 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
  2018-03-19 11:09:18,446 - url_helper.py[DEBUG]: [2/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration
  2018-03-19 11:09:18,468 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
  2018-03-19 11:09:19,470 - url_helper.py[DEBUG]: [3/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration
  2018-03-19 11:09:19,535 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
  2018-03-19 11:09:20,536 - url_helper.py[DEBUG]: [4/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration
  2018-03-19 11:09:20,556 - url_helper.py[DEBUG]: Please wait 1 seconds while we wait to try again
  2018-03-19 11:09:21,557 - url_helper.py[DEBUG]: [5/6] open 'http://169.254.169.254:80/openstack/2015-10-15/user_data' with {'url': 'http://169.254.169.254:80/openstack/2015-10-15/user_data', 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0} configuration
  2018-03-19 11:09:21,584 - openstack.py[DEBUG]: Failed reading optional path http://169.254.169.254:80/openstack/2015-10-15/user_data due to: 404 Client Error: Not Found

  Go through the code, I found the following logic which i think it's not correct.
  cloudinit/sources/helpers/openstack.py
  def should_retry_cb(_request_args, cause):
      try:
          code = int(cause.code)
          if code >= 400:
              return False
      except (TypeError, ValueError):
          # Older versions of requests didn't have a code.
          pass
      return True

  response = url_helper.readurl(path,
                 retries=self.retries,
                 ssl_details=self.ssl_details,
                 timeout=self.timeout,
                  exception_cb=should_retry_cb)

  1.When 404 returned, do not need to retry.
  2.exception_cb=should_retry_cb need to be changed to exception_cb=should_skip_retry_cb.  May be a Bug here:)

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1757054/+subscriptions


Follow ups