yahoo-eng-team team mailing list archive

[Chad Smith <1992711@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

cloud-init 22.3 reporting configuration doesn't report paired
start/finish event_types for all of cloud-init's boot stages.


Expected behavior:
  cloud-init boot stages (init-local, init-network, config-modules and config-final) to emit both event_type start and finish to represent entry and exit of each boot stage and result "SUCCESS" or "FAILURE".


Results seen:
 Only intermittently see init-network finish events, otherwise strictly event children within the named scope of init-local/*, init-network/*, modules-config/* and modules-final/* publish both start and finish events.


# desired init-network event only seen sometimes.
{"name": "init-network", "description": "searching for network datasources", "event_type": "finish", "origin": "cloudinit", "timestamp": 1665613954.3995728, "result": "SUCCESS"}


Test Procedure:

# terminal 1 on LXC host setup simple http server to log POST
name:event_type tuples

cat > report_server.py << EOF
#!/usr/bin/env python3

from http.server import BaseHTTPRequestHandler, HTTPServer
import logging
import json

class S(BaseHTTPRequestHandler):
    def _set_response(self):
        self.send_response(200)
        self.send_header('Content-type', 'text/plain')
        self.end_headers()

    def do_POST(self):
        content_length = int(self.headers['Content-Length'])
        post_data = self.rfile.read(content_length)
        json_data = json.loads(post_data.decode('utf-8'))
        msg =  f"{json_data['name']}:{json_data['event_type']}"
        logging.info(msg)
        self._set_response()


def run(server_class=HTTPServer, handler_class=S, port=8080):
    logging.basicConfig(level=logging.INFO)
    server_address = ('', port)
    httpd = server_class(server_address, handler_class)
    logging.info('Starting httpd...\n')
    try:
        httpd.serve_forever()
    except KeyboardInterrupt:
        pass
    httpd.server_close()
    logging.info('Stopping httpd...\n')

if __name__ == '__main__':
    from sys import argv

    if len(argv) == 2:
        run(port=int(argv[1]))
    else:
        run()

EOF

python3 report_server.py 2> /dev/null


#### Terminal 2
# Setup LXC container with a reporting configuration which will talk to the LXC host
cat > 99-reporting.cfg <<EOF
reporting:
  webserver:
    type: webhook
    endpoint: "http://_gateway:8080/";
    timeout: 5
    retries: 3
EOF


lxc launch ubuntu-daily:kinetic test-reporting
lxc exec test-reporting -- cloud-init status --wait
lxc file push 99-reporting.cfg test-reporting/etc/cloud/cloud.cfg.d
lxc exec test-reporting -- cloud-init clean --logs --reboot


# on the simple http server terminal we end up seeing only event
children, we only occasionally see init-network:finish  events.
Typically the parent event start and finish is not logged and it is only
child events within the boot stage init-local, init-network, modules-
config and modules-final this gets published in event handler


init-local/check-cache:start
init-local/check-cache:finish
init-local/search-LXD:start
init-local/search-LXD:finish
init-network/check-cache:start
init-network/check-cache:finish
init-network/setup-datasource:start
init-network/setup-datasource:finish
init-network/consume-user-data:start
init-network/consume-user-data:finish
init-network/consume-vendor-data:start
init-network/consume-vendor-data:finish
init-network/consume-vendor-data2:start
init-network/consume-vendor-data2:finish
init-network/activate-datasource:start
init-network/activate-datasource:finish
init-network/config-migrator:start
init-network/config-migrator:finish
init-network/config-seed_random:start
init-network/config-seed_random:finish
init-network/config-growpart:start
init-network/config-growpart:finish
init-network/config-resizefs:start
init-network/config-resizefs:finish
init-network/config-mounts:start
init-network/config-mounts:finish
init-network/config-set_hostname:start
init-network/config-set_hostname:finish
init-network/config-update_hostname:start
init-network/config-update_hostname:finish
init-network/config-users-groups:start
init-network/config-users-groups:finish
init-network/config-ssh:start
init-network/config-ssh:finish
modules-config/config-ssh-import-id:start
modules-config/config-ssh-import-id:finish
modules-config/config-locale:start
modules-config/config-locale:finish
modules-config/config-set-passwords:start
modules-config/config-set-passwords:finish
modules-config/config-grub-dpkg:start
modules-config/config-grub-dpkg:finish
modules-config/config-apt-configure:start
modules-config/config-apt-configure:finish
modules-config/config-ubuntu-advantage:start
modules-config/config-ubuntu-advantage:finish
modules-config/config-byobu:start
modules-config/config-byobu:finish
modules-final/config-reset_rmc:start
modules-final/config-reset_rmc:finish
modules-final/config-refresh_rmc_and_interface:start
modules-final/config-refresh_rmc_and_interface:finish
modules-final/config-rightscale_userdata:start
modules-final/config-rightscale_userdata:finish
modules-final/config-scripts-vendor:start
modules-final/config-scripts-vendor:finish
modules-final/config-scripts-per-once:start
modules-final/config-scripts-per-once:finish
modules-final/config-scripts-per-boot:start
modules-final/config-scripts-per-boot:finish
modules-final/config-scripts-per-instance:start
modules-final/config-scripts-per-instance:finish
modules-final/config-scripts-user:start
modules-final/config-scripts-user:finish
modules-final/config-ssh-authkey-fingerprints:start
modules-final/config-ssh-authkey-fingerprints:finish
modules-final/config-keys-to-console:start
modules-final/config-keys-to-console:finish
modules-final/config-install-hotplug:start
modules-final/config-install-hotplug:finish
modules-final/config-final-message:start
modules-final/config-final-message:finish

** Affects: cloud-init
     Importance: Undecided
         Status: Triaged

** Changed in: cloud-init
       Status: New => Triaged

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1992711

Title:
  cloud-init reporting does not report boot stage start;finish events
  for  init-local, init-network, modules-config, modules-final

Status in cloud-init:
  Triaged

Bug description:
  cloud-init 22.3 reporting configuration doesn't report paired
  start/finish event_types for all of cloud-init's boot stages.

  
  Expected behavior:
    cloud-init boot stages (init-local, init-network, config-modules and config-final) to emit both event_type start and finish to represent entry and exit of each boot stage and result "SUCCESS" or "FAILURE".

  
  Results seen:
   Only intermittently see init-network finish events, otherwise strictly event children within the named scope of init-local/*, init-network/*, modules-config/* and modules-final/* publish both start and finish events.

  
  # desired init-network event only seen sometimes.
  {"name": "init-network", "description": "searching for network datasources", "event_type": "finish", "origin": "cloudinit", "timestamp": 1665613954.3995728, "result": "SUCCESS"}

  
  Test Procedure:

  # terminal 1 on LXC host setup simple http server to log POST
  name:event_type tuples

  cat > report_server.py << EOF
  #!/usr/bin/env python3

  from http.server import BaseHTTPRequestHandler, HTTPServer
  import logging
  import json

  class S(BaseHTTPRequestHandler):
      def _set_response(self):
          self.send_response(200)
          self.send_header('Content-type', 'text/plain')
          self.end_headers()

      def do_POST(self):
          content_length = int(self.headers['Content-Length'])
          post_data = self.rfile.read(content_length)
          json_data = json.loads(post_data.decode('utf-8'))
          msg =  f"{json_data['name']}:{json_data['event_type']}"
          logging.info(msg)
          self._set_response()

  
  def run(server_class=HTTPServer, handler_class=S, port=8080):
      logging.basicConfig(level=logging.INFO)
      server_address = ('', port)
      httpd = server_class(server_address, handler_class)
      logging.info('Starting httpd...\n')
      try:
          httpd.serve_forever()
      except KeyboardInterrupt:
          pass
      httpd.server_close()
      logging.info('Stopping httpd...\n')

  if __name__ == '__main__':
      from sys import argv

      if len(argv) == 2:
          run(port=int(argv[1]))
      else:
          run()

  EOF

  python3 report_server.py 2> /dev/null

  
  #### Terminal 2
  # Setup LXC container with a reporting configuration which will talk to the LXC host
  cat > 99-reporting.cfg <<EOF
  reporting:
    webserver:
      type: webhook
      endpoint: "http://_gateway:8080/";
      timeout: 5
      retries: 3
  EOF

  
  lxc launch ubuntu-daily:kinetic test-reporting
  lxc exec test-reporting -- cloud-init status --wait
  lxc file push 99-reporting.cfg test-reporting/etc/cloud/cloud.cfg.d
  lxc exec test-reporting -- cloud-init clean --logs --reboot


  # on the simple http server terminal we end up seeing only event
  children, we only occasionally see init-network:finish  events.
  Typically the parent event start and finish is not logged and it is
  only child events within the boot stage init-local, init-network,
  modules-config and modules-final this gets published in event handler

  
  init-local/check-cache:start
  init-local/check-cache:finish
  init-local/search-LXD:start
  init-local/search-LXD:finish
  init-network/check-cache:start
  init-network/check-cache:finish
  init-network/setup-datasource:start
  init-network/setup-datasource:finish
  init-network/consume-user-data:start
  init-network/consume-user-data:finish
  init-network/consume-vendor-data:start
  init-network/consume-vendor-data:finish
  init-network/consume-vendor-data2:start
  init-network/consume-vendor-data2:finish
  init-network/activate-datasource:start
  init-network/activate-datasource:finish
  init-network/config-migrator:start
  init-network/config-migrator:finish
  init-network/config-seed_random:start
  init-network/config-seed_random:finish
  init-network/config-growpart:start
  init-network/config-growpart:finish
  init-network/config-resizefs:start
  init-network/config-resizefs:finish
  init-network/config-mounts:start
  init-network/config-mounts:finish
  init-network/config-set_hostname:start
  init-network/config-set_hostname:finish
  init-network/config-update_hostname:start
  init-network/config-update_hostname:finish
  init-network/config-users-groups:start
  init-network/config-users-groups:finish
  init-network/config-ssh:start
  init-network/config-ssh:finish
  modules-config/config-ssh-import-id:start
  modules-config/config-ssh-import-id:finish
  modules-config/config-locale:start
  modules-config/config-locale:finish
  modules-config/config-set-passwords:start
  modules-config/config-set-passwords:finish
  modules-config/config-grub-dpkg:start
  modules-config/config-grub-dpkg:finish
  modules-config/config-apt-configure:start
  modules-config/config-apt-configure:finish
  modules-config/config-ubuntu-advantage:start
  modules-config/config-ubuntu-advantage:finish
  modules-config/config-byobu:start
  modules-config/config-byobu:finish
  modules-final/config-reset_rmc:start
  modules-final/config-reset_rmc:finish
  modules-final/config-refresh_rmc_and_interface:start
  modules-final/config-refresh_rmc_and_interface:finish
  modules-final/config-rightscale_userdata:start
  modules-final/config-rightscale_userdata:finish
  modules-final/config-scripts-vendor:start
  modules-final/config-scripts-vendor:finish
  modules-final/config-scripts-per-once:start
  modules-final/config-scripts-per-once:finish
  modules-final/config-scripts-per-boot:start
  modules-final/config-scripts-per-boot:finish
  modules-final/config-scripts-per-instance:start
  modules-final/config-scripts-per-instance:finish
  modules-final/config-scripts-user:start
  modules-final/config-scripts-user:finish
  modules-final/config-ssh-authkey-fingerprints:start
  modules-final/config-ssh-authkey-fingerprints:finish
  modules-final/config-keys-to-console:start
  modules-final/config-keys-to-console:finish
  modules-final/config-install-hotplug:start
  modules-final/config-install-hotplug:finish
  modules-final/config-final-message:start
  modules-final/config-final-message:finish

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1992711/+subscriptions