yahoo-eng-team team mailing list archive

[Andrew Bogott <1914641@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

I use jinja templating for vendor data; it works with my .deb packaged
version of cloud-init, 20.2-2~deb10u1

Testing with the latest git checkout, I see a json parser chocking on
curly braces.  That suggests that it's skipping the jinja rendering
step, or trying to run it after json parsing, which won't work.


Here is the top part of my vendor data:
```
root@cloudinit-test:~# curl http://169.254.169.254/openstack/latest/vendor_data.json
{"domain": "codfw1dev.wikimedia.cloud", "cloud-init": "MIME-Version: 1.0\nContent-Type: multipart/mixed; boundary=\"XXXXboundary text\"\n\nThis is a multipart config in MIME format.\nIt contains a cloud-init config followed by\na first boot script.\n\n--XXXXboundary text\nMIME-Version: 1.0\nContent-Type: text/cloud-config; charset=\"us-ascii\"\n\n## template: jinja\n#cloud-config\n\nhostname: {{ds.meta_data.name}}\nfqdn: {{ds.meta_data.name}}.{{ds.meta_data.project_id}}.codfw1dev.wikimedia.cloud\n\n\n# /etc/block-ldap-key-lookup:\n#   Prevent non-root logins while the VM is being setup\n#   The ssh-key-ldap-lookup script rejects non-root user logins if this file\n#   is present.\n#\n# /etc/rsyslog.d/60-puppet.conf:\n#   Enable console logging for puppet\n#\n# /etc/systemd/system/serial-getty@ttyS0.service.d/override.conf:\n#   Enable root console on serial0\n#   (cloud-init will create any needed parent dirs)\nwrite_files:\n    - content: \"VM is work in progress\"\n      path: /etc/block-ldap-key-lookup\n    - content: \"daemon.* |/dev/console\"\n      path: /etc/rsyslog.d/60-puppet.conf\n    - content: |\n        [Service]\n        ExecStart=\n        ExecStart=-/sbin/agetty --autologin root --noclear %I $TERM\n      path: /etc/systemd/system/serial-getty@ttyS0.service.d/override.conf\n\n# resetting ttys0 so root is logged in\nruncmd:\n    - [systemctl, enable, serial-getty@ttyS0.service]\n    - [systemctl, restart, serial-getty@ttyS0.service]\n\n\nmanage_etc_hosts: true\n\npackages:\n    - gpg\n    - curl\n    - nscd\n    - lvm2\n    - parted\n    - puppet\n\ngrowpart:\n    mode: false\n\n# You'll see that we're setting apt_preserve_sources_list twice here.  That's\n#  because there's a bug in cloud-init where it tries to reconcile the\n#  two settings and if they're different the stage fails. That means that\n#  if one of them is set differently from the default (True) then nothing\n#  works.\napt_preserve_sources_list: False\napt:\n    preserve_sources_list: False\n 
```


And here are the errors:


```
2021-02-04 18:08:43,117 - util.py[WARNING]: Failed loading yaml blob. Invalid format at line 4 column 1: "while parsing a block mapping
  in "<unicode string>", line 4, column 1:
    hostname: {{ds.meta_data.name}}
    ^
expected <block end>, but found '<scalar>'
  in "<unicode string>", line 5, column 28:
    fqdn: {{ds.meta_data.name}}.{{ds.meta_data.project_id}}.cod ... 
                               ^"
2021-02-04 18:08:43,131 - util.py[WARNING]: Failed loading yaml blob. Invalid format at line 4 column 1: "while parsing a block mapping
  in "<unicode string>", line 4, column 1:
    hostname: {{ds.meta_data.name}}
    ^
expected <block end>, but found '<scalar>'
  in "<unicode string>", line 5, column 28:
    fqdn: {{ds.meta_data.name}}.{{ds.meta_data.project_id}}.cod ... 
                               ^"
2021-02-04 18:08:43,131 - util.py[WARNING]: Failed at merging in cloud config part from part-001
```

** Affects: cloud-init
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1914641

Title:
  jinja rendering broken in latest git checkout

Status in cloud-init:
  New

Bug description:
  I use jinja templating for vendor data; it works with my .deb packaged
  version of cloud-init, 20.2-2~deb10u1

  Testing with the latest git checkout, I see a json parser chocking on
  curly braces.  That suggests that it's skipping the jinja rendering
  step, or trying to run it after json parsing, which won't work.

  
  Here is the top part of my vendor data:
  ```
  root@cloudinit-test:~# curl http://169.254.169.254/openstack/latest/vendor_data.json
  {"domain": "codfw1dev.wikimedia.cloud", "cloud-init": "MIME-Version: 1.0\nContent-Type: multipart/mixed; boundary=\"XXXXboundary text\"\n\nThis is a multipart config in MIME format.\nIt contains a cloud-init config followed by\na first boot script.\n\n--XXXXboundary text\nMIME-Version: 1.0\nContent-Type: text/cloud-config; charset=\"us-ascii\"\n\n## template: jinja\n#cloud-config\n\nhostname: {{ds.meta_data.name}}\nfqdn: {{ds.meta_data.name}}.{{ds.meta_data.project_id}}.codfw1dev.wikimedia.cloud\n\n\n# /etc/block-ldap-key-lookup:\n#   Prevent non-root logins while the VM is being setup\n#   The ssh-key-ldap-lookup script rejects non-root user logins if this file\n#   is present.\n#\n# /etc/rsyslog.d/60-puppet.conf:\n#   Enable console logging for puppet\n#\n# /etc/systemd/system/serial-getty@ttyS0.service.d/override.conf:\n#   Enable root console on serial0\n#   (cloud-init will create any needed parent dirs)\nwrite_files:\n    - content: \"VM is work in progress\"\n      path: /etc/block-ldap-key-lookup\n    - content: \"daemon.* |/dev/console\"\n      path: /etc/rsyslog.d/60-puppet.conf\n    - content: |\n        [Service]\n        ExecStart=\n        ExecStart=-/sbin/agetty --autologin root --noclear %I $TERM\n      path: /etc/systemd/system/serial-getty@ttyS0.service.d/override.conf\n\n# resetting ttys0 so root is logged in\nruncmd:\n    - [systemctl, enable, serial-getty@ttyS0.service]\n    - [systemctl, restart, serial-getty@ttyS0.service]\n\n\nmanage_etc_hosts: true\n\npackages:\n    - gpg\n    - curl\n    - nscd\n    - lvm2\n    - parted\n    - puppet\n\ngrowpart:\n    mode: false\n\n# You'll see that we're setting apt_preserve_sources_list twice here.  That's\n#  because there's a bug in cloud-init where it tries to reconcile the\n#  two settings and if they're different the stage fails. That means that\n#  if one of them is set differently from the default (True) then nothing\n#  works.\napt_preserve_sources_list: False\napt:\n    preserve_sources_list: False\n 
  ```


  And here are the errors:

  
  ```
  2021-02-04 18:08:43,117 - util.py[WARNING]: Failed loading yaml blob. Invalid format at line 4 column 1: "while parsing a block mapping
    in "<unicode string>", line 4, column 1:
      hostname: {{ds.meta_data.name}}
      ^
  expected <block end>, but found '<scalar>'
    in "<unicode string>", line 5, column 28:
      fqdn: {{ds.meta_data.name}}.{{ds.meta_data.project_id}}.cod ... 
                                 ^"
  2021-02-04 18:08:43,131 - util.py[WARNING]: Failed loading yaml blob. Invalid format at line 4 column 1: "while parsing a block mapping
    in "<unicode string>", line 4, column 1:
      hostname: {{ds.meta_data.name}}
      ^
  expected <block end>, but found '<scalar>'
    in "<unicode string>", line 5, column 28:
      fqdn: {{ds.meta_data.name}}.{{ds.meta_data.project_id}}.cod ... 
                                 ^"
  2021-02-04 18:08:43,131 - util.py[WARNING]: Failed at merging in cloud config part from part-001
  ```

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1914641/+subscriptions