yahoo-eng-team team mailing list archive

[Ilia Baikov <2063525@xxxxxxxxxxxxxxxxxx>]

Public bug reported:


Description
===========
I'm developing open source project to allow host vendor_data that stored in Consul.
https://github.com/ib-systems/openstack-consul-dynamic-vendordata

nova-api is configured following way
```
vendordata_providers = DynamicJSON
vendordata_dynamic_targets = 'ocdv@http://10.10.10.10:8000/ocdv'
```
Vendor data returns next response:
```
{
  "ocdv": {
    "cloud-init": "#cloud-config\npackage_upgrade: True\npackages:\n - iotop"
  }
}
```
Everything is following OpenStack docs but no success due to cloud-init looks for "cloud-init" key in response, but no way to return string "#cloud-config" because from each service expected json reply.

Steps to reproduce
==================

* Configure nova-api to use DynamicJSON vendordata
```
[api]
use_forwarded_for = true
vendordata_providers = DynamicJSON
vendordata_dynamic_targets = 'ocdv@http://10.10.10.10:8000/ocdv'
```
* Add any data to your implemented vendor_data storage. In project that i'm developing i'm using Consul to store JSON vendor-data in next format
```
{"cloud-init": "#cloud-config\npackage_upgrade: True\npackages:\n - iotop"}
```

* Boot instance. cloud-init will ignore `cloud-init` key from ocdv service vendor data. `hostname` is added as simple way to check
```
# curl http://169.254.169.254/openstack/latest/vendor_data2.json | jq
```
```
{
  "ocdv": {
    "cloud-init": "#cloud-config\nhostname: ocdvworks\npackage_upgrade: True\npackages:\n - iotop"
  }
}
```
When vendor_data2.json return response like that:
```
{
    "cloud-init": "#cloud-config\nhostname: ocdvworks\npackage_upgrade: True\npackages:\n - iotop"
}
```
everything works correct following cloud-init docs https://cloudinit.readthedocs.io/en/latest/reference/datasources/openstack.html#vendor-data "Cloud-init will look for a cloud-init at the vendor_data2 path"

Expected result
===============
cloud-config is being applied on first boot. 

Actual result
=============
cloud-config is ignored since cloud-init key is wrapped into `ocdv` JSON object.

Environment
===========
1. OpenStack Zed

Conclusion
==========
We can add as much as simple check that vendor data provider equals to `cloud-init` that returns string to let cloud-init work as expected and provide cloud operators way to allow vendor-data for their vendor-specific operations like updating packages for all instances (for security reason for example), setting hostname, etc to divide vendor specific (vendor_data2.json) and user-specific settings (user-data).
```
    if service_name == "cloud-init":
        return res.text
    if res and res.text:
        # TODO(mikal): Use the Cache-Control response header to do some
        # sensible form of caching here.
        return jsonutils.loads(res.text)
```

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2063525

Title:
  cloud-config provided by dynamic vendor data not handled by cloud-init

Status in OpenStack Compute (nova):
  New

Bug description:
  
  Description
  ===========
  I'm developing open source project to allow host vendor_data that stored in Consul.
  https://github.com/ib-systems/openstack-consul-dynamic-vendordata

  nova-api is configured following way
  ```
  vendordata_providers = DynamicJSON
  vendordata_dynamic_targets = 'ocdv@http://10.10.10.10:8000/ocdv'
  ```
  Vendor data returns next response:
  ```
  {
    "ocdv": {
      "cloud-init": "#cloud-config\npackage_upgrade: True\npackages:\n - iotop"
    }
  }
  ```
  Everything is following OpenStack docs but no success due to cloud-init looks for "cloud-init" key in response, but no way to return string "#cloud-config" because from each service expected json reply.

  Steps to reproduce
  ==================

  * Configure nova-api to use DynamicJSON vendordata
  ```
  [api]
  use_forwarded_for = true
  vendordata_providers = DynamicJSON
  vendordata_dynamic_targets = 'ocdv@http://10.10.10.10:8000/ocdv'
  ```
  * Add any data to your implemented vendor_data storage. In project that i'm developing i'm using Consul to store JSON vendor-data in next format
  ```
  {"cloud-init": "#cloud-config\npackage_upgrade: True\npackages:\n - iotop"}
  ```

  * Boot instance. cloud-init will ignore `cloud-init` key from ocdv service vendor data. `hostname` is added as simple way to check
  ```
  # curl http://169.254.169.254/openstack/latest/vendor_data2.json | jq
  ```
  ```
  {
    "ocdv": {
      "cloud-init": "#cloud-config\nhostname: ocdvworks\npackage_upgrade: True\npackages:\n - iotop"
    }
  }
  ```
  When vendor_data2.json return response like that:
  ```
  {
      "cloud-init": "#cloud-config\nhostname: ocdvworks\npackage_upgrade: True\npackages:\n - iotop"
  }
  ```
  everything works correct following cloud-init docs https://cloudinit.readthedocs.io/en/latest/reference/datasources/openstack.html#vendor-data "Cloud-init will look for a cloud-init at the vendor_data2 path"

  Expected result
  ===============
  cloud-config is being applied on first boot. 

  Actual result
  =============
  cloud-config is ignored since cloud-init key is wrapped into `ocdv` JSON object.

  Environment
  ===========
  1. OpenStack Zed

  Conclusion
  ==========
  We can add as much as simple check that vendor data provider equals to `cloud-init` that returns string to let cloud-init work as expected and provide cloud operators way to allow vendor-data for their vendor-specific operations like updating packages for all instances (for security reason for example), setting hostname, etc to divide vendor specific (vendor_data2.json) and user-specific settings (user-data).
  ```
      if service_name == "cloud-init":
          return res.text
      if res and res.text:
          # TODO(mikal): Use the Cache-Control response header to do some
          # sensible form of caching here.
          return jsonutils.loads(res.text)
  ```

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2063525/+subscriptions