cloud-init team mailing list archive

[Ryan Harper <ryan.harper@xxxxxxxxxxxxx>]

On Thu, Feb 8, 2018 at 8:14 PM, Tamilmani Manoharan <tamanoha@xxxxxxxxxxxxx>
wrote:

> Ryan,
>
> The VM has to be configured with new data that should be read after
> network switch and these VM configuration happens even before
> system-networkd starts. We need some mechanism to listen on network switch
> events and issue dhcp. So we can’t rely on system-networkd on resending
> dhcp.
>

The process, as I understand based on what's been committed,
c03bdd3d8ed762cada813c5e95a40b14d2047b57

During early boot at cloud-init local time, which runs before system-wide
networking is enabled, the Azure Datasource will
DHCP on the primary interface, consume metadata, and within that, read the
PreprovisionedVM value and if so, goes down a path of polling
until it's time to come up.   When system execution continues cloud-init
local exits and has written a network configuration to DHCP on the primary
interface.  Then the networking layer is activated and system blocks until
it has come online (which will ensure a DHCP request has been completed).
After the system network is online, cloud-init net mode runs it will
refetch metadata due to the on-disk marker that the instance was a
reprovision VM.

If I've somehow misunderstood things, please let me know, but as I
understand things now; this scenario does not need cloud-init itself to
watch
netlink layer to reissue dhcp.


>
> *From:* Ryan Harper [mailto:ryan.harper@xxxxxxxxxxxxx]
> *Sent:* Thursday, February 8, 2018 5:51 PM
> *To:* Sushant Sharma (AZURE) <Sushant.Sharma@xxxxxxxxxxxxx>
> *Cc:* cloud-init@xxxxxxxxxxxxxxxxxxx; Tamilmani Manoharan <
> tamanoha@xxxxxxxxxxxxx>; Nisheeth Srivastava <
> Nisheeth.Srivastava@xxxxxxxxxxxxx>; Douglas Jordan <
> Douglas.Jordan@xxxxxxxxxxxxx>
>
> *Subject:* Re: [Cloud-init] Azure Networking Support in cloud-init
>
>
>
>
>
>
>
> On Thu, Feb 8, 2018 at 7:07 PM, Sushant Sharma (AZURE) <
> Sushant.Sharma@xxxxxxxxxxxxx> wrote:
>
> Hi Ryan,
>
> Thank you for your email. We just updated the PR with some more changes.
>
> To answer your specific questions
>
>
>
>    1. The specific scenario addressed here is to start the VM and have it
>    running, and once customer asks for a new VM in Azure, move the VM into
>    customer’s network and apply all customer specific configurations in the
>    VM. This is why the earlier PR by Douglas blocks cloud-init in Azure unless
>    we learn that VM is ready to be moved into customer’s Network. This
>    cloud-init block happens even before systemd-networkd starts.
>
>
>
> In this scenario which is before "networking" there's no need to "bounce"
> networking, rather once the VM is "released", it will issue a DHCP at
> network configuration time.  This works on Xenial (ifupdown based) as well
> as Artful and newer (systemd-networkd).
>
>
>
>
>    1.
>    2. The goal of this PR is to learn whenever the switch to customer’s
>    network has happened, and issue a dchp-request upon this event to learn new
>    IP address. The IP address may or may not change. Can you help us
>    understand why removing and adding nic may be needed in this scenario?
>
> Right now, on the linux side, I'm not aware of any distro which contains a
> networking configuration daemon which watches for carrier/no-carrier
> changes and subsequently issues a DHCP release and DHCP renew.  The
> exceptions to this are systemd-networkd behavior; though it does depend on
> how long
>
> the link goes away for w.r.t whether it would issue a new DHCP lease.  The
> bigger point here is that you're interested in *telling* the instance that
> it should reconfigure it's networking.  You're suggestion is to utilize the
> loss of carrier on the nic as an indication that it should and while that
> may be what needs to
>
> be done specifically on Azure, it's not a general case mechanism, and only
> works if using DHCP.
>
>
>
> As a stepping stone toward a real cloud to instance communication
> mechanism, hotplug can work more genericially on Linux as an indication
> that the instance network configuration needs to change (and possibly
> restarted) which could result in a DHCP release and renew.
>
>
>
>
>    1.
>
>
>
> Regarding the newer scenarios, I think it is great that you have shared
> the document where we can add the description and discuss solutions. Let’s
> do that separately in parallel to this PR.
>
> At the moment, we will appreciate if you and others can take a look at
> this PR and provide feedback so that it can be accepted.
>
>
>
> For the previous proposed which included spawning a python process to
> watch the netlink socket; I'm not comfortable with such an approach, for
> many of the reasons Robert already indicated in his
>
> review of the initial PR. I'll follow-up directly in the PR with more
> specific concerns.
>
>
>
> Ryan
>
>
>
>
>
> Thanks,
>
> Sushant
>
>
>
> *From:* Ryan Harper [mailto:ryan.harper@xxxxxxxxxxxxx]
> *Sent:* Friday, January 26, 2018 8:16 AM
> *To:* Sushant Sharma (AZURE) <Sushant.Sharma@xxxxxxxxxxxxx>
> *Cc:* cloud-init@xxxxxxxxxxxxxxxxxxx; Tamilmani Manoharan <
> tamanoha@xxxxxxxxxxxxx>; Nisheeth Srivastava <
> Nisheeth.Srivastava@xxxxxxxxxxxxx>
> *Subject:* Re: [Cloud-init] Azure Networking Support in cloud-init
>
>
>
>
>
>
>
> On Tue, Jan 9, 2018 at 5:31 PM, Sushant Sharma (AZURE) <
> Sushant.Sharma@xxxxxxxxxxxxx> wrote:
>
> Hi cloud-init members,
>
>
>
> We would like to discuss with you our proposal to add a network module in
> cloud-init to support various networking scenarios in Azure.
>
> To begin with, we would like to support move of a virtual machine (VM)
> from one network to another in Azure.
>
> As such, it will listen for media disconnect/connect (via netlink) and
> issues a re-DHCP when required (this design is based on how azure moves VM
> from one network to another).
>
>
>
> Thanks for starting the discussion here.  For this use-case, are you
> migrating
>
> the entire VM or are we changing an existing nic from one subnet to
> another?
>
> Since the link goes down (stopping traffic), is it possible to remove the
> nic
>
> and re-add it instead?
>
>
>
> Operating system behavior around link state change varies depending on the
>
> network service managing things.  In Ubuntu where ifupdown and
> isc-dhcp-client
>
> are utilized, as you know, netlink changes are not handled.  Under Ubuntu
> Artful
>
> and Bionic which utilize systemd-networkd, link state changes are watched;
> if
>
> the device loses carrier then when it is restored networkd will reacquire
> a lease in that
>
> case.
>
>
>
> Over time, we plan to support more advance networking scenarios in Azure.
> Please let us know your thoughts before we work on adding the module.
>
>
>
> I'm very much interested in enumerating additional scenarios.  Some
> user-stories
>
> which I think need to be address:
>
>
>
> 1. add additional network device and configure
>
> 2. remove network device (and update configuration)
>
> 3. add additional ip addresses to one or more network devices
>
> 4. remove ip address from one or more network devices
>
> 5. modify the configuration of an existing network device (changes outside
>
>    of 3 and 4)
>
>
>
> Cases 1 and 2 are generally covered by a udev hook handler.  3 and 4 can
> be
>
> partially addressed by updating cloud-init to read network config metadata
> and
>
> renderer a complete network configuration and may be combined with 1 and 2.
>
>
>
> What's not easily covered by a udev hook is the case where users modify
> existing
>
> network configuration without adding or removing devices.  To handle this
> sort
>
> of scenario a cloud will need to provide some notification mechanism to
> which
>
> cloud-init can react.  This may be something simple like a websocket
> cloud-init
>
> can select() on, or some other hypervisor event injection.  This area is
> not
>
> well defined and will certainly vary from provider to provider which will
>
> require some time to form a general solution.
>
>
>
> I'd like to continue the discussion in a shared document:
>
>
>
> https://hackmd.io/MzCsBYBMEMHYCMC0AOS4BmjwAYCM3F54A2JGZAY3gC
> YBTaSYa2IA?both
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhackmd.io%2FMzCsBYBMEMHYCMC0AOS4BmjwAYCM3F54A2JGZAY3gCYBTaSYa2IA%3Fboth&data=02%7C01%7CSushant.Sharma%40microsoft.com%7C17acfcd577e8446cde6d08d564d80c0f%7Cee3303d7fb734b0c8589bcd847f1c277%7C1%7C1%7C636525801436863680&sdata=AVN8VPHOOLds01uguh5GNklo4pBz%2FpXABX44cLe8OLI%3D&reserved=0>
>
>
>
>
>
>
>
> Thanks,
>
> Sushant
>
>
>
>
> --
> Mailing list: https://launchpad.net/~cloud-init
> <https://na01.safelinks.protection.outlook.com/?url=https:%2F%2Flaunchpad.net%2F~cloud-init&data=02%7C01%7CSushant.Sharma%40microsoft.com%7C17acfcd577e8446cde6d08d564d80c0f%7Cee3303d7fb734b0c8589bcd847f1c277%7C1%7C1%7C636525801436863680&sdata=0Ieqm%2FzkdIdB5Gz5mu3C0xN5m7nSVUDyVhBaSmDZVN4%3D&reserved=0>
> Post to     : cloud-init@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~cloud-init
> <https://na01.safelinks.protection.outlook.com/?url=https:%2F%2Flaunchpad.net%2F~cloud-init&data=02%7C01%7CSushant.Sharma%40microsoft.com%7C17acfcd577e8446cde6d08d564d80c0f%7Cee3303d7fb734b0c8589bcd847f1c277%7C1%7C1%7C636525801436863680&sdata=0Ieqm%2FzkdIdB5Gz5mu3C0xN5m7nSVUDyVhBaSmDZVN4%3D&reserved=0>
> More help   : https://help.launchpad.net/ListHelp
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhelp.launchpad.net%2FListHelp&data=02%7C01%7CSushant.Sharma%40microsoft.com%7C17acfcd577e8446cde6d08d564d80c0f%7Cee3303d7fb734b0c8589bcd847f1c277%7C1%7C1%7C636525801436863680&sdata=DyQQKKEVimW0VwODAnV731mjOZ50VX7oEH5piU12qb8%3D&reserved=0>
>
>
>
>
>