fuel-dev team mailing list archive

[Matthew Mosesohn <mmosesohn@xxxxxxxxxxxx>]

Andrey,

This is something a user could always fix himself. Typically you do
DHCP on eth0, then eth1,etc.... and the only case I can think of where
your scenario would occur is if we were trying to provision 50+ nodes
with an underpowered master. But what's more likely is eth0 will start
PXE boot and fail to receive the whole image and reboot and try again.
If you have systems PXE booting on eth0 for bootstrap then eth1 for
provision, then it's a really odd machine and I think we need to
discount that case.

If this scenario is real and reproducable, we can definitely work
around it and create system records in Cobbler (but not reserve an IP
for them) that correspond to all the other MAC addresses of the
system. We could forward them to a special profile that tells cobbler
to update its records for DNS and the system profile, then reboot it
and provision again. Or at minimum a warning state that we can use to
give feedback to the user that PXE is not working correctly on a
particular node.

If a user wants to deploy 100 nodes with 6 NICs in each system, a /24
should be enough, but with our non-optimal logic, it requires a /22
network to deploy: 100 IPs for bootstrap and 600 IPs for provisioning.

The solution in place doesn't work and doesn't meet expectations of
real use cases. We should pick an approach that will make the most
people happy.

-Matthew

On Mon, Jan 27, 2014 at 9:40 AM, Andrey Danin <adanin@xxxxxxxxxxxx> wrote:
> The problem is the Cobbler will force the node to boot to bootstrap if he
> doesn't recognize it via known MAC. Here are the boot process diagrams.
> The right one:
>   # node's embedded PXE software starts a boot-up procedure via eht0
>   # Cobbler knows eth0 MAC and puts to the node a PXE config to download and
> run an installation image (CentOS)
>   # The node goes to the provisioning state.
>
> The bad one:
>   # node runs PXE procedure over eth1
>   # Cobbler doesn't know eth1 MAC and sends to the node a PXE config with a
> bootstrap image.
>   # the node boots into bootstrap and an installation process fails.
>
>
> On Mon, Jan 27, 2014 at 9:25 AM, Matthew Mosesohn <mmosesohn@xxxxxxxxxxxx>
> wrote:
>>
>> Andrey,
>>
>> We can use the same tactic we use in Ubuntu to ensure that CentOS
>> starts installing on the correct MAC by adding the kernel parameter
>> ks.device=$MACADDR. It will try to DHCP on that interface and
>> eliminate the possibility of having 2 or more NICs on the same subnet
>> as the PXE server.
>>
>> On Mon, Jan 27, 2014 at 9:14 AM, Andrey Danin <adanin@xxxxxxxxxxxx> wrote:
>> > There is a little chance that a node will boot up via another interface
>> > and
>> > Cobbler will put it to the bootstrap stage instead of the target stage.
>> > We
>> > have to provide all MAC addresses of a node to Cobbler but it can't use
>> > one
>> > IP for all MACs. So we have to allocate a lot of IPs.
>> >
>> > We can improve our mechanism by the following:
>> >   * Assign IP to the node's admin interface only. Use it as a default
>> > behavior but add an UI switch to enable a greedy behavior.
>> >   * Run a predeployment DHCP check in order to find all interfaces which
>> > really can communicate with PXE network and assign IPs to these
>> > interfaces
>> > only.
>> >
>> >
>> >
>> > On Thu, Jan 23, 2014 at 12:29 AM, Ryan Moe <rmoe@xxxxxxxxxxxx> wrote:
>> >>
>> >> What is the reasoning behind assigning an IP from the admin network to
>> >> all
>> >> interfaces [1]? It has caused problems in deployments where IPs are
>> >> tightly
>> >> managed and/or machines have a large number of interfaces.
>> >>
>> >> Thanks,
>> >> Ryan
>> >>
>> >> [1]
>> >>
>> >> https://github.com/stackforge/fuel-web/blob/master/nailgun/nailgun/task/helpers.py#L435
>> >>
>> >> --
>> >> Mailing list: https://launchpad.net/~fuel-dev
>> >> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>> >> Unsubscribe : https://launchpad.net/~fuel-dev
>> >> More help   : https://help.launchpad.net/ListHelp
>> >>
>> >
>> >
>> >
>> > --
>> > Andrey Danin
>> > adanin@xxxxxxxxxxxx
>> > skype: gcon.monolake
>> >
>> > --
>> > Mailing list: https://launchpad.net/~fuel-dev
>> > Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>> > Unsubscribe : https://launchpad.net/~fuel-dev
>> > More help   : https://help.launchpad.net/ListHelp
>> >
>
>
>
>
> --
> Andrey Danin
> adanin@xxxxxxxxxxxx
> skype: gcon.monolake