yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #78390
[Bug 1828611] [NEW] disk_setup could deal with non-deterministic device naming on EC2 nitro instances
Public bug reported:
Cloud Provider: Amazon Web Services
As is documented in [1], instances on the nitro type hypervisor don't
attach the NVME disks in deterministic order, yielding a different order
of disks, example [2]. This makes it somewhat difficult to format and
partition volumes since you don't know the volume ids beforehand when
creating an instance (in an Autoscaling group for example).
My current thinking is that maybe a sort of special device name (much
like swap / ephemeralX) coul be used to locate a device, for example
ebs:root for the root drive - which is easy to detect, ebs:size=12G[0]
for the first volume found with 12GiB size. With an appropriate instance
profile and boto3 more elaborate selectors are conceivable (for example
based on tags).
Further complicating things is that the metadata endpoint doesn't expose
the correct device names, opting instead for fantasy names (sda1 for the
root volume, sdX for other volumes).
My Workaround for a 2 volume instance: Try and format both devices, then mount the disk by label:
#cloud-config
fs_setup:
- label: mylabel
device: /dev/nvme0n1
filesystem: xfs
partition: none
overwrite: false
- label: mylabel
device: /dev/nvme1n1
filesystem: xfs
partition: none
overwrite: false
mounts:
- ["/dev/disk/by-label/mylabel", "/mnt/label", "xfs", "defaults"]
[1] https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/nvme-ebs-volumes.html
[2] https://gist.github.com/nilsmeyer/eddcfa4b7fc5b04ebc0be9eaa3c7b7dd
** Affects: cloud-init
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1828611
Title:
disk_setup could deal with non-deterministic device naming on EC2
nitro instances
Status in cloud-init:
New
Bug description:
Cloud Provider: Amazon Web Services
As is documented in [1], instances on the nitro type hypervisor don't
attach the NVME disks in deterministic order, yielding a different
order of disks, example [2]. This makes it somewhat difficult to
format and partition volumes since you don't know the volume ids
beforehand when creating an instance (in an Autoscaling group for
example).
My current thinking is that maybe a sort of special device name (much
like swap / ephemeralX) coul be used to locate a device, for example
ebs:root for the root drive - which is easy to detect, ebs:size=12G[0]
for the first volume found with 12GiB size. With an appropriate
instance profile and boto3 more elaborate selectors are conceivable
(for example based on tags).
Further complicating things is that the metadata endpoint doesn't
expose the correct device names, opting instead for fantasy names
(sda1 for the root volume, sdX for other volumes).
My Workaround for a 2 volume instance: Try and format both devices, then mount the disk by label:
#cloud-config
fs_setup:
- label: mylabel
device: /dev/nvme0n1
filesystem: xfs
partition: none
overwrite: false
- label: mylabel
device: /dev/nvme1n1
filesystem: xfs
partition: none
overwrite: false
mounts:
- ["/dev/disk/by-label/mylabel", "/mnt/label", "xfs", "defaults"]
[1] https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/nvme-ebs-volumes.html
[2] https://gist.github.com/nilsmeyer/eddcfa4b7fc5b04ebc0be9eaa3c7b7dd
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1828611/+subscriptions
Follow ups