kernel-packages team mailing list archive

[Abhishek Gupta <abgupta@xxxxxxxxxxxxx>]

Hi guys,

We went the alternate route of trying to convince kernel folks of fixing
this problem through a patch. Unfortunately, the patch was rejected. The
discussion involved is quoted below. Please also note that in the above
patch we can remove the CPU hotadd request command as that is
extraneous:

# CPU hotadd request
 SUBSYSTEM=="cpu", ACTION=="add", DEVPATH=="/devices/system/cpu/cpu*[0-9]", TEST=="online", ATTR{online}="1"

Please include the memory hotadd patch as not including it may lead to
kernel panics related to memory management.

Thanks,
Abhishek

-----Original Message-----
From: Dave Hansen [mailto:dave@xxxxxxxx]
Sent: Thursday, July 25, 2013 9:35 AM
To: Kay Sievers
Cc: KY Srinivasan; Michal Hocko; gregkh@xxxxxxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx; devel@xxxxxxxxxxxxxxxxxxxxxx; olaf@xxxxxxxxx; apw@xxxxxxxxxxxxx; andi@xxxxxxxxxxxxxx; akpm@xxxxxxxxxxxxxxxxxxxx; linux-mm@xxxxxxxxx; kamezawa.hiroyuki@xxxxxxxxx; hannes@xxxxxxxxxxx; yinghan@xxxxxxxxxx; jasowang@xxxxxxxxxx
Subject: Re: [PATCH 1/1] Drivers: base: memory: Export symbols for onlining memory blocks

On 07/25/2013 08:15 AM, Kay Sievers wrote:
> Complexity, well, it's just a bit of code which belongs in the kernel.
> The mentioned unconditional hotplug loop through userspace is 
> absolutely pointless. Such defaults never belong in userspace tools if 
> they do not involve data that is only available in userspace and 
> something would make a decision about that. Saying "hello" to 
> userspace and usrspace has a hardcoded "yes" in return makes no sense 
> at all. The kernel can just go ahead and do its job, like it does for 
> all other devices it finds too.

Sorry, but memory is different than all other devices.  You never need a
mouse in order to add another mouse to the kernel.

I'll repaste something I said earlier in this thread:

> A system under memory pressure is going to have troubles doing a 
> hot-add.  You need memory to add memory.  Of the two operations ("add"
> and "online"), "add" is the one vastly more likely to fail.  It has to 
> allocate several large swaths of contiguous physical memory.  For that 
> reason, the system was designed so that you could "add" and "online"
> separately.  The intention was that you could "add" far in advance and 
> then "online" under memory pressure, with the "online" having *VASTLY* 
> smaller memory requirements and being much more likely to succeed.

So, no, it makes no sense to just have userspace always unconditionally
online all the memory that the kernel adds.  But, the way it's set up,
we _have_ a method that can work under lots memory pressure, and it is
available for users that want it.  It was designed 10 years ago, and
maybe it's outdated, or history has proved that nobody is going to use
it the way it was designed.

If I had it to do over again, I'd probably set up configurable per-node
sets of spare kernel metadata.  That way, you could say "make sure we
have enough memory reserved to add $FOO sections to node $BAR".  Use
that for the largest allocations, then depend on PF_MEMALLOC to get us
enough for the other little bits along the way.

Also, if this is a problem, it's going to be a problem for *EVERY* user
of memory hotplug, not just hyperv.  So, let's see it fixed generically
for *EVERY* user.  Something along the lines of:

1. Come up with an interface that specifies a default policy for
   newly-added memory sections.  Currently, added memory gets "added",
   but not "onlined", and the default should stay doing that.
2. Make sure that we at least WARN_ONCE() if someone tries to online an
   already-kernel-onlined memory section.  That way, if someone trips
   over this new policy, we have a _chance_ of explaining to them what
   is going on.
3. Think about what we do in the failure case where we are able to
   "add", but fail to "online" in the kernel.  Do we tear the
   newly-added structures down and back out of the operation, or do
   we leave the memory added, but offline (what happens in the normal
   case now)?

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1233466

Title:
  Hot-Add Memory failing for lack of udev rule

Status in “linux” package in Ubuntu:
  Fix Released
Status in “systemd” package in Ubuntu:
  In Progress

Bug description:
  It appears that the Hot-Add mechanism is not working because of lack
  of a udev rule. The required rule is attached to this bug. Please
  include it.

  Furthermore, please ensure that the following patches are included:

  https://git.kernel.org/cgit/linux/kernel/git/next/linux-
  next.git/commit/drivers/hv?id=20138d6cb838aa01bb1b382dcb5f3d3a119ff2cb

  https://git.kernel.org/cgit/linux/kernel/git/next/linux-
  next.git/commit/drivers/hv?id=c5e2254f8d63a6654149aa32ac5f2b7dd66a976d

  https://git.kernel.org/cgit/linux/kernel/git/next/linux-
  next.git/commit/drivers/hv?id=ed07ec93e83ec471d365ce084e43ad90fd205903

  The above patches should be in but just wanted to ensure that you guys
  got these.

  Thanks again for the great last minute help.
  Abhishek

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1233466/+subscriptions