← Back to team overview

ubuntu-phone team mailing list archive

Re: The problem with "no background processing for apps"

 

Hey all,

I also put my thoughts inline:
Hey Simon,

Thanks for your thoughts and ideas. Please find my suggestions and
replies inline:

On Thu, Oct 1, 2015 at 1:16 PM, Sturm Flut <sturmflut@xxxxxxxxxxxxxx> wrote:
Good morning dear list,

this has been brewing for quite some time now and the discussion
Krzysztof Tataradziński started a week ago didn't lead anywhere again,
so I'm starting it yet another time:

In my opinion the "no background processing for apps on the phone"
design decision is wrong, is already hurting us too much and has to be
revoked as soon as possible.

To state this very clearly: We are looking at a continuum of potential
solutions here.
It is unfortunately not as simple as just revoking one solution and
picking another one, but
all the different objectives have to be balanced. To this end, let me
clarify some of the guiding
principles and objectives that have to be accounted for in the context
of lifecycle policies:

(1.) Battery life: One of the scarcest resources on a mobile device
and something that we aim
to protect as much as possible. Allowing arbitrary background
processing hurts a lot here, with Android's
lifecycle approach being one of the examples where battery life
deteriorates and becomes unpredictable
depending on the apps that have been installed on the device. Now this
wouldn't be a problem for the
quite technical audience subscribed to this mailing list. But it is
certainly an issue for the average user who
should not be forced to maintain a list of running apps and processes
just to achieve sensible battery life.
Agreed, having a solution that protects battery life is important, however
that does not mean we can't have a way for the advanced user to allow
_some_ processes to run in background. Probably something that can
be turned on in the system settings per application. Giving full control over
what can run in background.

With convergence in mind, a user probably will need a way to allow his terminal
to run in background while it compiles something, lets also make sure
we can give solutions to a more technical audience. After all this is the audience
that uses Ubuntu Phone atm!
(2.) Security & privacy: One of the worst case examples are apps
running in the background, feeding on a device's
sensor data and sending it off to the cloud. There have been numerous
examples exploiting the issue on Android.
Again, the technical audience subscribed here would be fine, the
average user likely is not.
But won't this be possible with a system service and helpers as well?
Lets say we have a system service that lets a helper collect GPS data,
won't the helper be able to upload that data?
Because some apps might even need this, lets look at the famous sports tracker
example. It needs to collect your gps positions and upload them.
(3.) Predictable performance: A phone is expected to perform whenever
it is needed to, and the application being front & center
to the user should provide a smooth and crisp experience. To this end,
the system has to be able to assign as much performance-relevant
resources as possible to the currently focused app. If arbitrary
background processing is allowed, the respective scheduling problem
becomes
very very hard to solve (if possible at all).
Couldn't that be solved by giving the background processes time slots
when they can execute? And probably an API to register system resources
that can wake them up for a time frame? Lets say a socket.
Please note that there are way more aspects to this problem space than
just the three points I elaborated on before.
The key point is: For a technical or tech-savvy audience, the
lifecycle approach we are taking for the phone might seem overly
restricted.
However, for the average user, such a harsh policy comes with a lot of benefits.
I still agree for the average user will benefit from that, but until we reach
that point of completeness we just lack critical APIs for apps, and devs
might not even start to write their apps or contact us to tell: "Hey I'm missing
this or that, could you implement it please".

Another drawback of doing things completely different is of course the higher cost in porting applications to our platform. There is no way to port a Qt app that requires background processing to Ubuntu Phone without rewriting big parts of it to use our system services and helpers. So the porting is no longer effortless thus appdevs will wait for a critical mass of phone users to appear but the phone users will only appear when there is a critical mass of apps.... this problem again.
Why? Let me make a list of examples.

- Open Telegram for Ubuntu, send a picture to a friend, switch to
another app or scope. On any other platform the message would continue
to be sent in the background, but not on Ubuntu. Now you might say
"Telegram is going to become a Telepathy plugin, and that one can run in
the background", but there is no sign of a Telepathy system service and
even if it were already present, you're now forcing everybody to build
Telepathy plugins for their services. That's just not going to happen.
Let's be honest, WhatsApp and Viber and friends will not go this way for
a number of very good (economic) reasons.

- Notifications for the myriads of services out there. On any other
platform the app would just register a small background service that
wakes up every minute or so and then goes to sleep again. Now you might
say "We have the Ubuntu Push Notifications service for that", but that
requires the service provider to change his whole server side, which
noone except Telegram does. That's why we already have to run account-polld.

That's actually not true in the general case. Most often, the services
use the platform's push notification
infrastructure to deliver updates. Please note that we require
account-polld to work around the limitation that
facebook, twitter and google have not integrated with our infrastructure (yet).

- An app that processes location updates in the background, e.g. a
fitness tracker or a navigation app. Now this is IMO the best example
for how the "no background processing" decision complicates everything
to infinity: The simple solution would be to have a small background
service that just does whatever it has to do. But since we can't do
that, we have to come up with a very convoluted system that most likely
involves registering for some list of events with location-service,
which then calls a small handler binary provided by the app (like the
push notifications design). But how do you prevent that handler from
running amok, or from just keep running in the background? Ah, yeah,
kill it after five seconds, like the push notification handlers. But
five seconds on a slow CPU aren't much when I e.g. also have to do some
I/O and use D-Bus, while five seconds on a fast CPU are a lot of time if
I don't have to do much. So what's the right maximum runtime for a
handler that covers all use cases? There is none. Will location-service
support all the event/filter/callback options I need for my specific
app? Most likely not.

It will grant a certain amount of processing time to your app or
better its location update helper. A fitness tracker could easily just
store
the respective update for later processing.
Even a fitness tracker might give you stats while you are still running,
e.g. Endomondo tells you how fast you managed to run the last lap, for
that is has to process the data while the screen is off and not just store it.
Not sure if the small timeslot for a helper is enough for that.
A navigation app would
require more sophisticated schemes, but we are happy to evolve the
service as required (as pointed out earlier).

- Cloud and P2P services like Owncloud, Dropbox, Syncthing, Tox etc.
Apps that control devices via Bluetooth, like my FitBit or a Smartwatch.
What's the design for those? More system services? We can't afford to
write a system service for each and every use case. That doesn't scale.
It already isn't scaling right now. And even if we could, is the
proposed solution really to have 100 system services running for all
users, instead of five to ten third party background processes that the
user really needs and knowingly installed herself?

We would not write a system service for each of them, but instead
provide a framework to handle the specific problem area.
It is a lot of work to get those frameworks right, and your help is
obviously greatly appreciated in doing so. Other than that, it is
really really easy
to integrate cloud services with the content-hub infrastructure and
thus exposing the content in the system.

I can come up with many more examples for how this is simply
complicating our lives, and for what reason? A (tiny) gain in battery
life.
Please see the initial list of objectives and guiding principles. On a
very personal note, I don't think the aim should be to make "our" life
easier but
to make the life of users as pleasant as possible. If that means to
come up with novel and probably harsh solutions and policies to
address certain
problems ... that's what it is.
I agree that we should not make "our" life easier and aim for a good
solution, however we need to be careful to not aim too high for a goal
we can not reach in a suitable time.

We have many apps that cannot be built/ported right now because we
are waiting for APIs and system services that haven't been coming for a
year now and only have to be created because we can't just run in the
background. Do we even know if the effect on battery life is really
worth it? I challenge you to run e.g. Dekko as a lifecycle exception and
check if the background polling is noticeable. Because it's not like
those system services don't consume any resources at all, the work has
to be done by somebody. A "bad" Telepathy plugin will consume just as
much resources as a "bad" background process, the "better" design of
using Telepathy alone is not going to prevent this.

They do consume resources, for sure. The point is: A fixed set of
system services controlled and maintained by us results in predictable
battery life (modulo bugs).
The real question is not if battery life would deteriorate for
allowing a specific app to poll in the background. Instead, the
question has to target the general behavior of
the system with arbitrary apps being installed in the system.
So please correct me if I'm wrong, is the idea here that lets say Facebook provides his own closed telepathy plugin? If that is so, what's the difference from a "normal" background process? If that plugin behaves badly we can not do much about it right?
At least not without probably breaking its functionality.

Or is the idea here we will provide something more generic that will use app helpers
like we do in other scenarios?

What also really confuses me about this issue is that the "no background
processing" approach apparently only applies to mobile devices and
confined apps. Isn't that somehow against the whole idea of an Ubuntu
that is the same on all devices?

It certainly is not. A lifecycle policy is always specific to a
device, its resources and even the specific usage scenario.
With that, allowing multiple apps to run even if not focused in
desktop-like use-cases is perfectly fine. Even altering the behavior
in case of a phone being plugged in to a power supply would be perfectly fine.
+1, but it would be nice if the user could override that in some way. Right now I'm also able to override powersaving on my laptop if I really want to waste battery life.

I would really like to hear all your thoughts on this and how we're
going to solve the situation, because we quite frankly have actual
problems (broken GPS, no SD card support for apps, no Bluetooth, no SIM
Toolkit, etc.) which should have a much higher priority than building
convoluted and complex designs just to potentially save three hours of
battery life over the course of a week.

As pointed out in my introductory comment: battery life is one of
multiple concerns we have to address.

Cheers,

   Thomas
Just want to point out, I'm aware this is no easy decision and with whatever
we decide we have to stay with or break apps later. So there is no temporary solution we
can deprecate later without breaking apps, which is IMHO not an option.

Cheers,

Benjamin


Follow ups

References