ubuntu-phone team mailing list archive

[Thomas Voß <thomas.voss@xxxxxxxxxxxxx>]

On Fri, Oct 2, 2015 at 9:20 AM, Benjamin Zeller
<benjamin.zeller@xxxxxxxxxxxxx> wrote:
> Hey all,
>
> I also put my thoughts inline:
>
>> Hey Simon,
>>
>> Thanks for your thoughts and ideas. Please find my suggestions and
>> replies inline:
>>
>> On Thu, Oct 1, 2015 at 1:16 PM, Sturm Flut <sturmflut@xxxxxxxxxxxxxx>
>> wrote:
>>>
>>> Good morning dear list,
>>>
>>> this has been brewing for quite some time now and the discussion
>>> Krzysztof Tataradziński started a week ago didn't lead anywhere again,
>>> so I'm starting it yet another time:
>>>
>>> In my opinion the "no background processing for apps on the phone"
>>> design decision is wrong, is already hurting us too much and has to be
>>> revoked as soon as possible.
>>>
>> To state this very clearly: We are looking at a continuum of potential
>> solutions here.
>> It is unfortunately not as simple as just revoking one solution and
>> picking another one, but
>> all the different objectives have to be balanced. To this end, let me
>> clarify some of the guiding
>> principles and objectives that have to be accounted for in the context
>> of lifecycle policies:
>>
>> (1.) Battery life: One of the scarcest resources on a mobile device
>> and something that we aim
>> to protect as much as possible. Allowing arbitrary background
>> processing hurts a lot here, with Android's
>> lifecycle approach being one of the examples where battery life
>> deteriorates and becomes unpredictable
>> depending on the apps that have been installed on the device. Now this
>> wouldn't be a problem for the
>> quite technical audience subscribed to this mailing list. But it is
>> certainly an issue for the average user who
>> should not be forced to maintain a list of running apps and processes
>> just to achieve sensible battery life.
>
> Agreed, having a solution that protects battery life is important, however
> that does not mean we can't have a way for the advanced user to allow
> _some_ processes to run in background. Probably something that can
> be turned on in the system settings per application. Giving full control
> over
> what can run in background.
>
> With convergence in mind, a user probably will need a way to allow his
> terminal
> to run in background while it compiles something, lets also make sure
> we can give solutions to a more technical audience. After all this is the
> audience
> that uses Ubuntu Phone atm!

Sure, I'm not opposed to having dev-mode settings that can be
leveraged by tech-savvy users. However, as saviq pointed out,
we have to avoid apps not working as expected if the average user does
not opt-in. On top, if we just allow settings to be tweaked
there is no reason for app authors to pick up our platform principles
and guidelines. The next step is a wave of apps in the store that
only work if you switch on certain settings on the device. That's a
worst case scenario from my POV.

>>
>> (2.) Security & privacy: One of the worst case examples are apps
>> running in the background, feeding on a device's
>> sensor data and sending it off to the cloud. There have been numerous
>> examples exploiting the issue on Android.
>> Again, the technical audience subscribed here would be fine, the
>> average user likely is not.
>
> But won't this be possible with a system service and helpers as well?
> Lets say we have a system service that lets a helper collect GPS data,
> won't the helper be able to upload that data?
> Because some apps might even need this, lets look at the famous sports
> tracker
> example. It needs to collect your gps positions and upload them.

Well, that's debatable. We usually go for a very strict setting (e.g.
in scopes) that prevents network access if access to local
resources is granted. You only get network access if the only local
resources you access are the ones specific to your app.

We surely cannot solve the problem of spying applications once and for
all. But we can work hard to make sure that a user
stays on top of what apps _and_ the system is doing, putting her in a
position of control and power.

>>
>> (3.) Predictable performance: A phone is expected to perform whenever
>> it is needed to, and the application being front & center
>> to the user should provide a smooth and crisp experience. To this end,
>> the system has to be able to assign as much performance-relevant
>> resources as possible to the currently focused app. If arbitrary
>> background processing is allowed, the respective scheduling problem
>> becomes
>> very very hard to solve (if possible at all).
>
> Couldn't that be solved by giving the background processes time slots
> when they can execute? And probably an API to register system resources
> that can wake them up for a time frame? Lets say a socket.

Sure, moving processes to specific cgroups if they are not visible to
the user is possible and something we have on the list
to explore. However, we are talking about heuristics, not a solution
that will always work regardless of the installed apps.

>>
>> Please note that there are way more aspects to this problem space than
>> just the three points I elaborated on before.
>> The key point is: For a technical or tech-savvy audience, the
>> lifecycle approach we are taking for the phone might seem overly
>> restricted.
>> However, for the average user, such a harsh policy comes with a lot of
>> benefits.
>
> I still agree for the average user will benefit from that, but until we
> reach
> that point of completeness we just lack critical APIs for apps, and devs
> might not even start to write their apps or contact us to tell: "Hey I'm
> missing
> this or that, could you implement it please".
>

Well, I would argue that we already have a lot of feedback that we
have to systematically work on.
We either go down the route of systematically iterating on the system
capabilities or we open the flood gates and
just give up on our principles. Point being that taking features back
is almost impossible

> Another drawback of doing things completely different is of course the
> higher
> cost in porting applications to our platform. There is no way to port a Qt
> app
> that requires background processing to Ubuntu Phone without rewriting big
> parts
> of it to use our system services and helpers. So the porting is no longer
> effortless
> thus appdevs will wait for a critical mass of phone users to appear but the
> phone
> users will only appear when there is a critical mass of apps.... this
> problem again.
>

I'm all in for making porting as efficient as possible. That does not
imply, though, to give up
on our principles. In addition, our platform behaves and operates
similar to other major mobile OS's, with the
notable difference that we are working hard to ensure that "good
practice" is actually adhered to :)

I think we should avoid having battery optimizers and similar system
maintenance tools in the store.

>>> Why? Let me make a list of examples.
>>>
>>> - Open Telegram for Ubuntu, send a picture to a friend, switch to
>>> another app or scope. On any other platform the message would continue
>>> to be sent in the background, but not on Ubuntu. Now you might say
>>> "Telegram is going to become a Telepathy plugin, and that one can run in
>>> the background", but there is no sign of a Telepathy system service and
>>> even if it were already present, you're now forcing everybody to build
>>> Telepathy plugins for their services. That's just not going to happen.
>>> Let's be honest, WhatsApp and Viber and friends will not go this way for
>>> a number of very good (economic) reasons.
>>>
>>> - Notifications for the myriads of services out there. On any other
>>> platform the app would just register a small background service that
>>> wakes up every minute or so and then goes to sleep again. Now you might
>>> say "We have the Ubuntu Push Notifications service for that", but that
>>> requires the service provider to change his whole server side, which
>>> noone except Telegram does. That's why we already have to run
>>> account-polld.
>>>
>> That's actually not true in the general case. Most often, the services
>> use the platform's push notification
>> infrastructure to deliver updates. Please note that we require
>> account-polld to work around the limitation that
>> facebook, twitter and google have not integrated with our infrastructure
>> (yet).
>>
>>> - An app that processes location updates in the background, e.g. a
>>> fitness tracker or a navigation app. Now this is IMO the best example
>>> for how the "no background processing" decision complicates everything
>>> to infinity: The simple solution would be to have a small background
>>> service that just does whatever it has to do. But since we can't do
>>> that, we have to come up with a very convoluted system that most likely
>>> involves registering for some list of events with location-service,
>>> which then calls a small handler binary provided by the app (like the
>>> push notifications design). But how do you prevent that handler from
>>> running amok, or from just keep running in the background? Ah, yeah,
>>> kill it after five seconds, like the push notification handlers. But
>>> five seconds on a slow CPU aren't much when I e.g. also have to do some
>>> I/O and use D-Bus, while five seconds on a fast CPU are a lot of time if
>>> I don't have to do much. So what's the right maximum runtime for a
>>> handler that covers all use cases? There is none. Will location-service
>>> support all the event/filter/callback options I need for my specific
>>> app? Most likely not.
>>>
>> It will grant a certain amount of processing time to your app or
>> better its location update helper. A fitness tracker could easily just
>> store
>> the respective update for later processing.
>
> Even a fitness tracker might give you stats while you are still running,
> e.g. Endomondo tells you how fast you managed to run the last lap, for
> that is has to process the data while the screen is off and not just store
> it.
> Not sure if the small timeslot for a helper is enough for that.
>

Now that's certainly a solvable problem.

>> A navigation app would
>> require more sophisticated schemes, but we are happy to evolve the
>> service as required (as pointed out earlier).
>>
>>> - Cloud and P2P services like Owncloud, Dropbox, Syncthing, Tox etc.
>>> Apps that control devices via Bluetooth, like my FitBit or a Smartwatch.
>>> What's the design for those? More system services? We can't afford to
>>> write a system service for each and every use case. That doesn't scale.
>>> It already isn't scaling right now. And even if we could, is the
>>> proposed solution really to have 100 system services running for all
>>> users, instead of five to ten third party background processes that the
>>> user really needs and knowingly installed herself?
>>>
>> We would not write a system service for each of them, but instead
>> provide a framework to handle the specific problem area.
>> It is a lot of work to get those frameworks right, and your help is
>> obviously greatly appreciated in doing so. Other than that, it is
>> really really easy
>> to integrate cloud services with the content-hub infrastructure and
>> thus exposing the content in the system.
>>
>>> I can come up with many more examples for how this is simply
>>> complicating our lives, and for what reason? A (tiny) gain in battery
>>> life.
>>
>> Please see the initial list of objectives and guiding principles. On a
>> very personal note, I don't think the aim should be to make "our" life
>> easier but
>> to make the life of users as pleasant as possible. If that means to
>> come up with novel and probably harsh solutions and policies to
>> address certain
>> problems ... that's what it is.
>
> I agree that we should not make "our" life easier and aim for a good
> solution, however we need to be careful to not aim too high for a goal
> we can not reach in a suitable time.
>>
>>
>> We have many apps that cannot be built/ported right now because we
>>>
>>> are waiting for APIs and system services that haven't been coming for a
>>> year now and only have to be created because we can't just run in the
>>> background. Do we even know if the effect on battery life is really
>>> worth it? I challenge you to run e.g. Dekko as a lifecycle exception and
>>> check if the background polling is noticeable. Because it's not like
>>> those system services don't consume any resources at all, the work has
>>> to be done by somebody. A "bad" Telepathy plugin will consume just as
>>> much resources as a "bad" background process, the "better" design of
>>> using Telepathy alone is not going to prevent this.
>>>
>> They do consume resources, for sure. The point is: A fixed set of
>> system services controlled and maintained by us results in predictable
>> battery life (modulo bugs).
>> The real question is not if battery life would deteriorate for
>> allowing a specific app to poll in the background. Instead, the
>> question has to target the general behavior of
>> the system with arbitrary apps being installed in the system.
>
> So please correct me if I'm wrong, is the idea here that lets say Facebook
> provides
> his own closed telepathy plugin? If that is so, what's the difference from a
> "normal"
> background process? If that plugin behaves badly we can not do much about it
> right?
> At least not without probably breaking its functionality.
>

There are multiple benefits to it:

  (1.) Tight integration with our platform UI and UX.
  (2.) The ability to tightly control resources granted to the
respective backend.
  (3.) We will likely setup a more strict review procedure for those
extensions, together with domain-specific test-suites and potentially
audits.

In terms of handling extensions that behave poorly: With the context
of the problem domain (e.g., messaging) handling errors and confining
operation in general becomes a lot more tractable. The more specific
the use-case the more specific control mechanisms can be.

> Or is the idea here we will provide something more generic that will use app
> helpers
> like we do in other scenarios?
>>
>>
>>> What also really confuses me about this issue is that the "no background
>>> processing" approach apparently only applies to mobile devices and
>>> confined apps. Isn't that somehow against the whole idea of an Ubuntu
>>> that is the same on all devices?
>>>
>> It certainly is not. A lifecycle policy is always specific to a
>> device, its resources and even the specific usage scenario.
>> With that, allowing multiple apps to run even if not focused in
>> desktop-like use-cases is perfectly fine. Even altering the behavior
>> in case of a phone being plugged in to a power supply would be perfectly
>> fine.
>
> +1, but it would be nice if the user could override that in some way. Right
> now I'm also
> able to override powersaving on my laptop if I really want to waste battery
> life.
>>
>>
>>> I would really like to hear all your thoughts on this and how we're
>>> going to solve the situation, because we quite frankly have actual
>>> problems (broken GPS, no SD card support for apps, no Bluetooth, no SIM
>>> Toolkit, etc.) which should have a much higher priority than building
>>> convoluted and complex designs just to potentially save three hours of
>>> battery life over the course of a week.
>>>
>> As pointed out in my introductory comment: battery life is one of
>> multiple concerns we have to address.
>>
>> Cheers,
>>
>>    Thomas
>
> Just want to point out, I'm aware this is no easy decision and with whatever
> we decide we have to stay with or break apps later. So there is no temporary
> solution we
> can deprecate later without breaking apps, which is IMHO not an option.
>

+1, whatever we come up with has to be aligned with what we want to
support in the future.

Cheers,

  Thomas

> Cheers,
>
> Benjamin