← Back to team overview

ubuntu-phone team mailing list archive

Re: The problem with "no background processing for apps"

 


On 02.10.2015 16:22, Chris Wayne wrote:
> 
> 
> On Fri, Oct 2, 2015 at 4:53 AM, Andrea Bernabei
> <andrea.bernabei@xxxxxxxxxxxxx <mailto:andrea.bernabei@xxxxxxxxxxxxx>>
> wrote:
> 
> 
> 
>     On Fri, Oct 2, 2015 at 9:18 AM, Thomas Voß
>     <thomas.voss@xxxxxxxxxxxxx <mailto:thomas.voss@xxxxxxxxxxxxx>> wrote:
> 
>         On Fri, Oct 2, 2015 at 10:04 AM, Craig Harper
>         <dexteruk75@xxxxxxxxx <mailto:dexteruk75@xxxxxxxxx>> wrote:
>         > I dont think you need to open any flood gates.  I think we just need to
>         > break it down.  Keep the policies as they are Strict Policy or you could
>         > call it Battery Saver Mode, this could be the standard mode that the Ubuntu
>         > Touch uses now, If people want to optimise application for this mode and use
>         > all the helper functions and system services.  But creating other profiles
>         > that allow more user customised experience such a true multitasking,
>         > applications that need this mode and inform the users, that to use the
>         > application to its full potential you need to change the profile, with
>         > warnings about battery life, close applications when your not using them,
>         > etc etc.
>         >
> 
>         Please see my comment about UX later on.
> 
>         > This would also give developers a migration path, to move from a none
>         > optimised application, to a more Ubuntu Touch centric app, so still
>         > encouraging developer to port existing apps, without having to start of with
>         > how to get an app to work in a restrictive environment, once they see there
>         > application is getting users then they can invest time in optimising it.
>         > Maybe in the app store you can have Energy Efficiently rating for the app,
>         > to encourage developers to get there app optimised.
>         >
> 
>         Well, the same argument holds true in the case of a strict lifecycle
>         policy. Get a first port into the store,
>         potentially lacking functionality.  Users will like your app,
>         and you
>         can start integrating deeper with the system without
>         us compromising on platform principles and guidelines, or negatively
>         impacting users out there.
> 
> 
>     Why would users like your app if it cannot even do the main task it
>     is supposed to
>     accomplish? (<insert any sport tracking app here>)
>     I'd expect a list of reviews like:
>     "this doesn't work"
>     "1 star, useless"
>     "it's useless, doesn't work when screen is off"
>     "what were you thinking? Do you think I keep my screen on when running?"
> 
> 
> 
> Well, I did write a sport tracking app, and while people have complained
> a bit about the screen needing to stay on, they're generally pretty
> understanding and it hasn't been a big deal thus far...

This might work if you have some dock on your bike or so where you don't
constantly touch the screen but not when you want/need to keep the
device in your pocket.

Also, if keeping the screen permanently on in order to work around
battery saving optimizations, something's gone very wrong...

> 
>  
> 
>      
> 
>         > In this scenario if the device is running low on power, the user can make a
>         > choice to switch back the the Battery Saver Mode. maybe with some useful
>         > information about an estimate how long battery will last in this mode
>         > compared to Road Runner mode.
>         >
> 
>         Now that's not really a great user experience, is it? Did you
>         ever see
>         something like that happening on iOS?
>         It's common on Android, and one of the "features" that people
>         complain
>         about a lot, including numerous guides
>         for fixing battery drain (see [1]). We can certainly do a lot
>         better than that.
> 
>         > I think from this conversation, that if there is a complaint about Ubuntu
>         > Touch its this one, MultiTasking is a must, we must have away to run
>         > application in background without them being specially written.
>         >
> 
>         Now that would be somewhat alien. Developers aiming at mobile
>         platforms are already used to structure their
>         applications differently, precisely for integrating with execution
>         infrastructure and services offered by the respective
>         platforms. I don't think Ubuntu is (or even should be) different
>         here.
> 
>         I'm wondering: Did you try developing an app for android or iOS?
>         It's
>         an interesting exercise :)
> 
>         Cheers,
> 
>           Thomas
> 
>         > Craig
>         >
>         > On Fri, Oct 2, 2015 at 10:46 AM, Thomas Voß
>         <thomas.voss@xxxxxxxxxxxxx <mailto:thomas.voss@xxxxxxxxxxxxx>>
>         > wrote:
>         >>
>         >> On Fri, Oct 2, 2015 at 9:20 AM, Benjamin Zeller
>         >> <benjamin.zeller@xxxxxxxxxxxxx
>         <mailto:benjamin.zeller@xxxxxxxxxxxxx>> wrote:
>         >> > Hey all,
>         >> >
>         >> > I also put my thoughts inline:
>         >> >
>         >> >> Hey Simon,
>         >> >>
>         >> >> Thanks for your thoughts and ideas. Please find my
>         suggestions and
>         >> >> replies inline:
>         >> >>
>         >> >> On Thu, Oct 1, 2015 at 1:16 PM, Sturm Flut
>         <sturmflut@xxxxxxxxxxxxxx <mailto:sturmflut@xxxxxxxxxxxxxx>>
>         >> >> wrote:
>         >> >>>
>         >> >>> Good morning dear list,
>         >> >>>
>         >> >>> this has been brewing for quite some time now and the
>         discussion
>         >> >>> Krzysztof Tataradziński started a week ago didn't lead
>         anywhere again,
>         >> >>> so I'm starting it yet another time:
>         >> >>>
>         >> >>> In my opinion the "no background processing for apps on
>         the phone"
>         >> >>> design decision is wrong, is already hurting us too much
>         and has to be
>         >> >>> revoked as soon as possible.
>         >> >>>
>         >> >> To state this very clearly: We are looking at a continuum
>         of potential
>         >> >> solutions here.
>         >> >> It is unfortunately not as simple as just revoking one
>         solution and
>         >> >> picking another one, but
>         >> >> all the different objectives have to be balanced. To this
>         end, let me
>         >> >> clarify some of the guiding
>         >> >> principles and objectives that have to be accounted for in
>         the context
>         >> >> of lifecycle policies:
>         >> >>
>         >> >> (1.) Battery life: One of the scarcest resources on a
>         mobile device
>         >> >> and something that we aim
>         >> >> to protect as much as possible. Allowing arbitrary background
>         >> >> processing hurts a lot here, with Android's
>         >> >> lifecycle approach being one of the examples where battery
>         life
>         >> >> deteriorates and becomes unpredictable
>         >> >> depending on the apps that have been installed on the
>         device. Now this
>         >> >> wouldn't be a problem for the
>         >> >> quite technical audience subscribed to this mailing list.
>         But it is
>         >> >> certainly an issue for the average user who
>         >> >> should not be forced to maintain a list of running apps
>         and processes
>         >> >> just to achieve sensible battery life.
>         >> >
>         >> > Agreed, having a solution that protects battery life is
>         important,
>         >> > however
>         >> > that does not mean we can't have a way for the advanced
>         user to allow
>         >> > _some_ processes to run in background. Probably something
>         that can
>         >> > be turned on in the system settings per application. Giving
>         full control
>         >> > over
>         >> > what can run in background.
>         >> >
>         >> > With convergence in mind, a user probably will need a way
>         to allow his
>         >> > terminal
>         >> > to run in background while it compiles something, lets also
>         make sure
>         >> > we can give solutions to a more technical audience. After
>         all this is
>         >> > the
>         >> > audience
>         >> > that uses Ubuntu Phone atm!
>         >>
>         >> Sure, I'm not opposed to having dev-mode settings that can be
>         >> leveraged by tech-savvy users. However, as saviq pointed out,
>         >> we have to avoid apps not working as expected if the average
>         user does
>         >> not opt-in. On top, if we just allow settings to be tweaked
>         >> there is no reason for app authors to pick up our platform
>         principles
>         >> and guidelines. The next step is a wave of apps in the store that
>         >> only work if you switch on certain settings on the device.
>         That's a
>         >> worst case scenario from my POV.
>         >>
>         >> >>
>         >> >> (2.) Security & privacy: One of the worst case examples
>         are apps
>         >> >> running in the background, feeding on a device's
>         >> >> sensor data and sending it off to the cloud. There have
>         been numerous
>         >> >> examples exploiting the issue on Android.
>         >> >> Again, the technical audience subscribed here would be
>         fine, the
>         >> >> average user likely is not.
>         >> >
>         >> > But won't this be possible with a system service and
>         helpers as well?
>         >> > Lets say we have a system service that lets a helper
>         collect GPS data,
>         >> > won't the helper be able to upload that data?
>         >> > Because some apps might even need this, lets look at the
>         famous sports
>         >> > tracker
>         >> > example. It needs to collect your gps positions and upload
>         them.
>         >>
>         >> Well, that's debatable. We usually go for a very strict
>         setting (e.g.
>         >> in scopes) that prevents network access if access to local
>         >> resources is granted. You only get network access if the only
>         local
>         >> resources you access are the ones specific to your app.
>         >>
>         >> We surely cannot solve the problem of spying applications
>         once and for
>         >> all. But we can work hard to make sure that a user
>         >> stays on top of what apps _and_ the system is doing, putting
>         her in a
>         >> position of control and power.
>         >>
>         >> >>
>         >> >> (3.) Predictable performance: A phone is expected to
>         perform whenever
>         >> >> it is needed to, and the application being front & center
>         >> >> to the user should provide a smooth and crisp experience.
>         To this end,
>         >> >> the system has to be able to assign as much
>         performance-relevant
>         >> >> resources as possible to the currently focused app. If
>         arbitrary
>         >> >> background processing is allowed, the respective
>         scheduling problem
>         >> >> becomes
>         >> >> very very hard to solve (if possible at all).
>         >> >
>         >> > Couldn't that be solved by giving the background processes
>         time slots
>         >> > when they can execute? And probably an API to register
>         system resources
>         >> > that can wake them up for a time frame? Lets say a socket.
>         >>
>         >> Sure, moving processes to specific cgroups if they are not
>         visible to
>         >> the user is possible and something we have on the list
>         >> to explore. However, we are talking about heuristics, not a
>         solution
>         >> that will always work regardless of the installed apps.
>         >>
>         >> >>
>         >> >> Please note that there are way more aspects to this
>         problem space than
>         >> >> just the three points I elaborated on before.
>         >> >> The key point is: For a technical or tech-savvy audience, the
>         >> >> lifecycle approach we are taking for the phone might seem
>         overly
>         >> >> restricted.
>         >> >> However, for the average user, such a harsh policy comes
>         with a lot of
>         >> >> benefits.
>         >> >
>         >> > I still agree for the average user will benefit from that,
>         but until we
>         >> > reach
>         >> > that point of completeness we just lack critical APIs for
>         apps, and devs
>         >> > might not even start to write their apps or contact us to
>         tell: "Hey I'm
>         >> > missing
>         >> > this or that, could you implement it please".
>         >> >
>         >>
>         >> Well, I would argue that we already have a lot of feedback
>         that we
>         >> have to systematically work on.
>         >> We either go down the route of systematically iterating on
>         the system
>         >> capabilities or we open the flood gates and
>         >> just give up on our principles. Point being that taking
>         features back
>         >> is almost impossible
>         >>
>         >> > Another drawback of doing things completely different is of
>         course the
>         >> > higher
>         >> > cost in porting applications to our platform. There is no
>         way to port a
>         >> > Qt
>         >> > app
>         >> > that requires background processing to Ubuntu Phone without
>         rewriting
>         >> > big
>         >> > parts
>         >> > of it to use our system services and helpers. So the
>         porting is no
>         >> > longer
>         >> > effortless
>         >> > thus appdevs will wait for a critical mass of phone users
>         to appear but
>         >> > the
>         >> > phone
>         >> > users will only appear when there is a critical mass of
>         apps.... this
>         >> > problem again.
>         >> >
>         >>
>         >> I'm all in for making porting as efficient as possible. That
>         does not
>         >> imply, though, to give up
>         >> on our principles. In addition, our platform behaves and operates
>         >> similar to other major mobile OS's, with the
>         >> notable difference that we are working hard to ensure that "good
>         >> practice" is actually adhered to :)
>         >>
>         >> I think we should avoid having battery optimizers and similar
>         system
>         >> maintenance tools in the store.
>         >>
>         >> >>> Why? Let me make a list of examples.
>         >> >>>
>         >> >>> - Open Telegram for Ubuntu, send a picture to a friend,
>         switch to
>         >> >>> another app or scope. On any other platform the message
>         would continue
>         >> >>> to be sent in the background, but not on Ubuntu. Now you
>         might say
>         >> >>> "Telegram is going to become a Telepathy plugin, and that
>         one can run
>         >> >>> in
>         >> >>> the background", but there is no sign of a Telepathy
>         system service
>         >> >>> and
>         >> >>> even if it were already present, you're now forcing
>         everybody to build
>         >> >>> Telepathy plugins for their services. That's just not
>         going to happen.
>         >> >>> Let's be honest, WhatsApp and Viber and friends will not
>         go this way
>         >> >>> for
>         >> >>> a number of very good (economic) reasons.
>         >> >>>
>         >> >>> - Notifications for the myriads of services out there. On
>         any other
>         >> >>> platform the app would just register a small background
>         service that
>         >> >>> wakes up every minute or so and then goes to sleep again.
>         Now you
>         >> >>> might
>         >> >>> say "We have the Ubuntu Push Notifications service for
>         that", but that
>         >> >>> requires the service provider to change his whole server
>         side, which
>         >> >>> noone except Telegram does. That's why we already have to run
>         >> >>> account-polld.
>         >> >>>
>         >> >> That's actually not true in the general case. Most often,
>         the services
>         >> >> use the platform's push notification
>         >> >> infrastructure to deliver updates. Please note that we require
>         >> >> account-polld to work around the limitation that
>         >> >> facebook, twitter and google have not integrated with our
>         >> >> infrastructure
>         >> >> (yet).
>         >> >>
>         >> >>> - An app that processes location updates in the
>         background, e.g. a
>         >> >>> fitness tracker or a navigation app. Now this is IMO the
>         best example
>         >> >>> for how the "no background processing" decision
>         complicates everything
>         >> >>> to infinity: The simple solution would be to have a small
>         background
>         >> >>> service that just does whatever it has to do. But since
>         we can't do
>         >> >>> that, we have to come up with a very convoluted system
>         that most
>         >> >>> likely
>         >> >>> involves registering for some list of events with
>         location-service,
>         >> >>> which then calls a small handler binary provided by the
>         app (like the
>         >> >>> push notifications design). But how do you prevent that
>         handler from
>         >> >>> running amok, or from just keep running in the
>         background? Ah, yeah,
>         >> >>> kill it after five seconds, like the push notification
>         handlers. But
>         >> >>> five seconds on a slow CPU aren't much when I e.g. also
>         have to do
>         >> >>> some
>         >> >>> I/O and use D-Bus, while five seconds on a fast CPU are a
>         lot of time
>         >> >>> if
>         >> >>> I don't have to do much. So what's the right maximum
>         runtime for a
>         >> >>> handler that covers all use cases? There is none. Will
>         >> >>> location-service
>         >> >>> support all the event/filter/callback options I need for
>         my specific
>         >> >>> app? Most likely not.
>         >> >>>
>         >> >> It will grant a certain amount of processing time to your
>         app or
>         >> >> better its location update helper. A fitness tracker could
>         easily just
>         >> >> store
>         >> >> the respective update for later processing.
>         >> >
>         >> > Even a fitness tracker might give you stats while you are
>         still running,
>         >> > e.g. Endomondo tells you how fast you managed to run the
>         last lap, for
>         >> > that is has to process the data while the screen is off and
>         not just
>         >> > store
>         >> > it.
>         >> > Not sure if the small timeslot for a helper is enough for that.
>         >> >
>         >>
>         >> Now that's certainly a solvable problem.
>         >>
>         >> >> A navigation app would
>         >> >> require more sophisticated schemes, but we are happy to
>         evolve the
>         >> >> service as required (as pointed out earlier).
>         >> >>
>         >> >>> - Cloud and P2P services like Owncloud, Dropbox,
>         Syncthing, Tox etc.
>         >> >>> Apps that control devices via Bluetooth, like my FitBit or a
>         >> >>> Smartwatch.
>         >> >>> What's the design for those? More system services? We
>         can't afford to
>         >> >>> write a system service for each and every use case. That
>         doesn't
>         >> >>> scale.
>         >> >>> It already isn't scaling right now. And even if we could,
>         is the
>         >> >>> proposed solution really to have 100 system services
>         running for all
>         >> >>> users, instead of five to ten third party background
>         processes that
>         >> >>> the
>         >> >>> user really needs and knowingly installed herself?
>         >> >>>
>         >> >> We would not write a system service for each of them, but
>         instead
>         >> >> provide a framework to handle the specific problem area.
>         >> >> It is a lot of work to get those frameworks right, and
>         your help is
>         >> >> obviously greatly appreciated in doing so. Other than
>         that, it is
>         >> >> really really easy
>         >> >> to integrate cloud services with the content-hub
>         infrastructure and
>         >> >> thus exposing the content in the system.
>         >> >>
>         >> >>> I can come up with many more examples for how this is simply
>         >> >>> complicating our lives, and for what reason? A (tiny)
>         gain in battery
>         >> >>> life.
>         >> >>
>         >> >> Please see the initial list of objectives and guiding
>         principles. On a
>         >> >> very personal note, I don't think the aim should be to
>         make "our" life
>         >> >> easier but
>         >> >> to make the life of users as pleasant as possible. If that
>         means to
>         >> >> come up with novel and probably harsh solutions and
>         policies to
>         >> >> address certain
>         >> >> problems ... that's what it is.
>         >> >
>         >> > I agree that we should not make "our" life easier and aim
>         for a good
>         >> > solution, however we need to be careful to not aim too high
>         for a goal
>         >> > we can not reach in a suitable time.
>         >> >>
>         >> >>
>         >> >> We have many apps that cannot be built/ported right now
>         because we
>         >> >>>
>         >> >>> are waiting for APIs and system services that haven't
>         been coming for
>         >> >>> a
>         >> >>> year now and only have to be created because we can't
>         just run in the
>         >> >>> background. Do we even know if the effect on battery life
>         is really
>         >> >>> worth it? I challenge you to run e.g. Dekko as a
>         lifecycle exception
>         >> >>> and
>         >> >>> check if the background polling is noticeable. Because
>         it's not like
>         >> >>> those system services don't consume any resources at all,
>         the work has
>         >> >>> to be done by somebody. A "bad" Telepathy plugin will
>         consume just as
>         >> >>> much resources as a "bad" background process, the
>         "better" design of
>         >> >>> using Telepathy alone is not going to prevent this.
>         >> >>>
>         >> >> They do consume resources, for sure. The point is: A fixed
>         set of
>         >> >> system services controlled and maintained by us results in
>         predictable
>         >> >> battery life (modulo bugs).
>         >> >> The real question is not if battery life would deteriorate for
>         >> >> allowing a specific app to poll in the background.
>         Instead, the
>         >> >> question has to target the general behavior of
>         >> >> the system with arbitrary apps being installed in the system.
>         >> >
>         >> > So please correct me if I'm wrong, is the idea here that
>         lets say
>         >> > Facebook
>         >> > provides
>         >> > his own closed telepathy plugin? If that is so, what's the
>         difference
>         >> > from a
>         >> > "normal"
>         >> > background process? If that plugin behaves badly we can not
>         do much
>         >> > about it
>         >> > right?
>         >> > At least not without probably breaking its functionality.
>         >> >
>         >>
>         >> There are multiple benefits to it:
>         >>
>         >>   (1.) Tight integration with our platform UI and UX.
>         >>   (2.) The ability to tightly control resources granted to the
>         >> respective backend.
>         >>   (3.) We will likely setup a more strict review procedure
>         for those
>         >> extensions, together with domain-specific test-suites and
>         potentially
>         >> audits.
>         >>
>         >> In terms of handling extensions that behave poorly: With the
>         context
>         >> of the problem domain (e.g., messaging) handling errors and
>         confining
>         >> operation in general becomes a lot more tractable. The more
>         specific
>         >> the use-case the more specific control mechanisms can be.
>         >>
>         >> > Or is the idea here we will provide something more generic
>         that will use
>         >> > app
>         >> > helpers
>         >> > like we do in other scenarios?
>         >> >>
>         >> >>
>         >> >>> What also really confuses me about this issue is that the "no
>         >> >>> background
>         >> >>> processing" approach apparently only applies to mobile
>         devices and
>         >> >>> confined apps. Isn't that somehow against the whole idea
>         of an Ubuntu
>         >> >>> that is the same on all devices?
>         >> >>>
>         >> >> It certainly is not. A lifecycle policy is always specific
>         to a
>         >> >> device, its resources and even the specific usage scenario.
>         >> >> With that, allowing multiple apps to run even if not
>         focused in
>         >> >> desktop-like use-cases is perfectly fine. Even altering
>         the behavior
>         >> >> in case of a phone being plugged in to a power supply would be
>         >> >> perfectly
>         >> >> fine.
>         >> >
>         >> > +1, but it would be nice if the user could override that in
>         some way.
>         >> > Right
>         >> > now I'm also
>         >> > able to override powersaving on my laptop if I really want
>         to waste
>         >> > battery
>         >> > life.
>         >> >>
>         >> >>
>         >> >>> I would really like to hear all your thoughts on this and
>         how we're
>         >> >>> going to solve the situation, because we quite frankly
>         have actual
>         >> >>> problems (broken GPS, no SD card support for apps, no
>         Bluetooth, no
>         >> >>> SIM
>         >> >>> Toolkit, etc.) which should have a much higher priority
>         than building
>         >> >>> convoluted and complex designs just to potentially save
>         three hours of
>         >> >>> battery life over the course of a week.
>         >> >>>
>         >> >> As pointed out in my introductory comment: battery life is
>         one of
>         >> >> multiple concerns we have to address.
>         >> >>
>         >> >> Cheers,
>         >> >>
>         >> >>    Thomas
>         >> >
>         >> > Just want to point out, I'm aware this is no easy decision
>         and with
>         >> > whatever
>         >> > we decide we have to stay with or break apps later. So
>         there is no
>         >> > temporary
>         >> > solution we
>         >> > can deprecate later without breaking apps, which is IMHO
>         not an option.
>         >> >
>         >>
>         >> +1, whatever we come up with has to be aligned with what we
>         want to
>         >> support in the future.
>         >>
>         >> Cheers,
>         >>
>         >>   Thomas
>         >>
>         >> > Cheers,
>         >> >
>         >> > Benjamin
>         >>
>         >> --
>         >> Mailing list: https://launchpad.net/~ubuntu-phone
>         >> Post to     : ubuntu-phone@xxxxxxxxxxxxxxxxxxx
>         <mailto:ubuntu-phone@xxxxxxxxxxxxxxxxxxx>
>         >> Unsubscribe : https://launchpad.net/~ubuntu-phone
>         >> More help   : https://help.launchpad.net/ListHelp
>         >
>         >
> 
>         --
>         Mailing list: https://launchpad.net/~ubuntu-phone
>         Post to     : ubuntu-phone@xxxxxxxxxxxxxxxxxxx
>         <mailto:ubuntu-phone@xxxxxxxxxxxxxxxxxxx>
>         Unsubscribe : https://launchpad.net/~ubuntu-phone
>         More help   : https://help.launchpad.net/ListHelp
> 
> 
> 
>     --
>     Mailing list: https://launchpad.net/~ubuntu-phone
>     Post to     : ubuntu-phone@xxxxxxxxxxxxxxxxxxx
>     <mailto:ubuntu-phone@xxxxxxxxxxxxxxxxxxx>
>     Unsubscribe : https://launchpad.net/~ubuntu-phone
>     More help   : https://help.launchpad.net/ListHelp
> 
> 
> 
> 

Attachment: signature.asc
Description: OpenPGP digital signature


Follow ups

References