ubuntu-phone team mailing list archive

[Andrea Bernabei <andrea.bernabei@xxxxxxxxxxxxx>]

On Fri, Oct 2, 2015 at 10:45 AM, Thomas Voß <thomas.voss@xxxxxxxxxxxxx>
wrote:

> On Fri, Oct 2, 2015 at 11:29 AM, Andrea Bernabei
> <andrea.bernabei@xxxxxxxxxxxxx> wrote:
> >
> >
> > On Fri, Oct 2, 2015 at 10:18 AM, Thomas Voß <thomas.voss@xxxxxxxxxxxxx>
> > wrote:
> >>
> >> On Fri, Oct 2, 2015 at 11:14 AM, Andrea Bernabei
> >> <andrea.bernabei@xxxxxxxxxxxxx> wrote:
> >> >
> >> >
> >> > On Fri, Oct 2, 2015 at 8:46 AM, Thomas Voß <thomas.voss@xxxxxxxxxxxxx
> >
> >> > wrote:
> >> >>
> >> >> On Fri, Oct 2, 2015 at 9:20 AM, Benjamin Zeller
> >> >> <benjamin.zeller@xxxxxxxxxxxxx> wrote:
> >> >> > Hey all,
> >> >> >
> >> >> > I also put my thoughts inline:
> >> >> >
> >> >> >> Hey Simon,
> >> >> >>
> >> >> >> Thanks for your thoughts and ideas. Please find my suggestions and
> >> >> >> replies inline:
> >> >> >>
> >> >> >> On Thu, Oct 1, 2015 at 1:16 PM, Sturm Flut
> >> >> >> <sturmflut@xxxxxxxxxxxxxx>
> >> >> >> wrote:
> >> >> >>>
> >> >> >>> Good morning dear list,
> >> >> >>>
> >> >> >>> this has been brewing for quite some time now and the discussion
> >> >> >>> Krzysztof Tataradziński started a week ago didn't lead anywhere
> >> >> >>> again,
> >> >> >>> so I'm starting it yet another time:
> >> >> >>>
> >> >> >>> In my opinion the "no background processing for apps on the
> phone"
> >> >> >>> design decision is wrong, is already hurting us too much and has
> to
> >> >> >>> be
> >> >> >>> revoked as soon as possible.
> >> >> >>>
> >> >> >> To state this very clearly: We are looking at a continuum of
> >> >> >> potential
> >> >> >> solutions here.
> >> >> >> It is unfortunately not as simple as just revoking one solution
> and
> >> >> >> picking another one, but
> >> >> >> all the different objectives have to be balanced. To this end, let
> >> >> >> me
> >> >> >> clarify some of the guiding
> >> >> >> principles and objectives that have to be accounted for in the
> >> >> >> context
> >> >> >> of lifecycle policies:
> >> >> >>
> >> >> >> (1.) Battery life: One of the scarcest resources on a mobile
> device
> >> >> >> and something that we aim
> >> >> >> to protect as much as possible. Allowing arbitrary background
> >> >> >> processing hurts a lot here, with Android's
> >> >> >> lifecycle approach being one of the examples where battery life
> >> >> >> deteriorates and becomes unpredictable
> >> >> >> depending on the apps that have been installed on the device. Now
> >> >> >> this
> >> >> >> wouldn't be a problem for the
> >> >> >> quite technical audience subscribed to this mailing list. But it
> is
> >> >> >> certainly an issue for the average user who
> >> >> >> should not be forced to maintain a list of running apps and
> >> >> >> processes
> >> >> >> just to achieve sensible battery life.
> >> >> >
> >> >> > Agreed, having a solution that protects battery life is important,
> >> >> > however
> >> >> > that does not mean we can't have a way for the advanced user to
> allow
> >> >> > _some_ processes to run in background. Probably something that can
> >> >> > be turned on in the system settings per application. Giving full
> >> >> > control
> >> >> > over
> >> >> > what can run in background.
> >> >> >
> >> >> > With convergence in mind, a user probably will need a way to allow
> >> >> > his
> >> >> > terminal
> >> >> > to run in background while it compiles something, lets also make
> sure
> >> >> > we can give solutions to a more technical audience. After all this
> is
> >> >> > the
> >> >> > audience
> >> >> > that uses Ubuntu Phone atm!
> >> >>
> >> >> Sure, I'm not opposed to having dev-mode settings that can be
> >> >> leveraged by tech-savvy users. However, as saviq pointed out,
> >> >> we have to avoid apps not working as expected if the average user
> does
> >> >> not opt-in. On top, if we just allow settings to be tweaked
> >> >> there is no reason for app authors to pick up our platform principles
> >> >> and guidelines. The next step is a wave of apps in the store that
> >> >> only work if you switch on certain settings on the device. That's a
> >> >> worst case scenario from my POV.
> >> >
> >> >
> >> > If we really want users to be able to have full control over the
> devices
> >> > like
> >> > we say we do, then they should be able to enable/disable apps
> >> > permissions,
> >> > which is what BlackBerry10 does, for instance.
> >> >
> >> > And the apps will have to take care of that, warning the user whenever
> >> > they
> >> > cannot complete a task because of a permission that the user has
> denied.
> >> > That is giving power to the user, isn't it?
> >> >
> >>
> >> Sure, our trust infrastructure handles exactly that. The notable
> >> exception is that we don't offer a generic
> >> background execution service for reasons discussed here. Also: Putting
> >> users into power requires us to make
> >> sure that users are well informed and able to handle granted powers.
> >>
> >
> > Of course we should make sure that they are well informed. The problem
> here
> > is that
> > it seems we haven't moved anywhere regarding background processing for a
> > long time...
> >
> > how can we push this? Is there a roadmap you can share that shows how
> we're
> > going to
> > tackle the main usecases which have been brought over and over?
> >
>
> not as consolidated and clear as I would like it to be. So I will take
> the action to assemble the required information
> together with a central place to capture and handle use-cases brought
> up for background processing. For issue/use-case tracking
> purposes, I went ahead and created:
> https://launchpad.net/ubuntu-application-lifecycle
>
>
Great, thanks :)


> > I just fear we're kind of ignoring the problem, or at least that's what
> the
> > user sees from
> > the outside. Every time a new thread on background processing starts,
> we're
> > back to
> > square 1.
> >
>
> Rest assured that we are certainly not ignoring the problem. On the
> contrary, but we are also facing the reality of
> finite resources to tackle them. With that, we need a very clear
> understanding of use-cases and priorities.
>
>
Maybe we just need to reprioritize it based on users' feedback :/
This is one of the hottest topics on this ML, but yet unsolved after a long
time...

Finite resources is indeed a problem, that's when prioritization comes in :)

Let's not make that new project become "the graveyard of background
processing ideas",
the place where "eventually" someone will take inspiration from, if we
"ever" happen to
have 10x engineers :D


> Cheers,
>
>   Thomas
>
> > PS: I really appreciate you taking the time to reply to every single
> person
> > :)
> >
> >>
> >> >>
> >> >> >>
> >> >> >> (2.) Security & privacy: One of the worst case examples are apps
> >> >> >> running in the background, feeding on a device's
> >> >> >> sensor data and sending it off to the cloud. There have been
> >> >> >> numerous
> >> >> >> examples exploiting the issue on Android.
> >> >> >> Again, the technical audience subscribed here would be fine, the
> >> >> >> average user likely is not.
> >> >> >
> >> >> > But won't this be possible with a system service and helpers as
> well?
> >> >> > Lets say we have a system service that lets a helper collect GPS
> >> >> > data,
> >> >> > won't the helper be able to upload that data?
> >> >> > Because some apps might even need this, lets look at the famous
> >> >> > sports
> >> >> > tracker
> >> >> > example. It needs to collect your gps positions and upload them.
> >> >>
> >> >> Well, that's debatable. We usually go for a very strict setting (e.g.
> >> >> in scopes) that prevents network access if access to local
> >> >> resources is granted. You only get network access if the only local
> >> >> resources you access are the ones specific to your app.
> >> >>
> >> >> We surely cannot solve the problem of spying applications once and
> for
> >> >> all. But we can work hard to make sure that a user
> >> >> stays on top of what apps _and_ the system is doing, putting her in a
> >> >> position of control and power.
> >> >>
> >> >> >>
> >> >> >> (3.) Predictable performance: A phone is expected to perform
> >> >> >> whenever
> >> >> >> it is needed to, and the application being front & center
> >> >> >> to the user should provide a smooth and crisp experience. To this
> >> >> >> end,
> >> >> >> the system has to be able to assign as much performance-relevant
> >> >> >> resources as possible to the currently focused app. If arbitrary
> >> >> >> background processing is allowed, the respective scheduling
> problem
> >> >> >> becomes
> >> >> >> very very hard to solve (if possible at all).
> >> >> >
> >> >> > Couldn't that be solved by giving the background processes time
> slots
> >> >> > when they can execute? And probably an API to register system
> >> >> > resources
> >> >> > that can wake them up for a time frame? Lets say a socket.
> >> >>
> >> >> Sure, moving processes to specific cgroups if they are not visible to
> >> >> the user is possible and something we have on the list
> >> >> to explore. However, we are talking about heuristics, not a solution
> >> >> that will always work regardless of the installed apps.
> >> >>
> >> >> >>
> >> >> >> Please note that there are way more aspects to this problem space
> >> >> >> than
> >> >> >> just the three points I elaborated on before.
> >> >> >> The key point is: For a technical or tech-savvy audience, the
> >> >> >> lifecycle approach we are taking for the phone might seem overly
> >> >> >> restricted.
> >> >> >> However, for the average user, such a harsh policy comes with a
> lot
> >> >> >> of
> >> >> >> benefits.
> >> >> >
> >> >> > I still agree for the average user will benefit from that, but
> until
> >> >> > we
> >> >> > reach
> >> >> > that point of completeness we just lack critical APIs for apps, and
> >> >> > devs
> >> >> > might not even start to write their apps or contact us to tell:
> "Hey
> >> >> > I'm
> >> >> > missing
> >> >> > this or that, could you implement it please".
> >> >> >
> >> >>
> >> >> Well, I would argue that we already have a lot of feedback that we
> >> >> have to systematically work on.
> >> >> We either go down the route of systematically iterating on the system
> >> >> capabilities or we open the flood gates and
> >> >> just give up on our principles. Point being that taking features back
> >> >> is almost impossible
> >> >>
> >> >> > Another drawback of doing things completely different is of course
> >> >> > the
> >> >> > higher
> >> >> > cost in porting applications to our platform. There is no way to
> port
> >> >> > a
> >> >> > Qt
> >> >> > app
> >> >> > that requires background processing to Ubuntu Phone without
> rewriting
> >> >> > big
> >> >> > parts
> >> >> > of it to use our system services and helpers. So the porting is no
> >> >> > longer
> >> >> > effortless
> >> >> > thus appdevs will wait for a critical mass of phone users to appear
> >> >> > but
> >> >> > the
> >> >> > phone
> >> >> > users will only appear when there is a critical mass of apps....
> this
> >> >> > problem again.
> >> >> >
> >> >>
> >> >> I'm all in for making porting as efficient as possible. That does not
> >> >> imply, though, to give up
> >> >> on our principles. In addition, our platform behaves and operates
> >> >> similar to other major mobile OS's, with the
> >> >> notable difference that we are working hard to ensure that "good
> >> >> practice" is actually adhered to :)
> >> >>
> >> >> I think we should avoid having battery optimizers and similar system
> >> >> maintenance tools in the store.
> >> >>
> >> >> >>> Why? Let me make a list of examples.
> >> >> >>>
> >> >> >>> - Open Telegram for Ubuntu, send a picture to a friend, switch to
> >> >> >>> another app or scope. On any other platform the message would
> >> >> >>> continue
> >> >> >>> to be sent in the background, but not on Ubuntu. Now you might
> say
> >> >> >>> "Telegram is going to become a Telepathy plugin, and that one can
> >> >> >>> run
> >> >> >>> in
> >> >> >>> the background", but there is no sign of a Telepathy system
> service
> >> >> >>> and
> >> >> >>> even if it were already present, you're now forcing everybody to
> >> >> >>> build
> >> >> >>> Telepathy plugins for their services. That's just not going to
> >> >> >>> happen.
> >> >> >>> Let's be honest, WhatsApp and Viber and friends will not go this
> >> >> >>> way
> >> >> >>> for
> >> >> >>> a number of very good (economic) reasons.
> >> >> >>>
> >> >> >>> - Notifications for the myriads of services out there. On any
> other
> >> >> >>> platform the app would just register a small background service
> >> >> >>> that
> >> >> >>> wakes up every minute or so and then goes to sleep again. Now you
> >> >> >>> might
> >> >> >>> say "We have the Ubuntu Push Notifications service for that", but
> >> >> >>> that
> >> >> >>> requires the service provider to change his whole server side,
> >> >> >>> which
> >> >> >>> noone except Telegram does. That's why we already have to run
> >> >> >>> account-polld.
> >> >> >>>
> >> >> >> That's actually not true in the general case. Most often, the
> >> >> >> services
> >> >> >> use the platform's push notification
> >> >> >> infrastructure to deliver updates. Please note that we require
> >> >> >> account-polld to work around the limitation that
> >> >> >> facebook, twitter and google have not integrated with our
> >> >> >> infrastructure
> >> >> >> (yet).
> >> >> >>
> >> >> >>> - An app that processes location updates in the background, e.g.
> a
> >> >> >>> fitness tracker or a navigation app. Now this is IMO the best
> >> >> >>> example
> >> >> >>> for how the "no background processing" decision complicates
> >> >> >>> everything
> >> >> >>> to infinity: The simple solution would be to have a small
> >> >> >>> background
> >> >> >>> service that just does whatever it has to do. But since we can't
> do
> >> >> >>> that, we have to come up with a very convoluted system that most
> >> >> >>> likely
> >> >> >>> involves registering for some list of events with
> location-service,
> >> >> >>> which then calls a small handler binary provided by the app (like
> >> >> >>> the
> >> >> >>> push notifications design). But how do you prevent that handler
> >> >> >>> from
> >> >> >>> running amok, or from just keep running in the background? Ah,
> >> >> >>> yeah,
> >> >> >>> kill it after five seconds, like the push notification handlers.
> >> >> >>> But
> >> >> >>> five seconds on a slow CPU aren't much when I e.g. also have to
> do
> >> >> >>> some
> >> >> >>> I/O and use D-Bus, while five seconds on a fast CPU are a lot of
> >> >> >>> time
> >> >> >>> if
> >> >> >>> I don't have to do much. So what's the right maximum runtime for
> a
> >> >> >>> handler that covers all use cases? There is none. Will
> >> >> >>> location-service
> >> >> >>> support all the event/filter/callback options I need for my
> >> >> >>> specific
> >> >> >>> app? Most likely not.
> >> >> >>>
> >> >> >> It will grant a certain amount of processing time to your app or
> >> >> >> better its location update helper. A fitness tracker could easily
> >> >> >> just
> >> >> >> store
> >> >> >> the respective update for later processing.
> >> >> >
> >> >> > Even a fitness tracker might give you stats while you are still
> >> >> > running,
> >> >> > e.g. Endomondo tells you how fast you managed to run the last lap,
> >> >> > for
> >> >> > that is has to process the data while the screen is off and not
> just
> >> >> > store
> >> >> > it.
> >> >> > Not sure if the small timeslot for a helper is enough for that.
> >> >> >
> >> >>
> >> >> Now that's certainly a solvable problem.
> >> >>
> >> >> >> A navigation app would
> >> >> >> require more sophisticated schemes, but we are happy to evolve the
> >> >> >> service as required (as pointed out earlier).
> >> >> >>
> >> >> >>> - Cloud and P2P services like Owncloud, Dropbox, Syncthing, Tox
> >> >> >>> etc.
> >> >> >>> Apps that control devices via Bluetooth, like my FitBit or a
> >> >> >>> Smartwatch.
> >> >> >>> What's the design for those? More system services? We can't
> afford
> >> >> >>> to
> >> >> >>> write a system service for each and every use case. That doesn't
> >> >> >>> scale.
> >> >> >>> It already isn't scaling right now. And even if we could, is the
> >> >> >>> proposed solution really to have 100 system services running for
> >> >> >>> all
> >> >> >>> users, instead of five to ten third party background processes
> that
> >> >> >>> the
> >> >> >>> user really needs and knowingly installed herself?
> >> >> >>>
> >> >> >> We would not write a system service for each of them, but instead
> >> >> >> provide a framework to handle the specific problem area.
> >> >> >> It is a lot of work to get those frameworks right, and your help
> is
> >> >> >> obviously greatly appreciated in doing so. Other than that, it is
> >> >> >> really really easy
> >> >> >> to integrate cloud services with the content-hub infrastructure
> and
> >> >> >> thus exposing the content in the system.
> >> >> >>
> >> >> >>> I can come up with many more examples for how this is simply
> >> >> >>> complicating our lives, and for what reason? A (tiny) gain in
> >> >> >>> battery
> >> >> >>> life.
> >> >> >>
> >> >> >> Please see the initial list of objectives and guiding principles.
> On
> >> >> >> a
> >> >> >> very personal note, I don't think the aim should be to make "our"
> >> >> >> life
> >> >> >> easier but
> >> >> >> to make the life of users as pleasant as possible. If that means
> to
> >> >> >> come up with novel and probably harsh solutions and policies to
> >> >> >> address certain
> >> >> >> problems ... that's what it is.
> >> >> >
> >> >> > I agree that we should not make "our" life easier and aim for a
> good
> >> >> > solution, however we need to be careful to not aim too high for a
> >> >> > goal
> >> >> > we can not reach in a suitable time.
> >> >> >>
> >> >> >>
> >> >> >> We have many apps that cannot be built/ported right now because we
> >> >> >>>
> >> >> >>> are waiting for APIs and system services that haven't been coming
> >> >> >>> for
> >> >> >>> a
> >> >> >>> year now and only have to be created because we can't just run in
> >> >> >>> the
> >> >> >>> background. Do we even know if the effect on battery life is
> really
> >> >> >>> worth it? I challenge you to run e.g. Dekko as a lifecycle
> >> >> >>> exception
> >> >> >>> and
> >> >> >>> check if the background polling is noticeable. Because it's not
> >> >> >>> like
> >> >> >>> those system services don't consume any resources at all, the
> work
> >> >> >>> has
> >> >> >>> to be done by somebody. A "bad" Telepathy plugin will consume
> just
> >> >> >>> as
> >> >> >>> much resources as a "bad" background process, the "better" design
> >> >> >>> of
> >> >> >>> using Telepathy alone is not going to prevent this.
> >> >> >>>
> >> >> >> They do consume resources, for sure. The point is: A fixed set of
> >> >> >> system services controlled and maintained by us results in
> >> >> >> predictable
> >> >> >> battery life (modulo bugs).
> >> >> >> The real question is not if battery life would deteriorate for
> >> >> >> allowing a specific app to poll in the background. Instead, the
> >> >> >> question has to target the general behavior of
> >> >> >> the system with arbitrary apps being installed in the system.
> >> >> >
> >> >> > So please correct me if I'm wrong, is the idea here that lets say
> >> >> > Facebook
> >> >> > provides
> >> >> > his own closed telepathy plugin? If that is so, what's the
> difference
> >> >> > from a
> >> >> > "normal"
> >> >> > background process? If that plugin behaves badly we can not do much
> >> >> > about it
> >> >> > right?
> >> >> > At least not without probably breaking its functionality.
> >> >> >
> >> >>
> >> >> There are multiple benefits to it:
> >> >>
> >> >>   (1.) Tight integration with our platform UI and UX.
> >> >>   (2.) The ability to tightly control resources granted to the
> >> >> respective backend.
> >> >>   (3.) We will likely setup a more strict review procedure for those
> >> >> extensions, together with domain-specific test-suites and potentially
> >> >> audits.
> >> >>
> >> >> In terms of handling extensions that behave poorly: With the context
> >> >> of the problem domain (e.g., messaging) handling errors and confining
> >> >> operation in general becomes a lot more tractable. The more specific
> >> >> the use-case the more specific control mechanisms can be.
> >> >>
> >> >> > Or is the idea here we will provide something more generic that
> will
> >> >> > use
> >> >> > app
> >> >> > helpers
> >> >> > like we do in other scenarios?
> >> >> >>
> >> >> >>
> >> >> >>> What also really confuses me about this issue is that the "no
> >> >> >>> background
> >> >> >>> processing" approach apparently only applies to mobile devices
> and
> >> >> >>> confined apps. Isn't that somehow against the whole idea of an
> >> >> >>> Ubuntu
> >> >> >>> that is the same on all devices?
> >> >> >>>
> >> >> >> It certainly is not. A lifecycle policy is always specific to a
> >> >> >> device, its resources and even the specific usage scenario.
> >> >> >> With that, allowing multiple apps to run even if not focused in
> >> >> >> desktop-like use-cases is perfectly fine. Even altering the
> behavior
> >> >> >> in case of a phone being plugged in to a power supply would be
> >> >> >> perfectly
> >> >> >> fine.
> >> >> >
> >> >> > +1, but it would be nice if the user could override that in some
> way.
> >> >> > Right
> >> >> > now I'm also
> >> >> > able to override powersaving on my laptop if I really want to waste
> >> >> > battery
> >> >> > life.
> >> >> >>
> >> >> >>
> >> >> >>> I would really like to hear all your thoughts on this and how
> we're
> >> >> >>> going to solve the situation, because we quite frankly have
> actual
> >> >> >>> problems (broken GPS, no SD card support for apps, no Bluetooth,
> no
> >> >> >>> SIM
> >> >> >>> Toolkit, etc.) which should have a much higher priority than
> >> >> >>> building
> >> >> >>> convoluted and complex designs just to potentially save three
> hours
> >> >> >>> of
> >> >> >>> battery life over the course of a week.
> >> >> >>>
> >> >> >> As pointed out in my introductory comment: battery life is one of
> >> >> >> multiple concerns we have to address.
> >> >> >>
> >> >> >> Cheers,
> >> >> >>
> >> >> >>    Thomas
> >> >> >
> >> >> > Just want to point out, I'm aware this is no easy decision and with
> >> >> > whatever
> >> >> > we decide we have to stay with or break apps later. So there is no
> >> >> > temporary
> >> >> > solution we
> >> >> > can deprecate later without breaking apps, which is IMHO not an
> >> >> > option.
> >> >> >
> >> >>
> >> >> +1, whatever we come up with has to be aligned with what we want to
> >> >> support in the future.
> >> >>
> >> >> Cheers,
> >> >>
> >> >>   Thomas
> >> >>
> >> >> > Cheers,
> >> >> >
> >> >> > Benjamin
> >> >>
> >> >> --
> >> >> Mailing list: https://launchpad.net/~ubuntu-phone
> >> >> Post to     : ubuntu-phone@xxxxxxxxxxxxxxxxxxx
> >> >> Unsubscribe : https://launchpad.net/~ubuntu-phone
> >> >> More help   : https://help.launchpad.net/ListHelp
> >> >
> >> >
> >
> >
>