ubuntu-phone team mailing list archive

[Thomas Voß <thomas.voss@xxxxxxxxxxxxx>]

Hey all,

with the transition to Qt 5.2, we encountered two regressions when the
screen is off:

  (1.) Event processing in the shell is not working anymore (e.g.,
volume up/volume down).
  (2.) Event processing in apps stops working, too. This is especially
visible in the music player app, with the playlist not advancing due
to an EndOfStream event not being processed.

Saviq did an awesome job, and we tracked down the problem to a call to
eglSwapBuffers (in the shell and in the apps) blocking due to the
screen being turned off. More importantly, there was a change in Qt's
renderer implementation, blocking the overall UI thread and event
processing on a call to eglSwapBuffers. This fundamental architectural
shift in Qt with respect to tying rendering and signal/event servicing
together ultimately causes both issues mentioned before if the
rendering is blocked/waiting in eglSwapBuffers. We reached out to
upstream to seek clarification, the respective discussion is
summarized here:

  https://bugreports.qt-project.org/browse/QTBUG-37677

In summary, the behavioral change happened on purpose, and upstream
does not expect any call to eglSwapBuffers to block indefinitely.
Instead, the system is expected to either:

  * A synchronous notification from the system is expected such that
all Qt applications can flush their current rendering. Only after all
applications have ack'd the notification the system state is allowed
to change its state such that future calls to eglSwapBuffers would
block.
  * eglSwapBuffers immediately returns false, thereby indicating an
error, in the case of a potentially long-blocking call to
eglSwapBuffers.

Both would stop Qt from any further rendering, and with that, unblock
the UI thread for future operations. To resume rendering, the system
has to notify Qt applications that they have been exposed, i.e., that
they are visible to the user again.

Alternatively, all event processing that should happen independent of
the UI thread can be offloaded to another thread.

We spent some time evaluating the situation and both alternatives.
While we can certainly map our lifecycle events to the respective Qt
events, we would have to block turning off the screen until all
applications have ack'd and thus told us that they are good to go. In
addition, we would need to handle applications not responding,
including potential grace periods such that apps can flush their
rendering queue. Moreover, we think that interpreting a potentially
blocking call to eglSwapBuffers as an error case is not correct. From
our POV, a call to eglSwapBuffers is a legitimate outcome, and we (as
well as many others) leverage this semantics for throttling client
applications.

Right now, we are considering the second alternative as it is much
less intrusive. In the shell, we will leverage Mir's input event
filter chains and offload reacting to certain kind of input events to
a thread independent of Qt's UI thread. For the music player app, the
issue will be solved when the media hub lands. The music player app
will just "offload" music playback and playlist iteration to the media
hub and we can expose the music player app to the usual application
lifecycle (right now, it is granted an exception and is allowed to run
when not focused).

Please also note that the solution for the shell might serve as a
reference for app authors who want to process events (such as sensor
readings) at a higher rate than vblanc, and for that decouple their
processing from the UI thread.

The overall topic is quite complex and we might have missed something
in evaluating the different alternatives, so consider this mail an
early heads up together with a summary of our approach and ideas.
Gerry is working on a POC for Unity8 right now, and we will keep you
posted on the progress of that work.

Thanks,

  Thomas