meadl-devel team mailing list archive
-
meadl-devel team
-
Mailing list archive
-
Message #00078
Re: Accessible options menu
Hi Jorge,
For some reason, I am not able to push my branch on to launchpad, there is
some error coming in bazaar. I would try to fix it and upload my branch as
soon as possible.
Meanwhile, I have started working on the next problem of giving access the
options menu. I also ran the tests you mentioned. Here are my findings from
day 2-
-
I started with adding the code to trigger a sequence of key-events in
response to pressing the 'fake options menu key'.
-
I modified the code for keyDownUp function such that, whenever it
received options menu key-code, it would first send the key-event for
options menu. Then it waits for 3 seconds before firing in another key-event
of D-pad up key. Finally, after another 3 second wait, it would fire the
event for D-pad right key.
-
After trying this code on some applications, it seemed that the Dpad key
events were sent to the activity that opened the menu, instead of menu
itself.
-
This may be because the getCurrentInputConnection gives the input
connection for the focused activity(as mentioned in its doc), and since menu
is not a different activity, but a part of the activity that opened it, so
the key-events are sent to the activity that opened it.
-
I printed the human-readable format of current Input Connection before
and after opening the menu on Log, and as expected,both times the input
connections had the same value in Log.
-
This confirms that input connection which is used to send activity does
not change even after opening the menu and thus key-events are still sent to
activity.
-
I then moved on to try some other methods to inject key-events. First
method I came across was using the IEWindow Manager and the Service Manager.
-
After doing some research about IEWindow Manager class, I found out that
this was available in very early versions of andorid APIs but has been
removed since from android public api as its development was not stable and
was not safe for future.
-
Next, I came across the Instrumentation class of android. The main
purpose of this class is for unit testing the applications, by generating
automated sequence of key-events, and is not recommended for use in normal
applications.
-
The instrumentation class cannot be run in main thread, so I put the key
inject function of instrumentation class in a parallel thread. When I tried
running the code, pressing on fake menu button generated a security
Exception.
-
The exception stated that Injecting to another application requires
inject_event permission.I then started to find more about this error and
permission and came across following.
-
The android.permission.INJECT_EVENT permission is not that of the
application, the *WindowManagerService* must have this permission
otherwise requests to inject events into other applications' windows will be
rejected.
-
By default, WindowManagerService does not have this permission, that is
why it is impossible to generate events for somebody else's window.
-
The IEWindow manager would also have faced the same problem, if it was to
be used in worst case scenario.
-
There were some official android blogs also that said that it is not
possible to inject key-events into another application for security reasons
and there is no way around this.
http://www.mail-archive.com/android-developers@xxxxxxxxxxxxxxxx/msg13424.html
I would continue with my experimentation with existing methods to see if
there is still a way around or if there is a new method available.
I will also update the wiki with details of each of the above methods and
problems in them
Rishabh
On Fri, Apr 15, 2011 at 8:26 PM, Jorge Silva <jsilva@xxxxxxx> wrote:
> great job Rishabh, can you please upload your modified branch back to
> Launchpad so I can check it out? If you have done so already, please just
> send me the link.
>
> BTW. Now that this is solved, I'd like to do a couple more tests before we
> move on to extracting the contents of the options menu (because we may be
> able to solve it without having to go that far). Would it be possible for
> you to add code to trigger a sequence of key events in response to pressing
> the 'fake' options menu key you have created? Something like this:
>
> 1. Press the *modified key* to open the options menu
> 2. Wait a few seconds (to allow for the options menu to be shown) and then
> send a DPAD event.
> 3. Wait a few more seconds and send another DPAD event
>
> The goal would be to see if those events get sent to the options menu. If
> they aren't, then we should try the other methods for injecting key events
> on the URLs I sent earlier and see if any of those would do the right thing.
>
> I believe you can emulate a key press at any time from the IME by just
> calling the onKey() method with the right code, and to test the other
> methods of injecting keys, you would just have to modify the keyDownUp()
> method.
>
> Let me know if you have any questions...
>
> cheers!
> Jorge
>
>
> On 11-04-15 08:08 AM, Rishabh Garg wrote:
>
>> Hi Jorge,
>>
>> I think I have found the solution for triggering the options menu via a
>> key event. I have also tracked why something else was being triggered by
>> the options menu key-code, the problem which Kevin was facing on his
>> device. Here are my findings-
>>
>> *
>>
>> I started by changing the key-code value for one of the navigation
>> key(the right key) to the key-code of the menu key(82) in the xml
>> file. As Kevin mentioned earlier, this was triggering something
>> weird instead of the menu
>>
>> *
>>
>> I then tried replacing the menu key-code by all other possible
>> key-codes to check which of the key-codes behaved differently.
>>
>> *
>>
>> When experimenting with different key-codes resulted in nothing,I
>> decided to backtrack the flow of code that resulted in the key-event.
>>
>> *
>>
>> I printed the key-code received by keyUpDown function in the log,
>> to check if the correct event was received by the function.
>>
>> *
>>
>> After checking the log for various key-codes in xml, I found that
>> the keyUpDown function received key-codes for some specific keys
>> only, and this function was not called for rest of the cases.
>>
>> *
>>
>> I started to find the functions that called the keyUpDown function
>> and analyze them.
>>
>> *
>>
>> I found that that keyUpDown function was called by a handler
>> function for nav bar, which in turn was called by onKey function.
>> For some reason the onKey function was not able to call the
>> handler function for nav bar, and instead called some other
>> function which generated some other event.
>>
>> *
>>
>> It finally turned out that the onKey function uses another
>> function to check key-code it reveives, isNavigationKey function.
>> *
>>
>> isNavigationKey function checks whether the recieved key code
>> belonged to a set of key-codes(up,down,left,right,center or back).
>> If it does, it passes it to KeyUpDown function via handler , but
>> for any other key code, it passes it to handleCharacterfunction,
>> which uses prediction etc. to trigger some other key-event. This
>> is the reason some other event was triggered in Kevin's device.
>> *
>>
>> I added the key-code for menu in the set of key-codes in the
>> isNavigationKey function, which enabled it to pass the key-code
>> for menu to keyUpDown function.
>> *
>>
>> The KeyUpDown function was able to trigger the key-event for menu
>> correctly .
>>
>> I changed the key-code of*right button in nav bar* to act as *menu
>> button*.
>> After checking the new code on emulator, I have also thoroughly checked
>> it on my device. The menu is correctly opened for android homescreen as
>> well as all 3rd party applications(Gmail,Gtalk,
>> Youtube etc.) by the right button in nav bar.
>>
>> I have uploaded the new apk with these change on this link-
>> http://web.iiit.ac.in/~rishabh.garg/IMESettings.apk
>> <http://web.iiit.ac.in/%7Erishabh.garg/IMESettings.apk>. Could you
>>
>> please confirm if the above solution is working fine or if there is any
>> problem in it.
>> I would also update my findings on this wiki page -
>> http://wiki.scyp.atrc.utoronto.ca/w/Tekla/Accessible_Options_Menu.
>> .I think this solves the first 2 points of our solution blueprint-
>> 1. Open the options menu programatically
>> 2. Detect the event triggered when the options menu is open,
>> Regards
>> Rishabh Garg
>>
>>
>> On Thu, Apr 14, 2011 at 3:09 AM, Jorge Silva <jsilva.idi@xxxxxxxxx
>> <mailto:jsilva.idi@xxxxxxxxx>> wrote:
>>
>> Hola,
>>
>> As many of you already know, Tekla users are still locked out the
>> options menu because of two reasons:
>> 1. When the menu pops-up, it occludes the keyboard, and
>> 2. Even when the keyboard remains active, its key events are sent to
>> the
>> activity underneath, instead of to the menu on top
>>
>> The Google accessibility team has confirmed the issue is in the Android
>> framework and they are working on it, but we'll try a workaround that
>> has been drafted in the following blueprint:
>> https://blueprints.launchpad.net/meadl/+spec/accessible-options-menu
>>
>> Long-story short, in order to make this happen, the first thing we need
>> to do is make sure Tekla can open the options menu on any app,
>> programatically. The Goldeneye
>> (http://wiki.scyp.atrc.utoronto.ca/w/Goldeneye_Reading_Project)
>> project
>> devs already tried to do so by sending the corresponding key event but
>> the device they tested on wasn't very cooperative. We need to do the
>> following:
>> 1. Confirm whether in fact it is not possible to send the key event
>> using the current methods (code is at: http://goo.gl/dOenl)
>> 2. If the key event doesn't work, try other methods like:
>> http://stackoverflow.com/questions/3133318/open-the-options-menu-
>> programatically or
>> http://mylifewithandroid.blogspot.com/2009/01/generating-keypresses-
>> programmatically.html
>> <
>> http://mylifewithandroid.blogspot.com/2009/01/generating-keypresses-%0Aprogrammatically.html
>> >
>>
>> 3. If none of that works, then we should try looking for an intent
>>
>> Let me know if any of you would like to help trying this out... I am a
>> bit tied at the moment with other commitments. If you have any other
>> questions please feel free to reply to the thread.
>>
>> cheers!
>> Jorge
>> --
>> This message was sent from Launchpad by
>> Jorge Silva (https://launchpad.net/~jorge-silva
>> <https://launchpad.net/%7Ejorge-silva>)
>>
>> using the "Contact this team" link on the Tekla development team
>> team page
>> (https://launchpad.net/~meadl-devel
>> <https://launchpad.net/%7Emeadl-devel>).
>>
>> For more information see
>> https://help.launchpad.net/YourAccount/ContactingPeople
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~meadl-devel
>> <https://launchpad.net/%7Emeadl-devel>
>>
>> Post to : meadl-devel@xxxxxxxxxxxxxxxxxxx
>> <mailto:meadl-devel@xxxxxxxxxxxxxxxxxxx>
>>
>> Unsubscribe : https://launchpad.net/~meadl-devel
>> <https://launchpad.net/%7Emeadl-devel>
>>
>> More help : https://help.launchpad.net/ListHelp
>>
>>
>>
>>
>> --
>> Rishabh Garg
>> B.Tech CSE 2nd year
>> International Institute of Information Technology(IIIT)
>> Hyderabad,India
>>
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~meadl-devel
>> Post to : meadl-devel@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~meadl-devel
>> More help : https://help.launchpad.net/ListHelp
>>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~meadl-devel
> Post to : meadl-devel@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~meadl-devel
> More help : https://help.launchpad.net/ListHelp
>
--
Rishabh Garg
B.Tech CSE 2nd year
International Institute of Information Technology(IIIT)
Hyderabad,India
Follow ups
References
