sikuli-driver team mailing list archive

[Edmundo Valle Neto <question263287@xxxxxxxxxxxxxxxxxxxxx>]

New question #263287 on Sikuli:
https://answers.launchpad.net/sikuli/+question/263287

I am with a problem that I cannot solve in a simple way:

I need to manipulate the screen of a program that have comboboxes on it, to be more specific Im using Linux and the program is a Java program executed using Java Web Start. Its a normal address screen, I choose the state and then choose the city, the list of states is not that big but the list of cities is.

I can try to use Tesseract using something like "region.right(xxx).text()" and keep reading the options until I get the one I want, the problem with Tesseract is that it misses around 8% of all tries. I could not improve that changing fonts (fortunately I can do that because the program uses TinyLaF java package and I can change its theme), it still reads some chars wrong (Im not dealing with accentuated chars here).

The city names doesn't match a 100% when I choose it, so I need to try to choose it, read what was chosen and decide if I want to stick with that choice, I can see if what I choose is exactly what I want or use SequenceMatcher from difflib and get a ratio, I will not discuss that. What Im discussing is how do I read what I choose.

1. I can try to teach Tesseract to process a specific font, I don't have sure if Tesseract can reach 100% of correct readings this way.

2. I found the program inside the Java cache and the jar file that have the text files with all the city names, I can load and process my entire country using python when my script loads, sort the lists and know whats the position on the list is the city I want and I will know exactly how it was written. Its a lot of work to choose a simple item in a combobox and I will not have that luck with other programs.

In Linux I have wmctrl that can manipulate a window but does not reach its internal controls. In AutoHotKey for Microsoft Windows it seems to have a command that do this, called ControlGetText http://www.autohotkey.com/docs/commands/ControlGetText.htm

Anyone knows if there is a way to read the text of a control inside a program without using Tesseract using python or any external tool?

-- 
You received this question notification because you are a member of
Sikuli Drivers, which is an answer contact for Sikuli.