sikuli-driver team mailing list archive

[bengalih <question197639@xxxxxxxxxxxxxxxxxxxxx>]

New question #197639 on Sikuli:
https://answers.launchpad.net/sikuli/+question/197639

Hello Again!

Rai and I had a lot of discourse while developing my app to automate Netflix controls which I posted here:
https://answers.launchpad.net/sikuli/+question/193849

Unfortunately, Netflix has just released a new Silverlight control package which looks quite a bit different.
I set about yesterday afternoon updating my script to account for the image changes.  At first, things seemed that they were going to pretty easy...update some image files, change a few scan region areas.  I then went about testing.

All the resolutions on my laptop worked successfully (1024x768,1280x720,1280x800).  However, when I moved over to my big screen at 1920x1080, thing all of a sudden didn't work.  The previous Netflix control images scaled perfectly at all resolutions so the screen captures taken on my laptop at 1280x800 translated fine across ALL resolutions I attempted (I did however need to account for different scan regions).

After much trial and error I have discovered that the new Netflix images are scaling differently at different resolutions.  Not only this, but even at the same resolution there can be as many as 3 different sizings of the same image.  The reason all my laptop tests worked is because the images don't seem to change at all from 1024x768 until you get up to near 1600x900.

Let me explain further:

Netflix opens in a browser (IE) and can be viewed in 3 modes:
1) Normal browser  Non full screen - the default view
2) Normal browser Full Screen - after clicking on the full screen button
3) Full Screen browser (F11) -  this is by bringing the browser itself into fullscreen, the new Netflix control also seems to go full screen in this mode, but control resolution is slightly different than #2.

I have created this comparison chart to show the differences (in some cases minute) between the images at the different resolutions and different viewing methods:

http://i49.tinypic.com/23ksqyu.png

With the 7 resolutions I tested, that is 21 different scenarios and I came up with 8 distinct sizes for the icon (in this case the subtitle icon).  These are labeled (a)-(f) in the image.

I believe that Sikuli does not yet have a way to handle this properly (correct me if I am wrong).  If so, I am hoping perhaps for some suggestions on how I can work around the issue in my script.  When I first discovered the problem I initially created two definitions for button sets and called upon the appropriate one based on the discovered resolution.  Unfortunately this does not work completely because even at the same resolution the icons can change based on the viewing.

I tried playing around with match similarity settings and had some success, however as you can see from the bottom image of the Control Bar, some images are similar enough that being to lax in the settings can confuse them.  Also, other things from within the search region may get picked up.  I have restricted my search regions to smaller areas of the screen, but since I didn't want to have to define precise regions for every resolution, there is some area outside of the control bar that has the video underlay beneath it...this can further complicate searching if the minimum similarity is set to low.

I am hoping for some suggestions (for this non-coder :D) as to how I might be able to structure this in a KISS/DRY fashion to account for the changes.  My main goals would be (in order of precedence:

1) Accuracy of finding the images
2) Speed of finding the images
3) KISS/DRY in the code (in terms of not having to granularly define every region at every resolution precisely or switch similarity settings in a like fashion).

As always, I appreciate any help and guidance you can give. 





-- 
You received this question notification because you are a member of
Sikuli Drivers, which is an answer contact for Sikuli.