sikuli-driver team mailing list archive

[RaiMan <question224151@xxxxxxxxxxxxxxxxxxxxx>]

Question #224151 on Sikuli changed:
https://answers.launchpad.net/sikuli/+question/224151

RaiMan proposed the following answer:
at 1: see comment

at 2: rc2 is weaker in all aspects

at 3: see final comment. the pyramid algorithm does not affect accuracy,
it only helps to find the most probable area in the image to check for
the probe faster.

--- comment
accuracy with a given image and a given probe is as it results in using cv::matchTemplate. You cannot do anything else, than interpret the returned similarity value, which is between 0 and 1 and accept something above 0.99 as an exact match (exact means, that the match area contains the same pixels as the probe).
In fact the cv::matchTemplate returns an matrix of similarity values, where each value represents the similarity of the probe with the area in the image at this pixel as top left corner with the same size as the probe. You have to analyze this matrix and decide for a pixel, to be the top left corner of the match.
cv::matchTemplate lets you select the search algorithm. Sikuli uses the most suitable one for searching non-uniform images. Another one is used, when the probe is detected to contain images of nearly the same color.

--- You say something about "in terms of resolution changes, color scheme changes":
clearly said: this problem is not solved by Sikuli, because it cannot be solved by cv::matchTemplate: the possible match in the given image must have the same size in pixels as the given probe. if it differs in some color or brightness aspects, you will get lower similarity values. Then you have to decide, wether you accept these values as matches or not (problem: false positives). Again: a match score of 1 is only possible if the pixels in the image's match area are exactly the same one by one.

--- Resolution: hence it is no problem as long as resolution changes do not affect the individual pixel (e.g. an icon on the screen looks smaller or larger depending on resolution - no problem for Sikuli to find it with the same 64x64 image, as long as the rendering process does not change the pixels in the icon). This is sometimes a problem with the same webpage in different browsers: the same image is rendered slightly different depending on how it comes into existence. 
If images are slightly different in different environments (as long as the size in pixels stay the same), you can workaround this problem with Sikuli, if you check the smallest possible region, where the probe should be and accept a lower similarity as proof for existence.
If this is not possible, you need a different set of probe images for every scenario.

--- conclusion
If Sikuli does not meet your expectations, then it does not make sense, to invest in cv::matchTemplate, since Sikuli uses it in an optimum way.

The mentioned problems with resolution and color scheme changes can only
be solved with adequate pre- and post-imageprocessing and not by using
cv::matchTemplate in other ways, than Sikuli uses it.

-- 
You received this question notification because you are a member of
Sikuli Drivers, which is an answer contact for Sikuli.