sikuli-driver team mailing list archive

[RaiMan <question689217@xxxxxxxxxxxxxxxxxxxxx>]

Question #689217 on Sikuli changed:
https://answers.launchpad.net/sikuli/+question/689217

    Status: Open => Answered

RaiMan proposed the following answer:
SikuliX basically is a tool to find a specific given image (which
usually was captured before) on a screen or in another image (usually a
saved screenshot). The current implementation based on matchTemplate is
pixel-based, which means, that the given image must be visible in the
search area with the same size (width x height).  It is found, if all
pixels are the same (exact match). Besides that matchTemplate allows to
get matches with a lower confidence, which means, that not all pixels
are the same. The lower the confidence, the more pixels differ. It is up
to the user, to accept matches with lower confidence as matches (which
usually does not make sense with confidence values lower than 0.7 to
0.8).

This implementation has 2 main drawbacks:
- an image might not work in another environment, where it is not shown with the same size. 
- an image might contain too much background pixels, that vary over time or in other environments
- the environment (usually browser) might render the image differently as where it was captured (usually handling of edges)

For the first issue we can try with resize, for the others with shot-optimisation, masking and accepting lower confidence values.
And we can have extra shots for different environments.

If your application case and your usage of SikuliX fit, then the
matchTemplate based image search is precise.

It does not work reliably in situations of fast moving/changing objects
and it cannot be used, where objects are shown with 3D aspects.

The current implementation is sufficient for automating/testing of GUI
based applications and (though heavily used) has its limitations with
game bots (especially video-like games).

Conclusion: as long as the main target for SikuliX are GUI based
applications, then there is no need to use other techniques.

... but since the whole world of OpenCV is available in SikuliX (Java
level API), every user of SikuliX can implement own features relatively
easy.

-- 
You received this question notification because your team Sikuli Drivers
is an answer contact for Sikuli.