sikuli-driver team mailing list archive

[RaiMan <question678731@xxxxxxxxxxxxxxxxxxxxx>]

Question #678731 on Sikuli changed:
https://answers.launchpad.net/sikuli/+question/678731

    Status: Open => Answered

RaiMan proposed the following answer:
The challenges are:

1. moving images in 2D

2. images in 3D (in your case not real 3D, but some kind of perspective
view mapped into 2D)

3. moving images in 3D

-- at 1: The implementation depends on wether you know the direction of
movement or not.

If you know it, then you can wait in appropriate places, where the image
should appear somewhen in the future.

If you do not know it, you have to watch for changes in direct
surrounding of the actual place of the object. In most cases this will
allow to calculate the direction of movement. Then you have to follow
the movement again depending on, wether the object moves constantly in
one direction until something happens or wether it might change the
direction at any time without an obvious reason.

All this can be done with the standard features of SikuliX, but might
end up in time consuming actions, that have to be optimised if possible,
so the bot is always faster than the game.

at 2: The problem here is the perspective: size and shape of an image
depend on where it is in the scene. The perspective view has some
internal rules about size and distortion of an image at a specific
point.

Image detection in all places of such a scenario is currently not
possible, since you would have to adjust size and shape of the given
image according to the rules of the view, even if you only wanted to
look for the image in one exact place. Such features are currently not
available in SikuliX (apart from a resize feature).

at 3: does not make sense to talk about that in the moment regarding
SikuliX

... But SikuliX is an example of making features of OpenCV available at
a higher level (image search and change detection). SikuliX uses the
Java API of OpenCV, but this is also available for Python based
environments.

... and OpenCV has everything, to address the mentioned scenarios.

So if you really want to move to scenarios 2 or even 3, you have to leave SikuliX behind and dive into Python+OpenCV (which is a bit easier than Java+OpenCV).
There are tons of examples in the net .

But feel free to use the SikuliX sources as a base and add your features
based on OpenCV features not yet used in SikuliX.

You received this question notification because your team Sikuli Drivers
is an answer contact for Sikuli.