sikuli-driver team mailing list archive

[RaiMan <rmhdevelop@xxxxxx>]

Public bug reported:

complete info in related question
------------------------------------------------

Has anyone thought about or be interested in using/working with Sikuli
via a WebDriver API (e.g. the JSONWireProtocol)?

I don't think such a project exists yet, and I had plans to work on a
proof of concept, but never got around to it yet.

Basically, we would wrap the Sikuli (Java) API around the WebDriver API
such that any WebDriver language binding can invoke Sikuli.

In general we can map the click and sendKeys() methods and whatever else
is applicable, and we would only support a subset of the WebDriver API.

Find elements would be based on captured saved (PNG) images for Sikuli
to use and be based on the following WebDriver-based location
strategies:

By name - name of PNG image to find, stored in some default location we
define globally

By ID - optional to decide how to implement, could also be alias to by
name

By XPath - absolute (or relative) path to image in filesystem (probably
no validation of the "XPath", we just check that the filepath exists. In
essence, we override XPath to mean path to file system rather than
really XPath

By CSS selector (or any other unused strategy) - override it as a way to
send base64 encoded representation of a PNG image. For use in remote
WebDriver / Grid type test deployment where you can send a file locally
(read bytes into memory and encode as base64) to the server (which then
decodes the base64, saves to temp location, and uses it with Sikuli to
locate the "element"). In essence, this would be similar to WebDriver's
way of handling file uploads (via sendKeys) and FirefoxProfiles for
remote WebDriver/Grid deployments where the file/profile need not reside
on the actual node but can be from the machine sending the WebDriver
commands.

I think this type of solution offers a good way to integrate Sikuli with
Selenium/WebDriver/Appium/ios-driver tools for a system integration type
test or where those tools are lacking that Sikuli complements. Something
more powerful and flexible than (Firefox) SikuliWebDriver
(https://code.google.com/p/sikuli-api/wiki/SikuliWebDriver) and which
would work over local/remote/grid deployments.

This would be in essence, building a WebDriver server interface to
Sikuli so that WebDriver clients can call Sikuli as if it was a
WebDriver with minimal code changes on the client side to support.

Let me know what you think of this.

** Affects: sikuli
     Importance: Low
     Assignee: RaiMan (raimund-hocke)
         Status: In Progress

** Changed in: sikuli
       Status: New => In Progress

** Changed in: sikuli
   Importance: Undecided => Low

** Changed in: sikuli
     Assignee: (unassigned) => RaiMan (raimund-hocke)

** Changed in: sikuli
    Milestone: None => 2.0.0

** Description changed:

+ complete info in related question
+ ------------------------------------------------
+ 
  Has anyone thought about or be interested in using/working with Sikuli
  via a WebDriver API (e.g. the JSONWireProtocol)?
  
  I don't think such a project exists yet, and I had plans to work on a
  proof of concept, but never got around to it yet.
  
  Basically, we would wrap the Sikuli (Java) API around the WebDriver API
  such that any WebDriver language binding can invoke Sikuli.
  
  In general we can map the click and sendKeys() methods and whatever else
  is applicable, and we would only support a subset of the WebDriver API.
  
  Find elements would be based on captured saved (PNG) images for Sikuli
  to use and be based on the following WebDriver-based location
  strategies:
  
  By name - name of PNG image to find, stored in some default location we
  define globally
  
  By ID - optional to decide how to implement, could also be alias to by
  name
  
  By XPath - absolute (or relative) path to image in filesystem (probably
  no validation of the "XPath", we just check that the filepath exists. In
  essence, we override XPath to mean path to file system rather than
  really XPath
  
  By CSS selector (or any other unused strategy) - override it as a way to
  send base64 encoded representation of a PNG image. For use in remote
  WebDriver / Grid type test deployment where you can send a file locally
  (read bytes into memory and encode as base64) to the server (which then
  decodes the base64, saves to temp location, and uses it with Sikuli to
  locate the "element"). In essence, this would be similar to WebDriver's
  way of handling file uploads (via sendKeys) and FirefoxProfiles for
  remote WebDriver/Grid deployments where the file/profile need not reside
  on the actual node but can be from the machine sending the WebDriver
  commands.
  
  I think this type of solution offers a good way to integrate Sikuli with
  Selenium/WebDriver/Appium/ios-driver tools for a system integration type
  test or where those tools are lacking that Sikuli complements. Something
  more powerful and flexible than (Firefox) SikuliWebDriver
  (https://code.google.com/p/sikuli-api/wiki/SikuliWebDriver) and which
  would work over local/remote/grid deployments.
  
  This would be in essence, building a WebDriver server interface to
  Sikuli so that WebDriver clients can call Sikuli as if it was a
  WebDriver with minimal code changes on the client side to support.
  
  Let me know what you think of this.

-- 
You received this bug notification because you are a member of Sikuli
Drivers, which is subscribed to Sikuli.
https://bugs.launchpad.net/bugs/1412323

Title:
  [research] Sikuli over WebDriver API / JSONWireProtocol?

Status in Sikuli:
  In Progress

Bug description:
  complete info in related question
  ------------------------------------------------

  Has anyone thought about or be interested in using/working with Sikuli
  via a WebDriver API (e.g. the JSONWireProtocol)?

  I don't think such a project exists yet, and I had plans to work on a
  proof of concept, but never got around to it yet.

  Basically, we would wrap the Sikuli (Java) API around the WebDriver
  API such that any WebDriver language binding can invoke Sikuli.

  In general we can map the click and sendKeys() methods and whatever
  else is applicable, and we would only support a subset of the
  WebDriver API.

  Find elements would be based on captured saved (PNG) images for Sikuli
  to use and be based on the following WebDriver-based location
  strategies:

  By name - name of PNG image to find, stored in some default location
  we define globally

  By ID - optional to decide how to implement, could also be alias to by
  name

  By XPath - absolute (or relative) path to image in filesystem
  (probably no validation of the "XPath", we just check that the
  filepath exists. In essence, we override XPath to mean path to file
  system rather than really XPath

  By CSS selector (or any other unused strategy) - override it as a way
  to send base64 encoded representation of a PNG image. For use in
  remote WebDriver / Grid type test deployment where you can send a file
  locally (read bytes into memory and encode as base64) to the server
  (which then decodes the base64, saves to temp location, and uses it
  with Sikuli to locate the "element"). In essence, this would be
  similar to WebDriver's way of handling file uploads (via sendKeys) and
  FirefoxProfiles for remote WebDriver/Grid deployments where the
  file/profile need not reside on the actual node but can be from the
  machine sending the WebDriver commands.

  I think this type of solution offers a good way to integrate Sikuli
  with Selenium/WebDriver/Appium/ios-driver tools for a system
  integration type test or where those tools are lacking that Sikuli
  complements. Something more powerful and flexible than (Firefox)
  SikuliWebDriver (https://code.google.com/p/sikuli-
  api/wiki/SikuliWebDriver) and which would work over local/remote/grid
  deployments.

  This would be in essence, building a WebDriver server interface to
  Sikuli so that WebDriver clients can call Sikuli as if it was a
  WebDriver with minimal code changes on the client side to support.

  Let me know what you think of this.

To manage notifications about this bug go to:
https://bugs.launchpad.net/sikuli/+bug/1412323/+subscriptions