sikuli-driver team mailing list archive

[Chris Rhoads <question621227@xxxxxxxxxxxxxxxxxxxxx>]

New question #621227 on Sikuli:
https://answers.launchpad.net/sikuli/+question/621227

I'm having issues with sikuli matching images.
I'm writing a selenium script in Java with sikuli-api 1.2.0 that logs into a remote pc program, finds a terminal by image of its terminal ID, clicks it and then downloads a client zip file.

It mostly works except sikuli is only able to recognize and act on 1 image and Ive got about 30 I need to use.

The one image that it recognizes was taken with Nimbus screenshot tool, the rest were taken with the native Mac OSX screen shot tool.

wsRibId gets recognized - taken with Nimbus
lmiDownloadBarIcon - not recognized taken with native MAC tool

Both are in the same folder within the project.

Ive tried with and without similarity with their pattern objects....

Any help is greatly appreciated!


Code is below. 

import junit.framework.Assert;
import org.openqa.selenium.By;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;
import org.sikuli.basics.Debug;
import org.sikuli.script.*;

import javax.swing.*;
import java.awt.event.KeyEvent;

import static org.sikuli.script.Commands.click;

/**
 * Created by chrisrhoads on 4/11/17.
 *
 * Point Of Sale Terminal (POS) Remote Rib Visual Testing
 * Powered By Sikuli API -- www.sikuli.org
 *
 */
public class Main {

    //Start of Client download images
    private static final Pattern lmiDownloadBarIcon = new Pattern(Main.class.getResource("LogMeInDownloadBarIcon.png")).similar((float) 0.75);
    private static final Pattern lmiDownloadZip = new Pattern(Main.class.getResource("LogMeInZip.png")).similar((float) 0.75);
    private static final Pattern lmiClientLaunch = new Pattern(Main.class.getResource("LMIClientIcon.png")).similar((float) 0.75);
    private static final Pattern osxOpenConfirmation = new Pattern(Main.class.getResource("osxOpenConfirmation.png")).similar((float) 0.75);

    //End of Client download images




    //Start of Remote RIB Exit Objects
    private static final Pattern ribSignOut = new Pattern(Main.class.getResource("RIB_Client_SignOutBtn.png")); //shows up after hitting Ctrl+Alt+Delete with active sign in to VM
    private static final Pattern clientXit = new Pattern(Main.class.getResource("RIB_Client_Window_Exit.png"));
//    private static final Pattern clientXitConfirm = new Pattern(Main.class.getResource("RIB_Client_Window_Exit_Confirm"));


    //End of Remote RIB Exit Objects

    private static final Pattern sideBarAd = new Pattern(Main.class.getResource("sideBarAd.png"));


    public static void main(String[] args) throws FindFailed, InterruptedException {
        Screen s = new Screen();
        Debug.on(3);
        WebDriver driver = new ChromeDriver();

        driver.navigate().to("https://www.logmein.com";);
        driver.manage().window().maximize();
        WebDriverWait wait = new WebDriverWait(driver, 10);





        driver.findElement(By.linkText("Log In")).click();

        //Thread.sleep(3000);
        System.out.println("Login Button #1 was just clicked.");
        wait.until(ExpectedConditions.visibilityOfElementLocated(By.cssSelector("#email")));
        

        WebElement email = driver.findElement(By.cssSelector("#email"));
        WebElement password = driver.findElement(By.cssSelector("#password"));
        WebElement logInBtn = driver.findElement(By.cssSelector("#btnSubmit"));


//login creds are not legit for sake of public posting

        email.click();
        email.clear();
        email.sendKeys("chris.rhoads@xxxxxxxxxxxxx");

        password.click();
        password.clear();
        password.sendKeys("password");

        logInBtn.click();
    
        try
        {
            //wsLogInBtn1 = new Pattern();

            s.wait(wsRibId,30);
            s.click(wsRibId);
            System.out.println("First Attempt To Find Image.");
        }
        catch(FindFailed f)
        {
            System.out.println("Exception In First Attempt: " +f.getMessage());
            Assert.fail("Image wasn't found. Please use correct image.");
        }

        Thread.sleep(1000);
        System.out.println("Finding Remote RIB.");


        //In case image/object wasn't clicked in first attempt and cursor stays in the same screen, then do second attempt.
        if(s.exists(wsRibId) != null)
        {
            try
            {
                s.getLastMatch().doubleClick(wsRibId);
                System.out.println("Second Attempt To Find Image.");
                //System.out.println("Object: " +imageLocation + " is clicked successfully.");
            }
            catch(FindFailed f)
            {
                System.out.println("Exception In Second Attempt: " +f.getMessage());
                //System.out.println("FindFailed Exception Handled By Method: ClickObjectUsingSikuli. Please check image being used to identify the webelement. supplied image: " +imageLocation);
            }
        }



//****************************FAILURE POINT*************************************************************
At this point the zip file will be visible in the lower dock of the Chrome Window, which is where the script fails.
Ive tried with several other images as well, using keyEvents to tab to the downloads folder and then look for the 
FileName.zip and click or double click on it, I cant get anything to locate.

I had to switch to the mac screenshot tool in orfer to capture outside a web page but many tutorials I have seen
using snippet and navigated Windows file structure based purely on images and it works for them.

//****************************FAILURE POINT*************************************************************

    try
        {
            //wsLogInBtn1 = new Pattern();
            s.wait(lmiDownloadBarIcon,30);
            s.click(lmiDownloadBarIcon);
            System.out.println("First Attempt To Find Image.");
        }
        catch(FindFailed f)
        {
            System.out.println("Exception In First Attempt: " +f.getMessage());
            //System.out.println("FindFailed Exception Handled By Method: ClickObjectUsingSikuli. Please check image being used to identify the webelement. supplied image: " +imageLocation);
            Assert.fail("Image wasn't found. Please use correct image.");
        }

        Thread.sleep(1000);
        System.out.println("Finding Remote RIB.");
        if(s.exists(lmiDownloadBarIcon) != null)
        {
            try
            {
                s.getLastMatch().doubleClick(lmiDownloadBarIcon);
                System.out.println("Second Attempt To Find Image.");
                //System.out.println("Object: " +imageLocation + " is clicked successfully.");
            }
            catch(FindFailed f)
            {
                System.out.println("Exception In Second Attempt: " +f.getMessage());
                //System.out.println("FindFailed Exception Handled By Method: ClickObjectUsingSikuli. Please check image being used to identify the webelement. supplied image: " +imageLocation);
            }
        }



//************************CONSOLE OUTPUT********************************
[debug] Region: wait: waiting 30.0 secs for /Users/chrisrhoads/IdeaProjects/SikuliScript_POS/target/classes/LogMeIn_Computers_RibIdNum.png to appear in S(0)[0,0 1680x1050]
[debug] Region: wait: /Users/chrisrhoads/IdeaProjects/SikuliScript_POS/target/classes/LogMeIn_Computers_RibIdNum.png appeared (M[317,747 86x14]@S(S(0)[0,0 1680x1050]) S:0.98 C:360,754 [1090 msec])
[debug] Region: wait: waiting 3.0 secs for /Users/chrisrhoads/IdeaProjects/SikuliScript_POS/target/classes/LogMeIn_Computers_RibIdNum.png to appear in S(0)[0,0 1680x1050]
[debug] Region: checkLastSeen: still there
[debug] Region: wait: /Users/chrisrhoads/IdeaProjects/SikuliScript_POS/target/classes/LogMeIn_Computers_RibIdNum.png appeared (M[317,747 86x14]@S(S(0)[0,0 1680x1050]) S:0.98 C:360,754 [125 msec])
[debug] Region: init: (0, 0, 5, 5)
[debug] CLICK on L(360,754)@S(0)[0,0 1680x1050] (805 msec)
First Attempt To Find Image.
Finding Remote RIB.
[debug] Region: exists: waiting 3.0 secs for /Users/chrisrhoads/IdeaProjects/SikuliScript_POS/target/classes/LogMeIn_Computers_RibIdNum.png to appear in S(0)[0,0 1680x1050]
[debug] Region: checkLastSeen: still there
[debug] Region: exists: /Users/chrisrhoads/IdeaProjects/SikuliScript_POS/target/classes/LogMeIn_Computers_RibIdNum.png has appeared (M[317,747 86x14]@S(S(0)[0,0 1680x1050]) S:0.98 C:360,754 [133 msec])
[debug] Region: wait: waiting 3.0 secs for /Users/chrisrhoads/IdeaProjects/SikuliScript_POS/target/classes/LogMeIn_Computers_RibIdNum.png to appear in M[317,747 86x14]@S(0)
[debug] Region: checkLastSeen: still there
[debug] Region: wait: /Users/chrisrhoads/IdeaProjects/SikuliScript_POS/target/classes/LogMeIn_Computers_RibIdNum.png appeared (M[317,747 86x14]@S(S(0)[0,0 1680x1050]) S:0.98 C:360,754 [5 msec])
[debug] DOUBLE CLICK on L(360,754)@S(0)[0,0 1680x1050] (695 msec)
Second Attempt To Find Image.
Logging in to remote RIB Console
Remote Client Software Download started.
Sending Keystrokes to try to access the downloads folder
[debug] Region: wait: waiting 30.0 secs for /Users/chrisrhoads/IdeaProjects/SikuliScript_POS/target/classes/LogMeInDownloadBarIcon.png to appear in S(0)[0,0 1680x1050]
[debug] Region: handleFindFailed: P(/Users/chrisrhoads/IdeaProjects/SikuliScript_POS/target/classes/LogMeInDownloadBarIcon.png) S: 0.5
[debug] Region: wait: /Users/chrisrhoads/IdeaProjects/SikuliScript_POS/target/classes/LogMeInDownloadBarIcon.png did not appear [30205 msec]
Exception In First Attempt: /Users/chrisrhoads/IdeaProjects/SikuliScript_POS/target/classes/LogMeInDownloadBarIcon.png: (272x48) in S(0)[0,0 1680x1050] E:Y, T:3.0
Exception in thread "main" junit.framework.AssertionFailedError: Image wasn't found. Please use correct image.
	at junit.framework.Assert.fail(Assert.java:47)
	at Main.main(Main.java:228)
[debug] RunTimeAPI: final cleanup
[debug] FileManager: deleteFileOrFolder:
/var/folders/17/wvgjzfkd06z_qp2gmzfwfvgh0000gn/T/Sikulix_201706522

Process finished with exit code 1


-- 
You received this question notification because your team Sikuli Drivers
is an answer contact for Sikuli.