sikuli-driver team mailing list archive

[Steven Sheldon <question822175@xxxxxxxxxxxxxxxxxxxxx>]

New question #822175 on SikuliX:
https://answers.launchpad.net/sikuli/+question/822175

So I have been reading some previous answers concerning resolution and scaling.

I have a desktop application that I would like to create automated testing for, whereby a series of buttons are clicked in the software and then I look for an expected graphical output.

I'll be checking to find the buttons, examine the dialogs at each step of the command, and then the final graphical output.  I have written a sikulix script that does that.

Our in-house AI says that to accommodate different resolutions we could do something like the code below, which I have not tried to figure out yet.

My question is, do we have to code around the resolution/scaling problem, or does Sikulix yet have the ability to just "handle it"?

I *really* don't want to have to create separate image examples for every environment, as some of the Answers suggested.  That's kind of a brute-force method but requires me to basically do image capture for every possible resolution and scaling possibility.


---------------------------
AI Generated code that supposedly accounts for different resolution and scaling (untested):


class AdaptivePattern:
    def __init__(self, image_path):
        self.image_path = image_path
        self.base_resolution = 1920  # Base resolution width for reference
        self.scale_factors = [1.0, 1.25, 1.5, 0.75, 2.0]
        self.similarity = 0.85
        
    def get_resolution_scale(self):
        # Get current screen resolution
        current_width = Screen().getW()
        # Calculate scale based on resolution difference
        resolution_scale = current_width / self.base_resolution
        return resolution_scale
        
    def click(self):
        resolution_scale = self.get_resolution_scale()
        
        # Adjust scale factors based on resolution
        adjusted_scales = [s * resolution_scale for s in self.scale_factors]
        
        # Log resolution information
        print(f"Current resolution scale: {resolution_scale}")
        print(f"Screen width: {Screen().getW()}")
        print(f"Trying scales: {adjusted_scales}")
        
        for scale in adjusted_scales:
            try:
                pattern = Pattern(self.image_path)\
                    .resize(scale)\
                    .similar(self.similarity)
                click(pattern)
                print(f"Successfully clicked at scale {scale}")
                return True
            except FindFailed:
                continue
            except Exception as e:
                print(f"Error with scale {scale}: {str(e)}")
                continue
                
        raise Exception(f"Failed to find {self.image_path} at any scaling")

    def exists(self, timeout=0):
        resolution_scale = self.get_resolution_scale()
        adjusted_scales = [s * resolution_scale for s in self.scale_factors]
        
        for scale in adjusted_scales:
            try:
                pattern = Pattern(self.image_path)\
                    .resize(scale)\
                    .similar(self.similarity)
                return exists(pattern, timeout)
            except:
                continue
        return False

# Usage example
try:
    # Create adaptive pattern
    button = AdaptivePattern("button.png")
    
    # Check if exists first (optional)
    if button.exists(10):  # Wait up to 10 seconds
        button.click()
    else:
        print("Button not found after timeout")
        
except Exception as e:
    print(f"Error: {str(e)}")

-- 
You received this question notification because your team Sikuli Drivers
is an answer contact for SikuliX.