Interface ScreenAPI

interface ScreenAPI {
    GetPixelColor(x: number, y: number): string;
    FindImage(
        imagePath: string,
        threshold?: number,
        region?: WindowBounds,
    ): ImageMatchResult;
    FindImageBase64(
        base64Data: string,
        threshold?: number,
        region?: WindowBounds,
    ): ImageMatchResult;
}

Methods

GetPixelColor FindImage FindImageBase64

Methods

GetPixelColor

  • GetPixelColor(x: number, y: number): string

    Returns the color of a single pixel on screen as a hex string.

    Parameters

    • x: number

      Pixel X coordinate.

    • y: number

      Pixel Y coordinate.

    Returns string

    Hex color string, e.g. "#FF0000" for red.

    Example

    const color = Screen.GetPixelColor(100, 200);
    if (color === "#FF0000") Console.Log("red pixel");

FindImage

  • FindImage(
        imagePath: string,
        threshold?: number,
        region?: WindowBounds,
    ): ImageMatchResult

    Searches the screen for a template image loaded from a file path. Uses masked normalized cross-correlation: transparent pixels in the template PNG are ignored, so only opaque regions (text, borders) contribute to the match score.

    Parameters

    • imagePath: string

      Absolute path to a PNG template with transparent background.

    • Optionalthreshold: number

      Minimum correlation score to consider a match, between 0 and 1. Defaults to 0.8.

    • Optionalregion: WindowBounds

      Search region { x, y, width, height } in virtual screen coordinates. Omit to search all monitors.

    Returns ImageMatchResult

    An ImageMatchResult with the center coordinates and score, or null if no match meets the threshold.

    Remarks

    ALWAYS check for null before accessing .x or .y — the return value is null when no match meets the threshold. USE CASE: Pass a region from Window.Find() bounds to restrict the search area; this is significantly faster on multi-monitor setups.

    Example

    const match = Screen.FindImage("C:\\templates\\button.png");
    if (match) {
    Mouse.Move(match.x, match.y);
    Mouse.Click();
    }

    // Restrict search to a specific window's area
    const [win] = Window.Find({ process: "chrome.exe" });
    const match2 = Screen.FindImage("C:\\tpl.png", 0.9, win.bounds);

FindImageBase64

  • FindImageBase64(
        base64Data: string,
        threshold?: number,
        region?: WindowBounds,
    ): ImageMatchResult

    Searches the screen for a template image provided as Base64-encoded PNG data. Transparent pixels are masked out; only opaque regions contribute to the score.

    Parameters

    • base64Data: string

      Base64-encoded PNG image data with transparent background.

    • Optionalthreshold: number

      Minimum correlation score to consider a match, between 0 and 1. Defaults to 0.8.

    • Optionalregion: WindowBounds

      Search region { x, y, width, height } in virtual screen coordinates. Omit to search all monitors.

    Returns ImageMatchResult

    An ImageMatchResult with the center coordinates and score, or null if no match meets the threshold.

    Remarks

    ALWAYS check for null before accessing .x or .y. USE CASE: Embed template images directly in the script without relying on external files — encode the PNG to Base64 once and store as a constant.

    Example

    const B64_ICON = "iVBORw0KGgo..."; // your Base64 PNG
    const match = Screen.FindImageBase64(B64_ICON, 0.85);
    if (match) {
    Mouse.Move(match.x, match.y);
    Mouse.Click();
    }

Settings

Member Visibility

On This Page

Methods