PowerKeys Scripting API
    Preparing search index...

    Interface ScreenAPI

    interface ScreenAPI {
        GetPixelColor(x: number, y: number): string;
        FindImage(
            imagePath: string,
            threshold?: number,
            region?: ScreenSearchRegion | null,
            options?: ImageSearchOptions | null,
        ): Promise<ImageMatchResult | null>;
        DebugCapture(): Promise<string>;
        FindImageBase64(
            base64Data: string,
            threshold?: number,
            region?: ScreenSearchRegion | null,
            options?: ImageSearchOptions | null,
        ): Promise<ImageMatchResult | null>;
    }
    Index
    • Returns one physical virtual-screen pixel as an uppercase RGB hex string.

      Parameters

      • x: number

        Absolute pixel X coordinate.

      • y: number

        Absolute pixel Y coordinate.

      Returns string

      Hex color string, e.g. "#FF0000" for red.

      Coordinates use the same signed physical-pixel space as Mouse, Window, and overlays. Monitors left of or above the primary monitor have negative coordinates. Throws a runtime error when Windows cannot read the pixel, including coordinates outside the virtual screen.

      const color = Screen.GetPixelColor(100, 200);
      if (color === "#FF0000") Console.Log("red pixel");

      screen

    • Searches the screen for a template image loaded from the script asset sandbox or an explicitly user-selected file path. Scoring is zero-mean normalized cross-correlation under the template's alpha mask: transparent pixels are ignored, partially transparent pixels contribute proportionally, and the winning position is additionally verified in color and absolute brightness unless options.matchColor is false.

      Parameters

      • imagePath: string

        Script asset path, or an absolute path selected by a UI file/directory picker.

      • Optionalthreshold: number

        Minimum final score to consider a match, between 0 and 1. Defaults to 0.8.

      • Optionalregion: ScreenSearchRegion | null

        Non-empty search region in signed physical virtual-screen coordinates and extents. A WindowInfo.bounds value is accepted when its measured width and height are positive. Width multiplied by height may not exceed 67,108,864 pixels. Omit or pass null to search all monitors; the same capture-size limit applies to the full virtual screen.

      • Optionaloptions: ImageSearchOptions | null

        Search tuning: sizeTolerance, rotationTolerance, and matchColor. Omit or pass null for the default exact-pose, color-verified search. Unknown fields are rejected.

      Returns Promise<ImageMatchResult | null>

      Resolves with an ImageMatchResult, or null if no match meets the threshold.

      ALWAYS check for null before accessing .x or .y. The return value is null when no match meets the threshold. SCORES: a pixel-faithful template scores 0.9 or higher at its true location, while absent or unrelated content stays around 0.7 or lower, so the 0.8 default separates them. A template captured from a different rendering of a live scene (for example a tooltip over changing 3D content baked into the crop) can legitimately score 0.6-0.75; either lower threshold for that template or crop it from the same visual state it must match. Prefer masking changing backdrops as transparent pixels instead of including them. A template whose visible (nonzero-alpha) area is smaller than 16 pixels throws, and a near-uniform template matches by absolute color difference because correlation is undefined without texture. Unknown fields and out-of-range values in options throw. imagePath values are resolved canonically. Paths inside the script asset sandbox are accepted, and .. escapes are denied. Absolute paths outside the sandbox are accepted only when the path was selected by the user through UI.AddFilePicker or is inside a directory selected through UI.AddDirectoryPicker. That grant is scoped to this script, persists only on the computer where the picker was used, and survives local restarts; synced picker values never grant another installation or script. Use FindImageBase64 when embedding a template directly in a script. USE CASE: Pass a region from Window.Find() bounds to restrict the search area; this is significantly faster on multi-monitor setups and strongly recommended when polling in a loop or when using tolerance options. DPI CONTRACT: The captured screen and template are compared as physical image pixels. The template's decoded pixel dimensions are used as-is; PowerKeys does not scale it for monitor DPI. Capture the template at the same rendered scale as the target, or set options.sizeTolerance for small size differences. Returned x/y are the match center, not its top-left corner. The decoded template and captured search area are each limited to 67,108,864 pixels.

      const template = UI.AddFilePicker("save_button", "Save button template", { filters: ["png"] });
      const match = await Screen.FindImage(template.GetValue());
      if (match) {
      await Mouse.Move(match.x, match.y);
      await Mouse.Click();
      }

      // Restrict search to a specific window's area
      const [win] = Window.Find({ process: "chrome.exe" });
      const match2 = await Screen.FindImage(template.GetValue(), 0.9, win.bounds);

      // Tolerate a button that animates in slightly scaled and tilted
      const match3 = await Screen.FindImage(template.GetValue(), 0.8, win.bounds, {
      sizeTolerance: 0.1,
      rotationTolerance: 5,
      });

      screen

    • Captures the current screen and saves it to the debug directory. Useful for diagnosing why FindImage cannot find a template. The complete virtual screen may not exceed 67,108,864 pixels; a larger capture throws.

      Returns Promise<string>

      Resolves with the file path where the screenshot was saved.

      const path = await Screen.DebugCapture();
      Console.Log("Saved to: " + path);

      screen

    • Searches the screen for a template image provided as Base64-encoded PNG data. Scoring is zero-mean normalized cross-correlation under the template's alpha mask: transparent pixels are ignored, partially transparent pixels contribute proportionally, and the winning position is additionally verified in color and absolute brightness unless options.matchColor is false.

      Parameters

      • base64Data: string

        Base64-encoded PNG image data with transparent background.

      • Optionalthreshold: number

        Minimum final score to consider a match, between 0 and 1. Defaults to 0.8.

      • Optionalregion: ScreenSearchRegion | null

        Non-empty search region in signed physical virtual-screen coordinates and extents. A WindowInfo.bounds value is accepted when its measured width and height are positive. Width multiplied by height may not exceed 67,108,864 pixels. Omit or pass null to search all monitors; the same capture-size limit applies to the full virtual screen.

      • Optionaloptions: ImageSearchOptions | null

        Search tuning: sizeTolerance, rotationTolerance, and matchColor. Omit or pass null for the default exact-pose, color-verified search. Unknown fields are rejected.

      Returns Promise<ImageMatchResult | null>

      Resolves with an ImageMatchResult, or null if no match meets the threshold.

      ALWAYS check for null before accessing .x or .y. SCORES: a pixel-faithful template scores 0.9 or higher at its true location, while absent or unrelated content stays around 0.7 or lower. Invalid Base64 or undecodable image data throws, a template whose visible (nonzero-alpha) area is smaller than 16 pixels throws, and unknown fields or out-of-range values in options throw. USE CASE: Embed template images directly in the script without relying on external files. Encode the PNG to Base64 once and store as a constant. DPI CONTRACT: The template's decoded pixel dimensions are compared directly with physical screen pixels; no monitor-DPI scaling is applied. Capture the template at the same rendered scale as the target, or set options.sizeTolerance for small size differences. Returned x/y are the match center in signed physical virtual-screen coordinates. The decoded template and captured search area are each limited to 67,108,864 pixels.

      const B64_ICON = "iVBORw0KGgo..."; // your Base64 PNG
      const match = await Screen.FindImageBase64(B64_ICON, 0.85);
      if (match) {
      await Mouse.Move(match.x, match.y);
      await Mouse.Click();
      }

      screen