Automated clicks miss? Plan a coordinate-calibration test

Supply screenshot and viewport sizes for your test page to draft coordinate-conversion checks and known-button tests, not live-site actions.

Execution environment and usage

AI / software

Gemini

Where to operate

The Gemini web-chat input; this version requests a text draft.

Getting started

  1. Open an eligible account at gemini.google.com and start a chat without connecting external accounts.
  2. Use an owned, side-effect-free test page and record viewport and screenshot sizes.
  3. Request the draft/check and compare it with the original material. Suggestions are not evidence of execution.

Setup and plan requirements

Task inputs: Use an owned, side-effect-free test page and record viewport and screenshot sizes. Use an eligible tool that can read the supplied format; verify parsing, context limits and access before relying on it.

Prompt portability limits

The written request can be adapted to a chatbot that accepts the same input. It does not transfer the original agent, plugins or computer permissions.

Usage limitations

Supply viewport/screenshot sizes, scale and coordinate conventions; planning does not click or produce measured results.This independent pass reviewed local case records, without revalidating original sources or hands-on testing.

Environment sources

適用情境 / When to use

Supply screenshot and viewport sizes for your test page to draft coordinate-conversion checks and known-button tests, not live-site actions. Supply viewport/screenshot sizes, scale and coordinate conventions; planning does not click or produce measured results. Original case: He reports a score matching the requested click count.

步驟 / Steps

  1. Use an owned, side-effect-free test page and record viewport and screenshot sizes.
  2. Plan coordinate conversion and known-target tests, observing state after each click.
  3. Compare expected and actual targets and recheck after resizing.

提示詞 / Prompt

Plan coordinate tests for [owned test page], specifying [viewport/screenshot sizes], scaling and coordinate systems. Select positions and repetitions according to [test scope], checking each result and retesting after resizing. Produce a plan only; do not click production sites or bypass approvals.

留意事項 / Cautions

  • Preserve safety approvals rather than copying automatic acknowledgments from the demo.
  • Do not calibrate on checkout or production forms.

來源限制 / Evidence limits

One demonstration does not validate other scaling settings or sites; this site has not run it.

流程說明 / Workflow note

These steps and prompts are original, untested exercises by this site, not the author’s verbatim workflow. Reported results do not guarantee exercise outcomes.

原始工具組合 / Original tool stack

Gemini Computer Use; Java; Playwright

Source publication date: 2025-11-03

Source checked: 2026-10-20

Driving a web browser with Gemini’s Computer Use model in Java