Definition
Computer use is the ability of an AI model to operate a graphical interface the way a person does: it looks at screenshots, decides on an action, and returns mouse clicks, scrolling and keystrokes for a program to carry out. It lets AI agents work in software and websites that offer no API, and it is the basis of the browser agents that now visit websites on users' behalf.
The mechanics are a loop. The application sends the model a task and a screenshot; the model replies with one or more actions, such as 'click at these coordinates' or 'type this text'; the application executes them in a browser, virtual machine or desktop and sends back a new screenshot. The loop continues until the task is done or a person needs to step in. The model never touches the machine directly, which is why developers run it in sandboxed environments.
Anthropic introduced computer use as a public beta on October 22, 2024, with an upgraded Claude 3.5 Sonnet, and called it 'at times cumbersome and error-prone'. On the OSWorld benchmark at the time, Claude scored 14.9% in the screenshot-only category. The capability has since moved out of beta: Anthropic's computer use tool became generally available on the Claude API in August 2026 as a toolset with batch actions and zoom, alongside a separate browser toolset, and since March 2026 Pro and Max users can let Claude use their own computer in Cowork and Claude Code, as a research preview.
Other providers followed. OpenAI launched Operator in January 2025, folded it into ChatGPT agent in July 2025 and shut the standalone Operator site on August 31, 2025. ChatGPT agent has since been retired too: OpenAI's help center now points users to ChatGPT Work, launched in July 2026, which runs tasks in a cloud browser on a remote computer and uses computer use in the desktop app. OpenAI's API offers a computer tool for GPT-6 models, though OpenAI now recommends that GPT-6 Astra write Playwright or PyAutoGUI code instead of returning individual clicks. Google offers computer use for browser, Android and desktop environments in Gemini models; its earlier dedicated Gemini 2.5 Computer Use model has been shut down, and the capability is still labeled Preview.
Computer use differs from function calling, where a model calls a defined API with structured arguments. Function calling is faster and more reliable when an API exists; computer use covers everything else. It also differs from the fetching most AI search engines do, which downloads a page's HTML without rendering or interacting with it.
The main risks are reliability, speed and security. Each step needs a screenshot and a model call, layouts change, and pages can contain hidden instructions that try to redirect the agent (prompt injection). Anthropic, OpenAI and Google all add classifiers or confirmation steps for sensitive actions such as purchases or sending messages.
For website owners, computer use means some visitors are now agents completing tasks: comparing products, filling forms, booking or buying. Those agents rely on what they can see and operate. Clear labels, visible prices and stock, standard form controls, and checkout flows that do not depend on hover effects or fragile pop-ups make a site easier for them to complete, and bot protection that blocks all automated browsers will also block them. Verification standards such as Web Bot Auth let some agents identify themselves. See our guide on preparing your site for AI agents and the related shift toward agentic commerce.
Examples of Computer Use
- A finance team uses a Claude agent with the computer use toolset to copy invoice data from a legacy accounting system with no API into a spreadsheet, running it in an isolated virtual machine with approval required before any submission.
- A retailer reviews checkout recordings and finds that a cloud browser agent repeatedly fails at a size selector that only appears on hover, then replaces it with a standard dropdown.
- A QA team runs nightly computer use tests that sign up, add items to a cart and check out, flagging steps where the agent needs more than three attempts.
- A travel site sees a rise in automated sessions completing searches and holds, and adds clear confirmation pages so agents and their users can verify the booking details.
