---
title: "Computer Use Agent: What AI That Clicks and Types Can Do"
url: https://syntaxradar.com/radar/computer-use-ai-agents-operating-systems-claude-operator
author: "Abdul Rehman, Lead Systems Architect and Technical Director"
publishedAt: 2026-10-25T08:35:00.000Z
dateModified: 2026-10-29T10:33:58.900Z
category: "#DEV-SYSTEMS"
topic: "computer use agent"
primarySource: https://www.anthropic.com/news/3-5-models-and-computer-use
summary: "Computer use agent explained: how AI sees a screen, clicks and types, what OSWorld tests show about its limits, and the safety rules its makers recommend."
---


# Computer Use Agent: What AI That Clicks and Types Can Do

**Short answer:** Anthropic (October 2024) and OpenAI (January 2025) released AI agents that operate a computer by looking at screenshots, clicking and typing. A 2026 benchmark of long tasks shows they still finish only a fraction of them.

**Takeaway:** Computer use agents handle short, well-defined screen tasks but still fail most long workflows, so human oversight remains part of every serious deployment.

**Key figures (as reported by the source):**
- OpenAI CUA score on OSWorld: 38.1%
- Human score on OSWorld: 72.4%
- Best completion rate on OSWorld 2.0: 20.6%

A computer use agent is an AI system that operates a computer the way a person does: it looks at the screen, moves the cursor, clicks buttons and types text. Anthropic introduced the idea to developers in October 2024, and OpenAI followed in January 2025 with Operator. Both companies describe their systems as early and error-prone, and newer benchmark results back that up.

### What is a computer use agent?

It is software that takes a goal in plain language, such as "save a picture of a cat to my desktop", and carries it out by using a normal graphical interface. It does not need a special connection to the program it is using.

Anthropic's [October 2024 announcement](https://www.anthropic.com/news/3-5-models-and-computer-use) described its version as letting developers direct Claude to use computers "by looking at a screen, moving a cursor, clicking buttons, and typing text." The same post called the feature experimental and, at times, "cumbersome and error-prone."

OpenAI's version powers Operator, launched on January 23, 2025. According to [The Decoder's coverage](https://the-decoder.com/openais-operator-and-computer-using-agent-bring-autonomous-ai-agents-closer-to-reality/), it uses a model called Computer-Using Agent (CUA), which combines the vision of GPT-4o with reasoning skills trained through reinforcement learning (a method where a model improves by being rewarded for good outcomes). At launch it was open only to US ChatGPT Pro subscribers.

### How does it work?

It runs in a loop. It takes a screenshot, decides what to do next, performs one action, then takes another screenshot to see what changed.

OpenAI's model follows three phases at each step, as The Decoder describes them: capture a screenshot, reason about the next action, then act by clicking, scrolling or typing. It works from what is on the screen, without special integrations with the websites it visits.

Anthropic's [developer documentation](https://platform.claude.com/docs/en/docs/agents-and-tools/tool-use/computer-use-tool) shows the mechanics from the builder's side. The model asks for an action, such as a screenshot, a click, a key press or a scroll. The developer's own software carries it out inside an environment the developer controls, then sends the result back. The loop repeats until the model has no more actions to request.

Screenshots are the agent's only view of the world, and they carry a cost. The documentation says each one uses roughly 1,000 to 1,800 input tokens (the units an AI model reads and is billed by). Anthropic's advice for reliable runs is to keep tasks simple, add short pauses so the screen can update, and ask the model to check each step with a screenshot.

![A person at a wooden desk watching a laptop screen while a cursor moves across a web form](/uploads/blog/computer-use-ai-agents-operating-systems-claude-operator/gemini-supporting.jpg)

### How well do these agents actually perform?

The standard yardstick is OSWorld, a benchmark of real computer tasks run in a virtual machine (a simulated computer inside a computer). Humans completed 72.4% of its tasks, according to the comparison table in OpenAI's CUA announcement.

Anthropic reported that its upgraded Claude 3.5 Sonnet scored 14.9% on OSWorld in the screenshot-only category, against 7.8% for the next-best system. With more steps allowed, it reached 22.0%.

OpenAI's CUA then reported 38.1% on OSWorld, 58.1% on WebArena (simulated websites) and 87% on WebVoyager (live sites such as Amazon and Google Maps). These figures are from OpenAI. They show a strong result on simpler live-web tasks and a gap to humans on harder ones: the same table lists humans at 78.2% on WebArena, against the agent's 58.1%.

Scores have kept climbing since. A third-party [OSWorld leaderboard](https://leaderboard.steel.dev/leaderboards/osworld/) last updated in April 2026 lists Claude Opus 4.6 at 72.7%, roughly level with the human baseline. The leaderboard notes that self-reported rows are unaudited.

### Is it safe to let an AI control your computer?

Its makers say it needs guardrails. Anthropic's documentation recommends four precautions: run the agent in a dedicated virtual machine or container with minimal privileges, keep sensitive data and account logins away from it, limit its internet access to an approved list of domains, and require a human to confirm consequential actions such as accepting terms or making financial transactions.

The biggest named risk is prompt injection. That means hidden instructions placed in a web page or image that the model reads and may follow, even when they conflict with what the user asked. Anthropic says it scans screenshots with classifiers and has Claude check whether an instruction really came from the user when one is flagged. It still tells developers to apply the other precautions.

OpenAI describes its own controls for Operator. According to The Decoder, users must give permission before actions such as logging in or paying. A "watch mode" lets people observe every step on sensitive sites like email or banking. A separate monitoring model can halt tasks that look suspicious, and a detection system guards against malicious websites.

Anthropic's 2024 announcement added a broader worry: computer use may create a new route for threats such as spam, misinformation or fraud. The company said it built classifiers that can identify when computer use is being used and whether harm is occurring.

![A hand resting on a laptop trackpad beside a closed notebook in a quiet office](/uploads/blog/computer-use-ai-agents-operating-systems-claude-operator/gemini-supporting.jpg)

### What do the harder tests show about real office work?

The original OSWorld tasks are short. A newer benchmark, [OSWorld 2.0](https://arxiv.org/abs/2606.29537), was built to test long workflows, and its results are far lower.

The paper describes 108 long-horizon workflows. Each takes a person a median of about 1.6 hours. For Claude Opus 4.7 with maximum thinking, a task averaged about 318 tool calls, against about 30 in the first OSWorld.

The best result the authors report is Claude Opus 4.8, which completed 20.6% of tasks under the primary completion metric at 500 steps, with a 54.8% partial-credit score. GPT-5.5 was more economical with tokens but plateaued near 13%.

The authors conclude that current agents are "still far from professional-level computer use." They report that agents struggle most when a task depends on hidden information they must work out, and that they often lose track of constraints, miss details that appear mid-task, guess instead of asking the user, and skip checking their work.

The best systems score about 72% on short tasks and 20.6% on long ones. Short, well-defined jobs are within reach, and workflows that take a person most of an afternoon mostly are not.

### Do you need programming skills to use one?

No, not to use a finished product. Operator is a consumer feature inside ChatGPT, where the user types a request and watches the agent work in a browser window.

Building with Anthropic's tool is a developer job. The documentation describes an API in which the developer writes the loop, runs the actions and supplies the screenshots. Anthropic also notes that for tasks confined to web pages, its separate browser use tool is usually a better fit because it does not need a full desktop environment.

For related background on how AI systems connect to outside tools, see how the Model Context Protocol links models to software and [the wider rise of autonomous AI agents](/radar/rise-of-autonomous-ai-agents-multi-agent-workflows).

For anyone deciding whether to try one, the published guidance points the same way: start with low-risk tasks, as Anthropic advises, and keep a person in the loop for anything involving money, logins or agreements.

### What to watch next

Long-task scores matter most. Vendors tend to publish results on short benchmarks, and the longer ones are where the open questions sit. OSWorld 2.0 gives a harder baseline, and the 20.6% top result is the number to compare against when new models arrive.

Audited results matter too. The leaderboard owners already flag self-reported scores as unaudited, so independent verification will matter as vendors publish new claims.

The safety record needs watching too. Prompt injection remains an open problem that vendors manage with classifiers and human approval steps, so watch for independent testing of those defenses and for how often products still ask users to confirm actions.

## Frequently asked questions

### What is a computer use agent?
A computer use agent is an AI system that operates a computer through its screen, much as a person does. It takes screenshots, decides what to do, then moves the cursor, clicks and types. It needs no special connection to the software it uses, only the same visible interface a human would see.

### How is a computer use agent different from a chatbot?
A chatbot returns text. A computer use agent acts on a real interface: it looks at screenshots, clicks buttons and types into forms to finish a task. Anthropic describes the loop as the model requesting actions, the developer's software running them, and the results going back to the model.

### Is it safe to give an AI control of my computer?
Its makers urge caution. Anthropic recommends a dedicated virtual machine, keeping sensitive data and logins away from the agent, limiting web access to approved domains and requiring human confirmation for consequential actions. Prompt injection, hidden instructions in web pages or images, is the named risk that these steps are meant to contain.

### How good are computer use agents at real work?
They do well on short tasks and poorly on long ones. OpenAI reported 38.1% on OSWorld in 2025. The OSWorld 2.0 paper, built on long workflows that take a person about 1.6 hours, reports a best completion rate of 20.6%, and the authors say agents are far from professional-level computer use.

## Source
Based on Anthropic, OpenAI and the OSWorld 2.0 paper. https://www.anthropic.com/news/3-5-models-and-computer-use

---
Published on SyntaxRadar (https://syntaxradar.com). Canonical URL: https://syntaxradar.com/radar/computer-use-ai-agents-operating-systems-claude-operator. Last modified 2026-10-29T10:33:58.900Z.