Claude Computer Use in 2026: browser automation that holds

For most of the API era, getting a model to "use" software meant building a harness of tools: a search function here, a database query there, and, if you were brave, a Playwright script. Computer use changes the setup. You give Claude a screenshot of a real screen and a small set of primitives (click, type, scroll, key press), and the model drives the interface the way a person would. In 2026, on the current generation of Sonnet and Opus 4.8, this is stable enough for narrow production workflows. This post covers the mechanics, where it fits, how to deploy it safely and what it costs.

The mechanics: how the computer use loop works

The principle is simple. Every turn does the same four things:

1. Your harness takes a screenshot of a virtual display.
2. The screenshot goes to Claude as an image, together with the user's goal and the history of actions so far.
3. Claude replies with a tool call: computer with an action such as { action: "left_click", coordinate: [834, 412] }, or key, type, screenshot, scroll and so on.
4. The harness runs the action against the OS, takes a fresh screenshot, and the loop starts again.

The model does not run on your machine. It reasons over pixels and returns low-level intents. That is why the approach works on legacy software with no API, on sites that fight scrapers, and on internal tools nobody will ever wrap in REST.

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({
  apiKey: process.env.CODEXKEY_API_KEY!,
  baseURL: "https://api.codexkey.online/v1",
});

const response = await client.beta.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 1024,
  tools: [
    {
      type: "computer_20250124",
      name: "computer",
      display_width_px: 1280,
      display_height_px: 800,
      display_number: 1,
    },
  ],
  betas: ["computer-use-2025-01-24"],
  messages: [
    { role: "user", content: "Open the orders dashboard and export today's CSV." },
  ],
});

When computer use pays off, and when it does not

It is a tool with a sharp edge. It earns its place in these cases:

• Legacy software with no API. Internal ERPs, mainframe terminals, a vendor portal from 2009. Anywhere a proper integration would cost more than the work itself.
• Sites behind a login that block scrapers. A Playwright selector breaks there every week, while a model that reads the rendered page holds up much better.
• End-to-end tests for visual regressions. Not unit-level checks, but the question "does checkout still work after the cart redesign".
• RPA replacement. Most UiPath and Automation Anywhere bots are brittle click-by-coordinate scripts. Computer use swaps them for a system that adapts when a button moves.

Walk away in these cases:

• A real API exists. Computer use costs 100-1000 times more than a direct call.
• The path is high-volume and latency-sensitive. Every step is a round trip with image tokenisation.
• One wrong click is catastrophic: production money movement, infrastructure changes, deletions. Use an API with explicit allow-lists there.

Sandbox: Docker + xvfb + Playwright

Never point computer use at your real desktop. The setup that works in production is a disposable Docker container with a virtual framebuffer and a browser the model can drive:

FROM mcr.microsoft.com/playwright:v1.50.0-jammy

RUN apt-get update && apt-get install -y \
    xvfb x11vnc fluxbox imagemagick && \
    rm -rf /var/lib/apt/lists/*

WORKDIR /app
COPY package.json ./
RUN npm install

COPY . .

ENV DISPLAY=:99
CMD ["bash", "-c", "Xvfb :99 -screen 0 1280x800x24 & fluxbox & node harness.js"]

Inside the container your harness runs the loop. It starts Chromium through Playwright, takes page screenshots (or uses import from ImageMagick), sends them to Claude, parses the tool call and performs the action through page.mouse, page.keyboard and navigation primitives. Keep the container short-lived: one task, one container, a hard timeout, then destroy it. That sets your blast radius: whatever the model can break stays inside the container.

A minimal harness: the loop pattern

The working skeleton is short:

async function runTask(goal: string, maxSteps = 30) {
  const messages: Anthropic.MessageParam[] = [
    { role: "user", content: goal },
  ];

  for (let step = 0; step < maxSteps; step++) {
    const screenshot = await takeScreenshot();
    messages.push({
      role: "user",
      content: [
        { type: "image", source: { type: "base64", media_type: "image/png", data: screenshot } },
        { type: "text", text: `Step ${step}. What's next?` },
      ],
    });

    const res = await client.beta.messages.create({
      model: "claude-sonnet-5",
      max_tokens: 1024,
      tools: computerTools,
      betas: ["computer-use-2025-01-24"],
      messages,
    });

    if (res.stop_reason === "end_turn") return "done";
    const toolUse = res.content.find((c) => c.type === "tool_use");
    if (!toolUse) return "stuck";
    await executeAction(toolUse.input);
    messages.push({ role: "assistant", content: res.content });
  }
  throw new Error("max_steps exceeded");
}

Three details are easy to miss. Carry the assistant's earlier turns forward: without the action history Claude starts repeating itself. Always cap max_steps, because a runaway loop is the costliest failure of all. And write every screenshot and action to disk: when something breaks, the trace is the only debugger you have.

Reliability habits that make a difference

Months of running these loops in production lead to the same practices:

1. Decompose ruthlessly. "Process this 50-row spreadsheet" fails. The chain "open row 1, copy the email, paste it into the form, submit, go back to the spreadsheet" works, and you run it 50 times.
2. Restate the goal every turn. Putting "your goal is to export the CSV" in the user message at each step keeps the model on target while the page changes.
3. Take a screenshot at every step, even when the model did not ask for one. Sometimes the model acts on stale memory. A fresh image per turn removes that class of error.
4. Set a hard ceiling on steps and on time. A limit of 30 steps and 5 minutes per task catches infinite loops cheaply.
5. Check the result with a second model call. When the loop ends, take a last screenshot and ask Sonnet: "did the task succeed? Answer yes or no in one line." That gives you a cheap CI-style signal.

What it really costs

This is where intuition usually fails. Most of the spend is vision tokens, not output. A 1280×800 PNG takes roughly 1500-2000 input tokens depending on detail level. A 30-step task with one screenshot per step therefore runs 45 000-60 000 input tokens before the model says a word. Output is small by comparison: a couple of tool calls per step.

In practice a well-scoped task costs a few cents, rarely more than a couple of tens of cents from start to finish. Exact rates depend on the model; see our pricing notes for current numbers. Long exploratory runs on top-tier models can pass $1+ if you are careless. Two levers help: aggressive prompt caching for the system prompt and tool schemas, and shrinking screenshots to the lowest resolution the task tolerates. 1024×640 is often enough.

Common failures and what to do about them

The most frequent one is hallucinated coordinates: Claude clicks at (420, 380) while the button sits at (430, 395). The reliable fix is a retry with a coordinate grid. After a failed action, draw a faint grid over the screenshot and ask again. The model uses the grid as a visual anchor, and the second attempt almost always lands.

Other recurring problems:

• A stale screenshot after navigation. Wait for networkidle or a short setTimeout before the next capture.
• Modal dialogs the model overlooks. Add a line to the system prompt: "if you see a popup, dismiss it before continuing".
• Captchas. Stop. You should not automate them. End the task cleanly and hand it to a human queue.

Comparison with OpenAI Operator

OpenAI's Operator and Anthropic's computer use arrived at a similar design by 2026: a "screenshot in, action out" loop inside a browser sandbox. In practice they differ in two ways:

• Caution. Claude is noticeably more conservative about destructive actions: before deleting, paying or sending, it pauses and asks. In production that is a plus; in research it can be irritating.
• Reasoning depth on multi-step plans. Sonnet 5 with extended thinking does better where the task needs a plan of 10+ steps held in mind across page transitions.

Pick the one whose harness fits your stack. The capability gap is small, and the integration cost is not.

Bottom line

By 2026 computer use is a working tool you can finally trust on narrow, well-defined tasks. It does not replace proper APIs and it is the wrong choice for high-volume hot paths. For the long tail of "the system has no API and never will", though, it has quietly become the most practical option. Isolate it hard, cap the steps, log everything and treat it as an intern without supervision. For a few cents a task, you get a lot of value.