Claude has been multimodal for a while, but in 2026 vision is no longer a side feature. It is the entry point for most document and agent workflows. Sonnet 4.6 and Opus 4.8 both accept images natively in the same messages API you already use for text. On text-heavy inputs (PDFs rasterized to PNG, screenshots, scanned invoices, dashboards) the quality is the reason most teams pick Claude over the alternatives for backoffice automation.
Think of this post as a practical handbook: which formats Claude accepts, what they cost, what the model is good at and where it fails. It also has copy-paste code that runs through the codexkey gateway (https://api.codexkey.online/v1).
Input formats and limits
Claude vision takes images as base64-encoded blocks inside a user message. The supported formats:
- PNG (
image/png) - JPEG (
image/jpeg) - GIF (
image/gif): first frame only, no animation - WebP (
image/webp)
Hard limits to keep in mind:
- Max file size: 5 MB per image, base64-encoded. Anything larger has to be downscaled or split.
- Max dimensions: 8000 × 8000 pixels. Larger images are rejected outright.
- Recommended dimensions: 1568 px on the long edge for general content, 2000-2500 px for dense text. Going above ~1568 px does not improve recognition on ordinary photos and only inflates the bill.
- Max images per request: 100, though in practice 20 or more already fill the context for most tasks.
- No video, no audio, no image streaming. You cannot push frames into a live conversation. If you need video, sample frames yourself and send them as separate images.
Animated GIFs, screenshots from HiDPI monitors and PDF page exports all work. Claude ignores EXIF, color profiles and DPI metadata. Only the raw pixels matter.
Pricing: vision is not free
A common surprise: vision usually makes up most of a request's input cost. Claude splits images into pixel tiles and bills them as input tokens. Roughly:
- A 1092 × 1092 image ≈ 1600 input tokens.
- Cost scales close to linearly with
width × height ÷ 750. - A full-page A4 screenshot at 2000 px tall lands around 2400-2800 input tokens.
At Sonnet 4.6 input pricing (see Claude API pricing in 2026), one full-page screenshot costs about as much as 2000 tokens of prose. Send ten screenshots in one request and the input bill is 25-30k tokens before the user's prompt is even counted.
Two consequences:
- Crop aggressively. If you only need the chart in the corner, send only the chart. Half the pixels, half the price.
- Cache the system prompt. With prompt caching on, the text part of long agent prompts gets cheap and image tokens remain the main cost. That is the right trade, since images change every turn anyway.
Single image: minimal example
import anthropic, base64, pathlib
client = anthropic.Anthropic(
api_key="sk-...",
base_url="https://api.codexkey.online/v1",
)
img = base64.standard_b64encode(
pathlib.Path("invoice.png").read_bytes()
).decode("utf-8")
resp = client.messages.create(
model="claude-sonnet-5",
max_tokens=1024,
messages=[{
"role": "user",
"content": [
{
"type": "image",
"source": {
"type": "base64",
"media_type": "image/png",
"data": img,
},
},
{
"type": "text",
"text": (
"Extract invoice number, date, vendor, total amount, "
"and line items. Return strict JSON with keys: "
"invoice_number, date, vendor, total, currency, items[]."
),
},
],
}],
)
print(resp.content[0].text)
That is the whole vision API. The image block sits inside content like any other part of the message, and you can mix images and text freely in one turn.
Multi-image: comparison and grounding
Multi-image is a strong suit for Claude. Send two screenshots and ask what changed. Or send a chart plus a table and ask whether they agree.
def img_block(path, media_type="image/png"):
data = base64.standard_b64encode(pathlib.Path(path).read_bytes()).decode()
return {
"type": "image",
"source": {"type": "base64", "media_type": media_type, "data": data},
}
resp = client.messages.create(
model="claude-sonnet-5",
max_tokens=2048,
messages=[{
"role": "user",
"content": [
{"type": "text", "text": "Image 1 is yesterday's dashboard, Image 2 is today's."},
img_block("dash_yesterday.png"),
img_block("dash_today.png"),
{"type": "text", "text": "List every metric that changed by more than 5%, as JSON."},
],
}],
)
Claude reliably reads the surrounding text as labels for each image. You don't need to encode position metadata. Just describe the order in plain words.
Use cases that actually work
1. OCR and document extraction. Invoices, receipts, contracts, bank statements, ID cards. Claude copes with multi-column layouts, rotated scans and handwritten annotations far better than classical OCR pipelines, and you get structured output in one call instead of a chain of OCR → parser → validator.
2. Chart-to-data. Send a bar or line chart and ask for the underlying values as JSON. Accuracy is excellent on labeled charts and decent on unlabeled ones (Claude estimates from the gridlines). If you intend to act on the result, ask for confidence per value.
3. UI screenshot to test selectors. Show Claude a page screenshot and ask for Playwright selectors for the visible elements. Paired with an MCP browser tool, this is the basis of self-healing E2E tests.
4. PDF QA. Rasterize each page to a PNG about 1500 px tall and send the relevant pages with your question. That beats text-only PDF parsing, because tables, stamps, signatures and figures survive.
5. Multi-image comparison. Before and after deploy screenshots, two versions of a design, two scans of the same form. Ask for a structured diff.
6. Visual debugging. Paste a screenshot of the error stack trace and one of the relevant code panel. Claude reads both and explains the bug.
Prompt patterns that improve accuracy
- Force JSON. "Return strict JSON. Do not include prose. If a field is unreadable, use
null." With an explicit schema, Claude rarely invents fields. - Demand confidence. Put
confidence: low|medium|highnext to every extracted value. Low-confidence rows go to human review. - Anchor with text. Before each image, write a one-line description ("This is page 3 of a German tax form"). It primes the model and improves OCR on language-specific glyphs.
- Chunk wide tables. For tables with 20+ columns, crop two halves and send them as two images with overlapping columns, so alignment errors show up.
- Crop what you don't need. Headers, footers, watermarks and ads in screenshots all consume tokens. A 30% crop is a 30% discount.
- Dense text needs resolution. For 8-pt scanned text, 2500 px on the long edge is worth the extra tokens. For natural photos, 1568 px is plenty.
Limits and refusals
- No video. Sample frames yourself.
- No streaming images. You can't stream new images mid-completion. Send everything in the first request.
- CAPTCHAs and watermarks. Claude refuses to solve CAPTCHAs and is cautious about personal documents (passports, IDs) when no context is given. State the legitimate use case in the system prompt.
- Faces and identification. Claude describes people in general terms but won't name an individual from a photo.
- Adult and graphic content. Refused.
- Coordinates. Claude can say where things are ("top right", "below the heading"), but bounding-box coordinates are approximate, not pixel-perfect. For precise boxes, use a dedicated detection model.
Claude vs GPT-4o on vision
Both are strong, but judging by what teams ship, their sweet spots differ:
- Claude wins on text-heavy and structured documents. Invoices, tables, multi-page contracts, dashboards, code on a screen, scanned forms. The OCR is more accurate and the JSON output more disciplined.
- GPT-4o wins on natural photos and physical-world reasoning. Identifying objects in a cluttered room, reading body language, interpreting ambiguous scenes. It also has lower latency at small image sizes.
- Charts are a tie. Both extract values from clean charts well, and both struggle equally with stylized infographics.
- Claude wins on long multi-image context. Sending 20 screenshots of an app flow in one request and asking for a coherent analysis is where Claude's larger context and better cross-image grounding pay off.
For backoffice automation, document pipelines and agent workflows that look at app screenshots, Claude is the default. For consumer photo features, GPT-4o is competitive.
Bottom line
Vision plus tool use turns Claude from a chatbot into a document agent: it looks at a screen, decides what to do and calls a tool to do it. In 2026 most production "AI document" features run on this loop: rasterize, send to Claude, get JSON, validate, act. Start with one image, force JSON output, measure accuracy on a sample of 100 real documents, and only then scale up. Vision is reliable, but it is also where the token bill quietly grows. Crop early, cache prompts, and never send a 4K screenshot when 1500 px would do.