Run Qwen 3.8 27B locally: memory, quantization, setup

Qwen 3.8 27B exists as an open model: Alibaba put the weights on Hugging Face on August 14, 2026 under the Apache 2.0 license. With 4-bit quantization, a graphics card or a Mac with 24 GB of memory is enough. Below we compute memory with the formula "parameters × bits per weight / 8", go through the launch commands and compare running it yourself with paying per token.

Does Qwen 3.8 27B exist, and what is open

Yes. The model card describes it as a dense language model with a vision encoder: 27 billion parameters, 64 layers, hidden size 5120. The layers alternate in a pattern of "three Gated DeltaNet layers, then one full-attention layer", so only 16 layers use full attention. The native context is 262,144 tokens, extendable to 1 million. Input can be text, images and video. Thinking is on by default, and the reasoning_effort parameter sets its depth: xhigh, medium or low.

The name causes a lot of confusion, so here is a map from searches to what actually exists:

Dates and the family lineup come from the Qwen3.8 repository README. An overview of the lineup and the vendor's prices is in our Qwen 3.8 review.

How much memory you need: the formula

Memory for the weights is simple arithmetic: parameter count × bits per weight / 8. The catch is that "bits per weight" is fractional for quantized formats. We do not pull the values out of thin air. We derive them from the file sizes that Unsloth publishes for GGUF: bits per weight = size in GB × 8 / 27.

The nominal figure understates the size: by 18% for Q4_K_M, 16% for IQ3_S and 7% for Q8_0. Quantized formats store block scales, and some tensors stay in high precision. So go by the "actual" column.

On top of the weights comes the context cache. Here the hybrid design works in the model's favor: only the 16 full-attention layers store keys and values, with 4 KV heads of dimension 256 each. For an FP16 cache that is 2 (keys and values) × 16 layers × 4 heads × 256 × 2 bytes = 65,536 bytes per token. This is our own calculation from the model card parameters; we did not count the DeltaNet layer state or runtime buffers.

Adding it up: Q4_K_M with a 32K context gives 16.5 + 2.1 = 18.6 GB, plus 1-2 GB for working buffers. That matches Unsloth's table, where 4-bit versions need 16-19 GB, and the Ollama tag size of 18 GB. Here is how to match hardware, leaving a 1.5 GB margin:

On a Mac the system takes part of the memory, so read the "24 GB" row as an upper bound. Unsloth's rule: the sum of RAM and VRAM should roughly match the size of the quantization. With less memory the model still starts, but noticeably slower, because it spills to disk.

How to run Qwen 3.8 27B: Ollama, llama.cpp, LM Studio

Ollama

The shortest path. The model page lists a single version, 27B, with a 256K context and text and image input:

ollama run qwen3.8

Other tags exist: qwen3.8:27b-mtp-q8_0 (30 GB), qwen3.8:27b-mtp-bf16 (56 GB), and for Apple silicon qwen3.8:27b-mlx (18 GB). Ollama runs a local server with the OpenAI-compatible address http://localhost:11434/v1/: clients demand a key, but the server does not check it. You can point an agent at this model right away; the Ollama page lists commands such as ollama launch opencode --model qwen3.8. Installing the agent is covered in our OpenCode guide, and Qwen Code works with Ollama and vLLM, see the Qwen Code article.

llama.cpp and LM Studio

When you need a specific quantization level, llama.cpp is more convenient. It pulls the file from Hugging Face on its own:

llama-server -hf unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_M

On Windows, llama.cpp installs with winget install llama.cpp. LM Studio has a link, lmstudio://open_from_hf?model=unsloth/Qwen3.8-27B-GGUF, and you pick the quantization in the download window.

vLLM and SGLang for a server

For a server with several GPUs, the Qwen3.8 README gives this command:

vllm serve Qwen/Qwen3.8-27B --port 8000 --tensor-parallel-size 4 --max-model-len 262144 --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder

The model card suggests generation settings. In thinking mode: temperature 1.0, top_p 0.95, top_k 20. Without thinking: temperature 0.7, top_p 0.8, top_k 20 and presence_penalty 1.5. Thinking is switched off with enable_thinking: False in the request.

When to run it locally and when to use the API

Running locally makes sense in three cases: the data cannot leave your machine, you need to work offline, or you constantly push large volumes through and the hardware already sits on your desk. Otherwise, do the math first.

Assume 40 million input and 8 million output tokens a month. At codexkey's October 29, 2026 price list, qwen3.8-flash costs 40 × $0.10 + 8 × $0.28 = $6.24, and qwen3.8-max costs 40 × $0.28 + 8 × $0.86 = $18.08. A 24 GB graphics card will not pay for itself on savings like that, but it does give you privacy and offline work.

One caveat matters more than the price: qwen3.8-flash is not the 27B. You cannot compare the quality of these models blindly. Put both side by side and give them five to ten of your own tasks:

from openai import OpenAI

local = OpenAI(base_url="http://localhost:11434/v1/", api_key="ollama")
cloud = OpenAI(base_url="https://api.codexkey.online/v1", api_key="sk_cdx_YOUR_KEY")

question = [{"role": "user", "content": "Give three reasons to run an LLM locally."}]

for client, model in ((local, "qwen3.8"), (cloud, "qwen3.8-flash")):
    answer = client.chat.completions.create(model=model, messages=question)
    print(model, "->", answer.choices[0].message.content[:300])

Recent DeepSeek models are also open under MIT, but they will not fit on an ordinary computer: V4.1-Flash has 552 billion parameters and V4 Pro has 1.6 trillion. More in the DeepSeek 4.1 Flash review.

Qwen through the codexkey API instead of running it locally

codexkey offers API access to Qwen models: a sk_cdx_... key, payment by Russian card, SBP (the Russian Faster Payments System) or cryptocurrency, and the base URL https://api.codexkey.online/v1. The 27B and Flash-Next are not on the list. The current set comes from GET /v1/models and the models page.

The codexkey price list as of October 29, 2026, per 1 million tokens; for current figures see the models page:

For the terms of qwen3.8-flash-free, see its model page. You create a key by following the quickstart. Alibaba's chat, its official addresses and access from Russia are covered in the Qwen Studio article.

FAQ

Does Qwen 3.8 27B exist?

Yes. Qwen3.8-27B was published on Hugging Face on August 14, 2026 under Apache 2.0: a dense model with 27 billion parameters that accepts images and video as input. Take the weights from the Qwen organization on Hugging Face (huggingface.co/Qwen). Copies and quantizations under third-party accounts exist, but check what is inside them yourself.

How much memory does Qwen 3.8 27B need?

In 4-bit Q4_K_M quantization the file is about 16.5 GB, and with a 32K context cache and buffers it comes to 19-20 GB. In BF16 the weights alone take about 55 GB. The formula is parameters × bits per weight / 8 plus the context cache, which for this model is about 64 KB per token in FP16.

How do I run Qwen 3.8 27B in Ollama?

Install Ollama and run ollama run qwen3.8. About 18 GB downloads. The local server answers at http://localhost:11434/v1/ over the OpenAI-compatible protocol.

Will Qwen 3.8 27B run on a 16 GB graphics card?

For 3-bit versions Unsloth gives 12-14 GB of total memory, and for 4-bit versions 16-19 GB. On a 16 GB card a 3-bit version with a small context is realistic. Q4_K_M hits the limit and leaves little room for the cache.

How is Qwen 3.8 Flash different from the 27B?

The 27B is a dense open model that you run yourself. Qwen3.8-Flash in the API is hosted by the vendor and has a 1 million token context. The open Flash-Next (125 billion parameters and 51 billion embeddings, 6 billion active) is not mentioned on the Qwen3.8-Flash page in QwenCloud, so the two should not be treated as one model.

Is Qwen3.8-27B available in codexkey?

No. The catalog as of October 29, 2026 has qwen3.8-max, qwen3.8-flash, qwen3.8-flash-free and qwen3.8-omni-flash. If you need the 27B itself, run it locally, for example with ollama run qwen3.8.

Are the weights of Qwen 3.7 open?

The official sources we opened do not confirm it. The 3.7 series is mentioned in the Qwen3.8 README as the previous one on the same architecture, and the repository's news feed shows no 3.7 releases between the April 3.6 releases and the August 3.8 releases.