# DeepSeek V4.1 Flash cuts global KV cache to 890 bytes per token

**DeepSeek V4.1 Flash reduces global KV cache to about 890 bytes per token, shrinking the state that long conversations leave behind.** The technical report released on September 10, 2026 puts that at roughly one quarter of V4 Flash at the same context length. Persistent KV storage falls to roughly one eighth under identical workloads. Those figures describe different storage boundaries, and neither measures the entire model's memory footprint.

The engineering choice is to separate state worth keeping from state cheap enough to reconstruct. Some layers share global records. Some records use lower precision. Local attention state expires or is rebuilt from a short replay. For agents that repeatedly consume code, documents and tool results, those choices affect the cost of resuming a conversation long after its previous turn.

#### Input uses about 8B active parameters; generation still traverses 40 layers

A KV cache stores the key and value representations used by attention. When generating another token, the model reads those saved numerical states instead of recomputing the whole history. They are not prose summaries.

In a conventional decoder-only Transformer, individual layers usually create KV from their own intermediate representations. An uncached prompt consequently passes through the full network. V4.1 Flash changes the source of the later layers' global memory with a Causal Encoder-Decoder, or CED: a 20-layer causal encoder followed by a 20-layer decoder. The decoder's global KV comes from a projection of the encoder's final output.

The two execution paths are:

**Prefill: process the known input and establish caches**

Input -> first 20 encoder layers -> construct decoder global KV from encoder output.

The final 128 prompt tokens also pass through the decoder to establish its local-window state.

**Decode: generate the next output token**

Current token -> 20 encoder layers -> 20 decoder layers -> predict the next token.

Both halves read their corresponding caches and establish state for new positions. Prefill activates about 8B parameters; decode activates about 16B across the complete path. The 16B figure already includes the encoder. For sufficiently long inputs, avoiding most decoder work makes prefill computation approach a halving, but hardware, kernels and batching still determine elapsed time. The [official technical report](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/resolve/main/DeepSeek_V41_Tech_Report.pdf) defines these architectural and computational boundaries.

“Causal encoder” also needs a precise reading. Consider a sentence about Sam eating an apple. A bidirectional encoder can use the later action and object when representing Sam. A causal encoder cannot access those later positions while representing the first word. At the final word, it can use the preceding words. Causality here restricts information flow. It does not establish causal-reasoning ability, and it does not force known prompt tokens to execute serially: a mask allows parallel prefill while blocking future positions.

There is a research precedent. In YoCo, introduced on May 8, 2024, Yutao Sun, Li Dong and coauthors explored generating global KV in an earlier network and reusing it in a later one, allowing prefill to exit early. V4.1 Flash follows that direction while retaining layer-specific local attention and introducing bounded replay. The [original YoCo paper](https://arxiv.org/abs/2405.05254) explains the precursor; the two architectures should not be treated as identical.

#### 1. Global history and local state have different storage needs

V4.1 Flash divides its attention memory into global KV, which grows with context, and local sliding-window attention, or SWA, with a 128-token window. Every layer keeps its own local state.

A fixed window bounds one local cache. It does not bound the total local storage across a service. Many conversations and continuation points can leave many separate windows behind, including states useful for regenerating an answer or resuming a branch. This distinction becomes important when deciding what to persist.

Global storage includes both main KV, which supplies attention, and indexer K, which helps retrieve relevant positions. Counting only the main records would leave part of the retrieval system out of the storage accounting.

#### 2. CSA2 shares four global caches across 38 layers

Compressed Sparse Attention 2, or CSA2, assigns layers three modes. Full creates new global KV and selects positions for the current query. Reindex reuses global KV and indexer K but makes its own index query to select positions again. Reuse also inherits the most recent corresponding Top-K selection.

Sharing records reduces duplicate storage; sharing selections avoids some repeated search. Each layer still computes its own attention query Q, maintains local SWA KV and performs its other network operations. Reuse does not skip a layer.

| Network section | Layer arrangement | Global KV groups |
|---|---|---:|
| Encoder layers 1-2 | Local SWA only | 0 |
| Encoder layers 3-20 | Three groups of one Full plus five Reuse layers | 3 |
| All 20 decoder layers | One Full, four Reindex and 15 Reuse layers | 1 |

That leaves four global KV groups for 38 layers with global attention. A group is a growing collection of historical records, not one vector. Dividing 38 by four would not produce a valid savings factor against V4: the predecessor already compressed its cache, and the groups have different sequence-compression settings.

CSA2 selects at most 512 global entries for a query, alongside the local window. Unselected entries remain stored because a later query may need them. A read limit is not a retention limit.

The decoder further reduces repeated search through hierarchical indexing. Its first Full layer scans all causally visible positions and constructs a shared candidate pool of at most 2,048 blocks with eight positions each: 16,384 positions in total. Later Reindex layers choose their Top-512 within that pool; Reuse layers inherit the result. Post-training exposes the model to the same candidate restriction.

The bounded pool caps scoring work for later indexers. The initial global scan still grows with context, as does the global cache. Describing the entire model as constant-cost long-context inference would erase that qualification. The [report's CSA2 section](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/resolve/main/DeepSeek_V41_Tech_Report.pdf) distinguishes the first scan from subsequent retrieval.

#### 3. The encoder merges positions; the decoder preserves them

The encoder's three global KV groups use a sequence-compression ratio of two: two tokens produce one record. The decoder uses a ratio of one, retaining global records at individual token positions.

The decoder therefore relies heavily on cross-layer sharing for storage savings without coarsening every historical position. Historical resolution, the number of positions read and the number of layers sharing records are separate design choices.

V4 used a mixture of CSA and HCA in its global branches; V4.1 switches to CSA2. An inference integration cannot assume that changing a compression setting in an older engine implements the new sharing and indexing paths.

#### 4. FP4 shrinks main KV, but scales still occupy storage

Main KV moves from FP8 to FP4. The report uses the four-bit E2M1 format with one eight-bit E4M3 scale per 16 channels. Counting values and scales alone gives:

`4 + 8 / 16 = 4.5 bits per value`

That arithmetic does not describe the average precision of every cache structure. Local SWA KV remains FP8, and global storage includes indexing data and different configurations across the network.

DeepSeek also uses quantization-aware training, or QAT, during post-training to adapt to low-precision main KV. Reads dequantize the stored values for attention. Lower storage and transfer requirements do not establish that attention matrix multiplication itself runs in FP4, or that inference speeds up by a fixed ratio.

#### 5. Persistent storage falls by dropping long-lived local state

The report says SWA KV made up nearly half of the predecessor's persistent KV storage. Global records may remain useful after a longer interruption, while local state is mostly valuable during the minutes when a session is active. Giving both the same retention treatment preserves local data that may never be used again.

V4.1 keeps global KV for long-term reuse and places encoder SWA state in a short-lived memory pool with a minutes-scale expiry. SWA KV no longer enters long-term persistent storage. When the global prefix hits but encoder local state is missing, the system replays the last 128 tokens of the cached prefix to approximately restore local state while processing new input. The cached global prefix can remain in place.

The report's old storage mix suggests this rough accounting.

`about 1/2 of the old storage retained as global KV x about 1/4 of its size = about 1/8`

Architecture, precision and retention policy jointly produce the result. The one-eighth comparison assumes identical workloads; it is not a universal disk-capacity multiplier.

The decoder follows a different lifecycle. On every prefill, it processes the final 128 tokens of the complete prompt through its 20 layers to establish local state for generation. Those local states serve the upcoming decode and do not enter persistent prefix storage.

Approximation is the consequential tradeoff. Local attention dependencies accumulate through layers. For a 20-layer decoder, the report gives a theoretical exact replay length of `20 x 128 = 2,560 tokens`. Bounded replay uses only the last 128, truncating earlier local dependencies. The report describes a small quality impact but explicitly does not claim mathematical equivalence to full computation. Decoder post-training simulates the replay constraint.

A deployment test should distinguish a cold request, a request with both global and local hits, and a global hit whose local state has expired. Average latency can conceal reconstruction work. Quality tests should include tasks that need earlier details, rather than questions answerable from the final paragraph alone. The [report's inference section](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/resolve/main/DeepSeek_V41_Tech_Report.pdf) provides the retention and replay conditions behind those test cases.

#### 6. CED saves input computation; the other mechanisms save different costs

CED allows decoder global memory to be built from encoder output. CSA2 reduces duplicate global storage and some repeated retrieval. FP4 reduces the size of main KV records. Sparse selection and hierarchical indexing bound the records read and the search space of later indexers. Bounded replay reduces long-term local storage and limits decoder reconstruction work.

At 890 bytes per token, one million tokens require about 0.89 GB of global KV in decimal units. This excludes model weights, local caches, workspaces and other runtime costs. It is useful capacity arithmetic, not evidence that the entire model fits on a small device.

The report also shows only about 25% growth in per-token decode FLOPs as context expands from 4K to 1M, a 256-fold increase in length. Its precision-weighted accounting assigns BF16, FP8 and FP4 weights of 1, 0.5 and 0.25 respectively. The curve reflects the combined design; it does not measure end-to-end latency or throughput.

#### Verify the runtime before budgeting the savings

The official model repository includes readable reference inference code. Its README explicitly distinguishes that implementation from a production serving engine. Generation uses a plain autoregressive path, and the short decode self-test in `model.py` checks plumbing with uninitialized weights. Passing it does not establish model-answer quality. These limits are documented in the [official inference README](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/blob/main/inference/README.md).

Protocol support is another separate boundary. The official deepseek-recipe project provides Rust and Python tooling for format and protocol conversion. It does not run inference or execute tool calls on behalf of a service. Correctly rendering a prompt therefore proves only part of an integration; the runtime must still implement the relevant attention, cache and replay paths. The [deepseek-recipe repository](https://github.com/deepseek-ai/deepseek-recipe) defines that scope.

For cost evaluation, long inputs, short outputs and repeated prefixes are promising workload characteristics. A service with little prefix reuse will capture less of the persistent-cache benefit. That is an architectural inference to test, not a measured claim about every deployment. The report does not supply a margin improvement that can be copied into a business plan. Measure GPU time, retained cache storage and transfers separately, then calculate the cost per served request.

#### Frequently asked questions

##### Does the 890-byte figure include model weights?

No. It describes global KV cache per token. One million tokens is about 0.89 GB in decimal units, excluding weights, local SWA caches, workspaces and other runtime overhead.

##### Do 8B prefill and 16B decode mean two separate models?

They describe active parameters in different phases of one CED architecture. Long inputs mostly traverse the first 20 layers; generation traverses all 40. The decoder also processes the final 128 prompt tokens to establish local state.

##### Does Top-512 discard the rest of the conversation?

No. It limits global entries read for the current attention query. Other entries remain available for subsequent queries, and total global storage still grows with context.

##### Is bounded replay an exact restoration of local state?

No mathematical equivalence is guaranteed. The 20-layer decoder has a theoretical exact replay length of 2,560 tokens, while bounded replay uses 128. Post-training adapts to the truncation; deployment tests should still measure task quality.

#### Authority Sources

- [DeepSeek V4.1 Flash technical report: architecture, precision and cache deployment conditions](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/resolve/main/DeepSeek_V41_Tech_Report.pdf).
- [YoCo original paper: global-cache reuse and early prefill exit](https://arxiv.org/abs/2405.05254).
- [Official DeepSeek V4.1 Flash model card: model and open-weight materials](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash).
- [Official inference README: reference implementation and self-test limitations](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/blob/main/inference/README.md).
- [deepseek-recipe: protocol tooling and its execution boundary](https://github.com/deepseek-ai/deepseek-recipe).
#### Author Insight

I would test the first request after local state expires before expanding the storage budget. The headline figure is striking, but users experience the wait and answer quality when they return to a conversation several minutes later. A stable global-only cache-hit path would justify considering more concurrent sessions. Merely fitting the history in memory would not answer that operational question.

#### Terminology

| Term | Meaning in this article |
|---|---|
| Prefill | Processing known input and establishing caches |
| Decode | Generating successive output tokens |
| CED | Causal Encoder-Decoder; encoder output supplies decoder global KV |
| CSA2 | Attention combining compression, sparse selection and cross-layer sharing |
| SWA | Sliding-window attention; a 128-token local window in this model |
| Global KV | Context-growing attention records that can be shared across layers |
| Bounded replay | Reconstructing approximate local state from a restricted suffix |
| Precision-weighted FLOPs | Operation accounting with weights for numerical precision, not a latency measurement |