How to Run Kimi K3 Locally: Hardware, Memory, and Deployment Limits

Running the full Kimi K3 locally is possible with released weights, but practical deployment still requires cluster-scale memory, accelerators, and interconnects.

After the July 27 open-weight release, the question is no longer whether a checkpoint exists, but whether your system can download, load, and serve it at useful speed. The public repository is about 1.56 TB across 96 safetensor shards, while the model contains 2.8 trillion total parameters and activates 104 billion per token. Those numbers keep a normal PC, Mac, home NAS, or single-GPU server outside the practical full-model range, so the sections below separate storage, memory, accelerator topology, runtime support, and realistic fallback paths.

What Is Available Now That Kimi K3 Weights Are Released?

The Kimi K3 open-weight release includes the full model checkpoint and technical report. The public model repository currently shows about 1.56 TB of files and 96 numbered safetensor shards. That figure is useful for planning downloads and disk capacity, but it is not a minimum VRAM specification.

The released model summary confirms 2.8T total parameters, 104B activated parameters, 93 layers, 69 Kimi Delta Attention layers, and 24 Gated MLA layers. Its Stable LatentMoE routing selects 16 of 896 routed experts per token and also uses two shared experts. The weights are MXFP4, activations are MXFP8, and the advertised maximum context is 1,048,576 tokens.

Deployment support is also more concrete than it was before release. vLLM, SGLang, and TokenSpeed are listed as recommended inference engines, but each requires K3-aware kernels, model code, sharding, and memory settings. A generic command shown by a client library does not turn the checkpoint into a consumer-scale local model.

Why Does Sparse MoE Still Require Enormous Memory?

Kimi K3 performs computation with 104B activated parameters for each token, but all MoE experts still consume storage and serving memory. The router can select different experts for the next token, so the full 2.8T weight set must remain accessible somewhere in the deployment.

Selecting 16 of 896 routed experts reduces the expert work performed for one token. It does not mean a machine can keep only 16 experts, discard the rest, and still run the released model unchanged. Expert pruning, distillation, or streaming would create a different operating tradeoff and must not be confused with normal sparse activation.

The 104B activated figure is therefore a compute-scale description, not a shortcut for estimating checkpoint size. Multiplying 104B by four bits and claiming the model needs only about 52 GB would ignore inactive-but-still-required expert weights, dense components, shared experts, attention layers, vision weights, and runtime state.

What Is the Minimum Weight-Memory Floor?

At 2.8 trillion parameters, the simplest lower-bound calculation is total parameters multiplied by stored bits per parameter. MXFP4 gives a four-bit payload floor: 2.8T × 4 bits is about 1.4 TB, or roughly 1.27 TiB, for the raw weight payload alone.

The released repository is about 1.56 TB, which shows why serving memory extends beyond a simple weight calculation. Checkpoint packaging, block scales, tensor alignment, configuration files, tokenizer assets, vision components, and other model data push the real download above the theoretical four-bit floor.

Four different budgets must be planned separately: persistent download storage, temporary staging space, host RAM, and accelerator HBM or VRAM. A running service then needs additional room for KDA state, MLA KV cache, activations, communication buffers, kernels, graph capture, and failure headroom. The 1.56 TB repository size is therefore neither a complete RAM requirement nor a complete GPU-memory requirement.

What Hardware Can Actually Run Kimi K3 Locally?

There is no honest consumer-oriented minimum GPU list for the full model. A system that can technically map or stream the checkpoint is not automatically capable of stable, interactive serving. Practical Kimi K3 hardware requirements depend on weight residency, supported MXFP4 kernels, interconnect bandwidth, cache capacity, context length, concurrency, and the serving engine.

Published day-zero Kimi K3 serving support places the realistic starting class at an eight-accelerator enterprise node. Current vLLM materials describe eight GB300-class or MI350X/MI355X-class accelerators as starting points, while SGLang publishes topology-aware examples including B300 1×8, GB300 2×4, B200 2×8, H200 2×8, H100 4×8, and MI350X/MI355X 1×8.

These are published runtime recipes and starting configurations, not a single certified minimum for every workload. Older or smaller accelerators may require more nodes, different quantization kernels, reduced context, or additional expert parallelism. Production traffic also needs capacity for concurrent requests, failed workers, and performance headroom rather than merely fitting the checkpoint once.

Why Is One Workstation or NAS Still the Wrong Topology?

Raw capacity division understates the problem. Even if enough aggregate memory is assembled, expert parallelism turns routing into network traffic. Tokens must reach the accelerators holding their selected experts and then return to the rest of the model pipeline.

Enterprise accelerator nodes provide more than memory. They combine high-bandwidth GPU links, RDMA-capable networking, collective communication libraries, and kernels designed for tensor, expert, data, or pipeline parallelism. A collection of consumer GPUs connected through ordinary PCIe or a home network may show enough nominal capacity while remaining far too slow or fragile for useful serving.

A NAS is valuable for storing checkpoint shards, logs, datasets, retrieval indexes, and application data, but network storage does not replace accelerator memory bandwidth. The better home-server role is usually to keep private data and retrieval close to the user while leaving frontier-model inference to a suitable cluster or hosted endpoint. In that design, a home server separates the local data layer from frontier inference.

How Do KDA State, MLA KV Cache, and Context Length Raise the Budget?

Kimi K3 does not use one uniform full-attention cache across all 93 layers. Its 69 KDA layers and 24 Gated MLA layers create two different serving-memory demands: a KDA state pool with fixed model geometry for admitted requests, and a paged MLA KV pool that grows with stored tokens.

This split means KDA can reduce the long-context growth found in conventional attention, but it does not make a one-million-token request free. As KDA state and MLA KV memory compete for accelerator capacity, the KDA side can cap admitted requests while the MLA side limits total cached tokens.

Batch size, concurrency, average prompt length, generated reasoning length, multimodal inputs, cache precision, and prefill/decode strategy all change the usable capacity. The 1M-token figure is a maximum model capability, not a recommended default. A first deployment should start with a shorter maximum context, batch size one, and low concurrency before measuring out-of-memory failures, prefill time, decode speed, and cross-node traffic.

How Can You Run Kimi K3 Locally After the Open-Weight Release?

Running Kimi K3 locally now means building a distributed inference service around the released checkpoint, not installing a normal desktop application. The safest sequence is to validate storage, runtime support, topology, and a small operating point before increasing context or traffic.

1. Prepare storage. Reserve at least the roughly 1.56 TB repository footprint plus extra space for partial downloads, caches, container images, logs, and temporary files.
2. Select a supported engine. Use a Kimi K3-specific vLLM, SGLang, or TokenSpeed deployment path with the required model code, kernels, and container or branch version.
3. Match the topology. Choose an enterprise multi-GPU or multi-node layout with enough HBM and the NVLink, MNNVL, or RDMA path expected by its tensor and expert parallel settings.
4. Start below the headline limits. Reduce maximum model length, batch size, and concurrency, then verify loading, OOM behavior, output correctness, prefill speed, decode speed, and all-to-all traffic.
5. Scale only after measurement. Add context, concurrent requests, cache features, multimodal inputs, or speculative decoding one variable at a time.

A command such as vllm serve or sglang serve describes how to start a compatible distributed runtime; it does not remove the hardware requirement. When the required accelerator class is unavailable, the realistic choices are a hosted API, a hybrid architecture that keeps files and retrieval local, or a smaller local model that fits the home server's actual memory and reliability budget.

FAQ

Can I run Kimi K3 locally on a normal PC, Mac, or home NAS?

Not at practical full-model speed. The repository is about 1.56 TB before serving overhead, while a normal local system also lacks the accelerator memory and high-bandwidth topology expected by current runtimes. Experimental expert streaming or heavy offloading may prove that a launch is technically possible, but it is not equivalent to responsive or production-ready serving.

How much storage and memory does Kimi K3 require?

The transparent MXFP4 weight-payload floor is about 1.4 TB, while the public repository is about 1.56 TB. You then need extra disk space, host RAM, accelerator HBM or VRAM, KDA state, MLA KV cache, activations, communication workspace, and operating headroom. There is no single number that represents all of those layers.

What is the minimum published GPU setup for Kimi K3?

The smallest published day-zero configurations are eight-card enterprise accelerator nodes, with current vLLM and SGLang paths centered on B300, GB300, or MI350X/MI355X-class hardware. Treat those as runtime starting points, not a universal guaranteed minimum; context, concurrency, engine version, and production targets can require more resources.

Can Kimi K3 run from SSD or NAS offloading?

SSD or NAS storage can hold checkpoint shards, and experimental runtimes may stream weights through host memory. The limiting issue is repeated movement of expert weights and state through storage, network, RAM, and PCIe links. Those paths are far slower than accelerator HBM and high-speed GPU fabrics, so an experimental launch may deliver unusable latency.

Does Ollama run the full Kimi K3 model locally?

The current Ollama Kimi K3 entry uses the kimi-k3:cloud tag. Running a local Ollama client does not mean the 1.56 TB checkpoint is loaded on the local machine; the listed route is cloud-backed.

Final Takeaway

Kimi K3 is now genuinely available as an open-weight model, so cluster operators can download and deploy the complete checkpoint rather than rely on pre-release estimates. The release changes verification and tooling availability, but it does not change the physical scale of a 2.8T-parameter model.

The most useful Kimi K3 memory numbers answer different questions: about 1.4 TB is the transparent four-bit weight-only floor, about 1.56 TB is the current repository footprint, and 104B is the activated compute scale per token. None of those numbers alone describes the complete memory required for a running service.

For most individuals and home-server users, the stopping boundary is clear: without an enterprise eight-accelerator node or a distributed cluster, use hosted inference, keep the private data and retrieval layer local, or select a smaller model. That is the practical way to benefit from Kimi K3 without treating a NAS, workstation, or single GPU as a frontier-model supernode.