Run Maxwell-V1-Flash Locally: RAM, VRAM, and Long-Context Requirements

Eva Wong is the Technical Writer and resident tinkerer at ZimaSpace. A lifelong geek with a passion for homelabs and open-source software, she specializes in translating complex technical concepts into accessible, hands-on guides. Eva believes that self-hosting should be fun, not intimidating. Through her tutorials, she empowers the community to demystify hardware setups, from building their first NAS to mastering Docker containers.

Running Maxwell-V1-Flash locally starts with roughly 73.6 GB of original checkpoint files, a compatible NVIDIA GPU environment, and enough additional memory for the runtime and conversation cache. Its 3B active-parameter figure describes the experts used for each token; it does not turn the complete model into a 3B download. For the original weights, a 96GB-class GPU is a more credible starting point than a consumer 24GB card, while a full 256K context needs a separate memory budget.

This guide separates verified model specifications from planning estimates, explains why sparse attention still consumes long-context memory, and walks through the developer's documented local serving route. It also identifies where a ZimaCube 2 can support the workflow without suggesting that a compact NAS replaces a high-memory inference server. Specifications and deployment instructions were checked on November 18, 2026; the configurations below are estimates, not measured minimum requirements.

What is Maxwell-V1-Flash, and which specifications affect your hardware?

The Maxwell Team's repository is named Maxwell-V1-Flash, while its model card calls the model Maxwell-V1.0-Flash. It is a text-only, open-weight mixture-of-experts language model released under Apache 2.0. The official description lists 35B total parameters and 3B active parameters per token, both excluding embeddings. Those exclusions matter when comparing the headline parameter count with the actual files you must download.

Verified specification What it means for local deployment
35B total / 3B active, excluding embeddings Reduced active computation does not remove the storage and memory requirement for the expert weights.
45 backbone layers; 32 query heads and 4 KV heads; head dimension 128 These dimensions determine a useful baseline estimate for the main attention cache.
256 routed experts; 12 selected per token; 2 shared experts per MoE layer Expert routing needs architecture support in the inference engine.
256K context advertised in the model card Long prompts require additional cache, prefill time, and room for generated output.
Block-sparse attention and manifold-constrained hyper-connections A runtime that supports another MoE model is not automatically compatible.
73,625,607,468 bytes across the official safetensors files Approximately 73.6 decimal GB, or 68.6 GiB, before runtime allocations.

These facts come from the official model card, published configuration, and checkpoint file metadata. The card lists BF16 as the main weight precision, while the repository contains both BF16 and F32 tensors. File size is consequently a better download estimate than multiplying a rounded headline parameter count by two bytes.

Official Maxwell brand artwork from the Maxwell-V1-Flash model repository
Maxwell’s official brand artwork. The repository name is Maxwell-V1-Flash; the model card uses Maxwell-V1.0-Flash. Source: Maxwell-V1-Flash official model card.

How much RAM, VRAM, and storage should you plan?

There is no developer-published minimum-RAM table in the deployment instructions we checked. The following configurations are practical planning targets derived from the checkpoint footprint and cache arithmetic. They are not benchmarked configurations, and GPU architecture, kernel compatibility, and the memory reported by the actual runtime remain decisive.

Workload System RAM target GPU memory target Working storage
One user, original weights, initial 8K–16K test 128 GB 96GB class; an 80GB card is a tighter configuration requiring validation At least 150 GB free on local NVMe
One user, 32K–64K coding or document sessions 128–192 GB 96GB class or a validated multi-GPU arrangement, with measured cache headroom 200–300 GB free for weights, container layers, logs, and revisions
128K–256K sessions or multiple active users 192–256 GB as an operational target Plan above 96 GB aggregate usable VRAM; 128–160 GB or more offers a more credible budget, subject to parallel-runtime support 300 GB or more free; retain a separate backup location

RAM and VRAM serve different purposes. System RAM supports loading, file caching, preprocessing, and other services. GPU VRAM holds the resident model, attention cache, activations, temporary workspaces, and runtime allocations. Adding 128 GB of host RAM does not automatically make a 24GB GPU capable of serving the original checkpoint.

Likewise, the RAM targets are headroom recommendations, not a claim that every loader must hold a complete extra model copy. Streaming and memory-mapped loading can change the host-memory peak. Start with a modern Linux x86-64 machine, roughly 12–16 capable CPU cores for the surrounding workflow, a supported NVIDIA GPU, and local NVMe. CPU model and core count are secondary to the GPU memory and software compatibility checks here.

An 80GB accelerator may have room for the original weights and a small cache, but that observation is not an endorsement of every 80GB GPU. Some memory is reserved or consumed by the runtime, and custom kernels may impose additional architecture requirements. Check the exact card against the developer's image and measure a short-context run before buying hardware around it.

Why does sparse attention still need memory at 256K context?

Maxwell Block Sparse Attention, or MBSA, selects a limited number of blocks for the main attention computation. The official explanation describes eight prefix blocks, eight local blocks, and up to sixteen remotely selected blocks, each containing 128 tokens. That bounds the selected main-attention span at 4,096 tokens per KV group in sparse mode.

The full-history KV cache remains available. The selected blocks reduce attention work; they do not mean that only 4,096 tokens are stored. The reference implementation also maintains indexer keys. This is the most important distinction when sizing a machine for long conversations or repository analysis.

For orientation, a conventional BF16 main-KV calculation using the published backbone dimensions is:

45 layers × 2 (K and V) × 4 KV heads × 128 dimensions
× 2 bytes × cached tokens × independent sequences
Cached tokens, one sequence Illustrative main K/V payload only Practical implication
8,192 0.70 GiB A sensible first inference test.
32,768 2.81 GiB Already needs headroom beyond the weights.
65,536 5.63 GiB Long coding sessions start consuming a meaningful memory reserve.
131,072 11.25 GiB Prefill latency and cache allocation both need measurement.
262,144 22.50 GiB Weights plus this payload alone are about 91.1 GiB, before other allocations.

This is an explanatory calculation, not a measured vLLM allocation. It excludes indexer state, auxiliary paths, block allocation, activation peaks, CUDA graphs, fragmentation, and runtime workspaces. Cache precision, implementation details, and parallel placement can change the actual footprint. Even an apparently generous 96GB configuration should not be assumed to handle the full advertised context.

Also note a source discrepancy: the model card advertises 262,144 tokens, but the current configuration file sets max_position_embeddings to 524,288 and uses a different RoPE base than the card's specification table. A larger configuration field does not establish tested 512K quality or reliable serving. This guide uses the advertised 256K capability and recommends explicit, smaller serving limits during validation.

Can quantization make Maxwell fit a 24GB or 32GB GPU?

Potentially, but the existence of a smaller checkpoint is only one part of the answer. A four-bit calculation based on the original tensor footprint suggests an order of magnitude around 18–19 GB for raw weights. Real quantized files also contain scales, metadata, and tensors kept at higher precision. Cache and runtime memory remain additional costs.

That arithmetic does not establish a working 24GB deployment. Maxwell's expert layers, hyper-connections, attention implementation, and checkpoint loader all need compatible quantization support. A community GGUF, FP8, or four-bit repository should state the exact runtime build, required patches, supported GPU, and a reproducible generation example before you treat it as a purchase recommendation.

The developer's deployment example points to its own vLLM container and original weights. It does not provide a universal Ollama command or an official Q4 memory guarantee. If you choose a community conversion, first reproduce a short prompt, verify the chat template and tool calls, then measure memory and output quality against the original. Our guide to checking community models, chat templates, and remote code covers the trust checks that apply before loading a custom repository.

For a small consumer GPU, a smaller model with a verified runtime is often the easier starting point. Compare deployments in the AI model hardware requirements directory rather than choosing exclusively by active-parameter count.

How do you start the official local vLLM server?

The documented route uses maxwellmodel/vllm-openai:v0.30.0, not an assumption that any stock vLLM image works. The model card specifies NVIDIA driver 580 or newer for its CUDA 13.0 environment, and mentions a CUDA compatibility environment option for older drivers. That option still depends on a compatible GPU and driver combination; it is not a substitute for checking NVIDIA's compatibility requirements.

Install Docker with NVIDIA GPU access, check nvidia-smi, and download the official repository using the Hugging Face CLI:

export MODEL_DIR="$PWD/Maxwell-V1-Flash"
hf download maxwell-model/Maxwell-V1-Flash --local-dir "$MODEL_DIR"

The following adapts the official launch command for a first, local-only 8K test. It retains the documented image, block size, and parsers, while adding a smaller context, one sequence, and a loopback binding:

docker run --rm --init \
  --name vllm-maxwell \
  --gpus all \
  --shm-size 8g \
  --network host \
  -v "$MODEL_DIR":/models/Maxwell-V1-Flash:ro \
  --entrypoint vllm \
  maxwellmodel/vllm-openai:v0.30.0 \
  serve /models/Maxwell-V1-Flash \
  --trust-remote-code \
  --block-size 128 \
  --served-model-name Maxwell-V1.0-Flash \
  --host 127.0.0.1 --port 8000 \
  --tensor-parallel-size 1 \
  --max-model-len 8192 \
  --max-num-seqs 1 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder

The single-GPU setting follows the developer's example; it does not make insufficient VRAM disappear. If the checkpoint cannot fit, use a compatible larger-memory machine or validate the developer image's multi-GPU capabilities before changing the parallel size. Two cards do not become one transparent memory pool merely because their capacities add up.

Because this launch enables custom model code, review the repository and container source, record the model revision and container digest, and avoid mounting credentials or unrelated personal folders. The read-only model mount limits writes to the checkpoint directory. It does not sandbox every capability of a container with GPU access.

After startup, verify the model endpoint and a small completion:

curl http://127.0.0.1:8000/v1/models

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"Maxwell-V1.0-Flash","messages":[{"role":"user","content":"Explain a NAS snapshot in two sentences."}],"max_tokens":512}'

Thinking is enabled by default in the official template. A small output allowance may be consumed by reasoning before the visible answer, so inspect reasoning fields and finish reasons instead of treating a blank final answer as immediate proof of a broken model. Follow the bundled template and preserve assistant reasoning and tool-call fields in multi-turn agent conversations, as the developer instructs.

How should you test long context and multiple users?

First record a repeatable 8K baseline: model revision, image digest, GPU model, driver, precision, startup peak, steady memory, time to first token, and generation throughput. Then increase the configured limit to 16K or 32K and test a prompt that actually approaches the intended workload. A server starting with a high limit does not prove that a long prefill will succeed.

Increase one variable at a time. Keep concurrency at one while testing context; then keep context fixed while adding users. Two independent 128K sessions can require substantially more cache than one, even though the weights are shared. Reserve output tokens within the total context budget, and include tool messages and reasoning history in your token accounting.

For document work, retrieval can reduce the need to repeatedly send a whole archive. For coding agents, exclude generated files and dependencies, summarize completed steps, and keep working directories separate. Our Git worktree guide for multiple coding agents addresses repository isolation; it does not eliminate each agent's inference-memory needs.

If a test fails, distinguish a loader or unsupported-kernel error from out-of-memory during loading, prefill, or decoding. More disk space cannot fix a GPU allocation failure. Reducing context helps cache pressure but does not shrink the original weights. Record the failing stage before changing the machine or quantization.

Maxwell Team benchmark overview for Maxwell-V1.0-Flash, reproduced from the official model card
Developer-reported capability benchmarks, reproduced from the official model card. These results are not local GPU memory or throughput measurements. Source: Maxwell-V1-Flash official model card.

Where does ZimaCube 2 fit in a Maxwell deployment?

The complete original model is not a sensible reason by itself to buy a compact Zima system. The documented path is a high-memory NVIDIA inference environment. ZimaCube 2 should not be presented as a guaranteed host for this checkpoint, a 256K session, or an arbitrary external accelerator.

A more realistic role for ZimaCube 2 as the storage side of a local AI workflow is keeping model archives, document collections, project backups, and evaluation results available to a separate GPU workstation or server. Where your chosen software supports it, the storage machine can also host surrounding services while the GPU machine handles generation.

Keep the active checkpoint on the inference machine's local NVMe for predictable loading, and use the NAS for a versioned archive rather than assuming network storage behaves like VRAM. Confirm available RAM, supported services, networking, backup space, and any physical expansion requirements for the exact ZimaCube 2 configuration. That is a storage-and-services recommendation, not a claim that the product card below meets Maxwell's full compute requirement.

FAQ

Is Maxwell-V1-Flash really a 3B model?

It activates about 3B parameters per token, excluding embeddings, but has about 35B total parameters on the same basis. You still need access to the full expert checkpoint. Its original safetensors files occupy approximately 73.6 GB.

Can I run the original weights on an RTX 3090 or RTX 4090?

A single 24GB card cannot hold the original checkpoint entirely in VRAM. A smaller community conversion needs its own proven loader and kernel support. Do not infer compatibility from a generic four-bit memory calculator.

Does an 80GB GPU support 256K context?

Do not plan on it for the original checkpoint. The illustrative main BF16 cache alone adds 22.5 GiB at 256K, putting weights plus that cache around 91.1 GiB before additional allocations. A short-context experiment is a different workload.

Can I use Ollama, llama.cpp, Apple Silicon, or CPU-only inference?

The first-party deployment route checked here is the developer's NVIDIA vLLM image. These other environments require separate, architecture-specific evidence. A downloadable conversion or large amount of unified memory is not sufficient proof of a usable runtime.

Does the configuration file prove 512K support?

No. Its 524,288 position setting differs from the 256K capability advertised in the model card. Validate quality, runtime support, and memory independently before relying on a larger context.

Where can I check the original specifications and runtime requirements?

Use the Maxwell-V1-Flash model card, official checkpoint files, Maxwell GitHub repository, and vLLM GPU installation requirements. Check the developer's pinned image separately from upstream vLLM documentation.

Conclusion

Maxwell-V1-Flash is a serious local-inference project because its active compute is much smaller than its total weight footprint, while its sparse attention still retains full conversation history. Budget around the original 73.6GB checkpoint, leave substantial runtime headroom, and treat 128K–256K serving as a separate capacity test.

Start with the developer's supported environment, an explicit 8K limit, and one request. Measure before extending context or buying more hardware. Use ZimaCube 2 where shared storage, backups, and surrounding services fit your architecture, and choose the inference GPU according to the model's measured requirements.

Tech & AI HUB

More to Read

Get More Builds Like This

Stay in the Loop

Get updates from Zima - new products, exclusive deals, and real builds from the community.

Stay in the Loop preferences

We respect your inbox. Unsubscribe anytime.