weight precision, parameter count, context length, KV cache, concurrent sequences and tokens per second. Time the job and identify CPU, storage or external-service waits before replacing hardware.
Imports, processing and simultaneous users.Ray Serve Hardware Requirements: RAM, CPU, Storage & Model Size & Inference
Plan Ray Serve RAM, CPU, persistent storage and service dependencies, with workload-based ZimaOS sizing, backups and practical hardware choices.
Ray Serve hardware requirements at a glance
Ray Serve supports distributed model-serving deployments and replicas. Size it around weight precision, parameter count, context length, KV cache, concurrent sequences and tokens per second, including the complete service stack and durable data. Select a supported CPU/GPU backend first. Add model weights, runtime buffers and KV cache; host RAM and GPU VRAM are different budgets. Zima hardware is not a validated CUDA inference host.
- RAM
- The linked source does not establish a fixed numerical whole-host baseline for this complete workload. Size the full stack during the busy operation described below.
- CPU
- The linked source does not establish a fixed numerical whole-host baseline for this complete workload. Measure CPU use during realistic activity.
- Storage
- Persist model weights, tokenizer/configuration, downloaded artifacts and any server state.
- Required services
- Follow the service topology in the linked upstream deployment for the chosen edition; include every required database, worker and storage service.
- Main sizing variable
- weight precision, parameter count, context length, KV cache, concurrent sequences and tokens per second.
- Practical Zima starting point
- Supporting storage/control services; inference or generation requires a separately verified backend. A Zima device is not certified here for the chosen model, accelerator or interactive generation speed.
From official requirements to the right setup
Plan the complete Ray Serve workload before selecting a server.
-
Official requirements
Choose the exact edition, stable release and supported container/runtime. Follow the service topology in the linked upstream deployment for the chosen edition; include every required database, worker and storage service.
-
Confirm your needs
Test a representative busy operation: weight precision, parameter count, context length, KV cache, concurrent sequences and tokens per second. Record peak resident memory, CPU, disk latency and completion time, including other containers.
-
Leave room to grow
Place active state on reliable persistent storage. Persist model weights, tokenizer/configuration, downloaded artifacts and any server state. Budget working space for an upgrade or import and test recovery before relying on the deployment.
-
Run it on ZimaOS
Verify the project has a supported headless/runtime deployment before using ZimaOS. Select a supported CPU/GPU backend first. Add model weights, runtime buffers and KV cache; host RAM and GPU VRAM are different budgets. Zima hardware is not a validated CUDA inference host.
Check every playback client
- Exact edition and stable release
- Compatible amd64 image/runtime
- All required companion services
- Realistic peak activity
- Persistent data and file ownership
- Free host ports and reachable URLs
- Working space and backup retention
- Successful database/file restore
Official minimum requirements
Reviewed October 30, 2026 against the linked upstream deployment documentation. Numerical values retain their stated minimum, recommendation or example scope; select release-compatible instructions before installation.
Select a supported CPU/GPU backend first. Add model weights, runtime buffers and KV cache; host RAM and GPU VRAM are different budgets. Zima hardware is not a validated CUDA inference host. Numerical examples above retain their stated deployment scope; co-hosted services and independent backups need additional capacity.
| Requirement | Official minimum | What this supports |
|---|---|---|
| RAM guidance | The linked source does not establish a fixed numerical whole-host baseline for this complete workload. Size the full stack during the busy operation described below. | Official guidance keeps its original test, recommendation or example scope. |
| CPU guidance | The linked source does not establish a fixed numerical whole-host baseline for this complete workload. Measure CPU use during realistic activity. | Core counts do not predict a fixed user, device or document capacity. |
| Storage planning | Persist model weights, tokenizer/configuration, downloaded artifacts and any server state. | Installation space is separate from retained content, working files and backups. |
| Service topology | Follow the service topology in the linked upstream deployment for the chosen edition; include every required database, worker and storage service. | Include every stateful companion service in the host budget. |
| Persistent state | Persist model weights, tokenizer/configuration, downloaded artifacts and any server state. | An image rebuild or container replacement is not a backup. |
| Deployment boundary | Select a supported CPU/GPU backend first. Add model weights, runtime buffers and KV cache; host RAM and GPU VRAM are different budgets. Zima hardware is not a validated CUDA inference host. | Check the linked project instructions for the selected edition and release. |
When to upgrade your hardware
Use observed Ray Serve workload pressure to decide when an upgrade is warranted.
Busy operations exceed the available window
The complete stack runs short of memory
Persistent data approaches available capacity
Inspect the application and each required service for OOM events, sustained swap or growing queues. Reduce unnecessary concurrency before adding capacity.
Shared hosts and background work.Persist model weights, tokenizer/configuration, downloaded artifacts and any server state. Track primary state and backup retention separately.
Long-lived installations and growing collections.Plan hardware growth with confidence
Address storage, memory, compute and deployment limits separately.
Use suitable active storage
Put latency-sensitive application/database state on durable local SSD storage and keep recoverable copies.
SATA or PCIe SSD according to measured data growth.Add memory for the complete stack
Choose a higher-RAM host when peak application and companion-service usage leaves inadequate OS headroom.
ZimaBoard 2 1664 or ZimaCube 2 Pro according to the measured constraint.Separate capacity from compute
A multi-drive system helps when retained content and backup storage are the problem; it is not automatically a faster application.
ZimaCube 2 when a broader multi-drive workload justifies it.Fix the observed bottleneck
Select a supported CPU/GPU backend first. Add model weights, runtime buffers and KV cache; host RAM and GPU VRAM are different budgets. Zima hardware is not a validated CUDA inference host.
Check service configuration, I/O and network behavior before selecting a stronger CPU.Can it run on ZimaOS?
Official one-click ZimaOS availability has not been certified for this guide. Use the supported upstream runtime and service topology; consider Custom Install only when a maintained compatible container and all required host capabilities are available.
Custom install Ray Serve in ZimaOS
Follow the service topology in the linked upstream deployment for the chosen edition; include every required database, worker and storage service. Persist: Persist model weights, tokenizer/configuration, downloaded artifacts and any server state.
Custom install Ray Serve in the ZimaOS app โUse the official Ray Serve deployment instructions
Keep the selected stable release, persistent paths and companion services aligned with upstream documentation.
Review Ray Serve official installation โCheck workload-specific deployment details
Select a supported CPU/GPU backend first. Add model weights, runtime buffers and KV cache; host RAM and GPU VRAM are different budgets. Zima hardware is not a validated CUDA inference host.
Review Ray Serve deployment details โWhich Zima hardware fits Ray Serve?
Supporting storage/control services; inference or generation requires a separately verified backend.
Which workload best matches your Ray Serve deployment?
Model artifacts, workflow files and supporting services
Use Zima for durable assets and validated supporting services while an appropriately sized compatible worker handles compute.
- Supporting storage optionZimaBoard 2 1664
These are practical workload recommendations, not official app requirements, compatibility certification, measured capacity limits or performance guarantees. Include all co-hosted services and independent backups.
| Zima hardware | Best for | Example workload | Core configuration | Recommended boundary | Next step |
|---|---|---|---|---|---|
| ZimaBoard 2 1664 | Model artifacts and supporting services | Retained assets, configurations and a verified compatible control layer; heavy inference/generation runs on its own supported worker. |
|
Select a supported CPU/GPU backend first. Add model weights, runtime buffers and KV cache; host RAM and GPU VRAM are different budgets. Zima hardware is not a validated CUDA inference host. | Get Now |
| ZimaCube 2 Pro | Model artifacts and supporting services | Retained assets, configurations and a verified compatible control layer; heavy inference/generation runs on its own supported worker. |
|
Select a supported CPU/GPU backend first. Add model weights, runtime buffers and KV cache; host RAM and GPU VRAM are different budgets. Zima hardware is not a validated CUDA inference host. | Get Now |
What the Press Says
Highlights from trusted reviewers worldwide.
โZimaCube 2: Not just another NAS, tested with 25TB storage, local AI agents, 4K transcoding, and real homelab workflows.โRead full review
โThe ZimaBoard 2 is a compact x86 server board that can be turned into a mini NAS, home server, media box, or self-hosting hub.โRead full review
โZimaCube 2: A modern, high-performance NAS with plenty of room to growโbuilt for users who want more than basic storage.โRead full review
โCoverage focused on ZimaCube 2's open hardware foundation, no monthly fee, and self-hosting flexibility.โRead full review
Loved by the Community
Stories and reviews from people who build with Zima every day.
โ
โ
โ
โ
โ
Zima Blade Little yet Powerful
Maybe I am not digital natives but I live with PCs since 12 years old in 1984 when IBM PC clone come to my home. Many years have passed and many operating system I've tried. For me Zima blade and CasaOS was a quantum leap for home PC enthusiast and server lab machine to make me stay curious and relevant for this era.
โ
โ
โ
โ
โ
Very good!!
I use ZimaCube Pro as 5th Proxmox cluster node. It runs several VMs and containers, including a VM with GPU passthrough to run a self-hosted LLM. A specific LXC container runs a Samba server for NAS capabilities using four of six RAID 6 SATA HDDs with ZFS.
โ
โ
โ
โ
โ
Great innovation for mini server!
It is very useful and makes a powerful mini server for many purposes, including university and college students in engineering and electronics. Thank you so much for making this server.
โ
โ
โ
โ
โ
Avaliaรงรฃo ZimaBoard 2
Construรญ um servidor de uso pessoal. O desempenho estรก muito bom e funciona perfeitamente onde quer que eu esteja. A surpresa รฉ nรฃo dependermos de grandes estruturas para termos nosso prรณprio servidor de dados. Como iniciante, estou gostando bastante do ZimaOS, pois ele รฉ simples e eficiente.
Frequently asked questions
Answers to Ray Serve RAM, CPU, storage, Docker, database and ZimaOS sizing questions.
How much RAM should I plan for Ray Serve?
The linked source does not establish a fixed numerical whole-host baseline for this complete workload. Size the full stack during the busy operation described below. Include ZimaOS, the application and its companion services rather than reserving the entire machine for one container.
What drives the CPU requirement?
The linked source does not establish a fixed numerical whole-host baseline for this complete workload. Measure CPU use during realistic activity. The workload to test is: weight precision, parameter count, context length, KV cache, concurrent sequences and tokens per second.
Which data must survive a container update?
Persist model weights, tokenizer/configuration, downloaded artifacts and any server state. Preserve a consistent database/file set and verify that your recovery procedure restores a working application.
What other services belong in the budget?
Follow the service topology in the linked upstream deployment for the chosen edition; include every required database, worker and storage service.
Does this workload need a dedicated GPU?
Select a supported CPU/GPU backend first. Add model weights, runtime buffers and KV cache; host RAM and GPU VRAM are different budgets. Zima hardware is not a validated CUDA inference host.
Can I deploy it on ZimaOS?
Use an amd64 container and the current upstream service topology through Custom Install when a suitable maintained image is available. Container availability alone is not a compatibility certification.
What should I check before upgrading hardware?
Select a supported CPU/GPU backend first. Add model weights, runtime buffers and KV cache; host RAM and GPU VRAM are different budgets. Zima hardware is not a validated CUDA inference host. Measure the same representative operation before and after a configuration or hardware change.
Which Zima device is a practical starting point?
Supporting storage/control services; inference or generation requires a separately verified backend. Choose the inference worker from the model/backend requirements rather than assuming an 8 GB or 16 GB NAS can run every model.
What sources informed this Ray Serve hardware guide?
This guide was reviewed against the linked project deployment documentation on October 30, 2026. Zima device choices are editorial workload guidance, separate from upstream requirements and measured performance.
