This can multiply VRAM demand several times without changing model parameter count. For a 7B model, the official table rises from roughly 6 GB at 4-bit QLoRA to 16 GB LoRA, about 60 GB pure-bf16 full tuning and roughly 120 GB for the regular full-training path.
For users increasing training fidelity, updating more parameters or moving away from aggressive quantization.LLaMA Factory GPU Hardware Requirements: VRAM, RAM & Storage
Learn LLaMA Factory GPU hardware requirements for VRAM, CUDA, LoRA, QLoRA, full fine-tuning, model size and ZimaOS hardware.
LLaMA Factory GPU hardware requirements at a glance
LLaMA Factory has no single VRAM minimum because training memory depends on model size and fine-tuning method. The official project publishes estimated GPU-memory tables for full fine-tuning, LoRA and 2/4/8-bit QLoRA.
- GPU
- The current ZimaOS package is LLaMA Factory(Nvidia GPU), so a compatible NVIDIA GPU is the practical deployment gate for this App. Upstream LLaMA Factory also supports CUDA, ROCm and Ascend paths, but those should not be assumed for the NVIDIA-specific ZimaOS package.
- VRAM
- Official estimated VRAM varies sharply by method: 7B LoRA about 16 GB, 7B 4-bit QLoRA about 6 GB, 14B LoRA about 32 GB, 14B 4-bit QLoRA about 12 GB, 30B 4-bit QLoRA about 24 GB and 70B 4-bit QLoRA about 48 GB.
- Full fine-tuning
- Full-parameter training is dramatically heavier. The official table estimates roughly 60 GB for 7B pure-bf16 full tuning and around 120 GB for the regular bf16/fp16 full-training path; larger models scale rapidly from there.
- System RAM
- The upstream project does not publish one universal system-RAM minimum. Host memory must cover the OS, Python/PyTorch process, dataloader workers, dataset preprocessing, optimizer offload if used and other ZimaOS services in addition to GPU VRAM.
- Storage
- No universal disk minimum is published. Plan SSD/NVMe capacity for base model weights, datasets, checkpoints, LoRA adapters, merged models, Hugging Face caches, Docker layers and repeated experiment outputs.
- Best Zima starting point
- For the current NVIDIA GPU App, ZimaCube 2 Creator Pack is the most direct preconfigured Zima path because it includes a dedicated NVIDIA RTX PRO 2000. Verify the exact installed GPU VRAM before matching it to any official 6/12/16/24/48-GB training tier.
From official requirements to the right setup
LLaMA Factory sizing starts with model parameter count and fine-tuning method, then checks GPU VRAM, CUDA compatibility, host RAM, checkpoint storage and dataset throughput.
-
Official requirements
Choose the exact model size first: 7B, 14B, 30B, 70B or another parameter class. A hardware recommendation without model size is not meaningful for fine-tuning.
-
Confirm your needs
Choose the tuning method next. Full fine-tuning, pure-bf16 full tuning, LoRA/freeze tuning and 2/4/8-bit QLoRA have very different official estimated VRAM requirements.
-
Leave room to grow
Validate the GPU software path. Current upstream requirements list CUDA 11.6 minimum and 12.2 recommended, while the current prebuilt CUDA Docker image is based on Ubuntu 22.04, CUDA 12.4, Python 3.11 and PyTorch 2.6.0.
-
Run it on ZimaOS
Run a representative training job and monitor peak VRAM, system RAM, checkpoint growth and GPU utilization. Leave headroom for sequence length, batch size, gradient checkpointing, optimizer state and multimodal inputs rather than sizing exactly to an estimated table value.
Check every playback client
- Exact model size and architecture
- Full tuning, LoRA, QLoRA or another method
- Quantization level: 2-bit, 4-bit, 8-bit or none
- Exact NVIDIA GPU and available VRAM
- CUDA, PyTorch and container GPU compatibility
- Sequence length, batch size and gradient accumulation
- Dataset, checkpoint and model-cache storage
- Other ZimaOS containers and GPU workloads
Official minimum requirements
LLaMA Factory publishes software dependency minimums plus an estimated GPU-memory matrix rather than one universal hardware minimum. GPU memory must be selected from the target model and fine-tuning method.
Do not publish a generic 'LLaMA Factory needs 16 GB VRAM' rule. Sixteen gigabytes is only one official estimateโfor example 7B LoRAโwhile a 7B 4-bit QLoRA job is estimated around 6 GB and full-parameter 7B training can require tens to more than 100 GB.
| Requirement | Official minimum | What this supports |
|---|---|---|
| 7B model | LoRA 16 GB; 4-bit QLoRA 6 GB; 2-bit QLoRA 4 GB | These are official estimated GPU-memory figures. Full training is much heavier: about 60 GB pure-bf16 or 120 GB in the regular bf16/fp16 full-training table. |
| 14B model | LoRA 32 GB; 4-bit QLoRA 12 GB; 2-bit QLoRA 8 GB | The upstream table estimates roughly 120 GB for 14B pure-bf16 full tuning and 240 GB for the regular full-training path. |
| 30B model | LoRA 64 GB; 4-bit QLoRA 24 GB; 2-bit QLoRA 16 GB | At this size, even parameter-efficient training can exceed a single consumer GPU and may require a larger card or multi-GPU configuration. |
| 70B model | LoRA 160 GB; 4-bit QLoRA 48 GB; 2-bit QLoRA 24 GB | The official table illustrates why 70B fine-tuning usually requires high-memory or multi-GPU hardware even with quantization. |
| CUDA | 11.6 minimum; 12.2 recommended | Current project requirements list these upstream CUDA versions. The current prebuilt CUDA Docker image itself is based on CUDA 12.4, so deployment compatibility should be verified against the actual container/image used. |
| Core software | Python 3.11+, PyTorch 2.0+; current recommended PyTorch 2.6.0 | The current project requirement table also lists compatible Transformers, Datasets, Accelerate, PEFT and TRL versions plus optional bitsandbytes, DeepSpeed, vLLM and FlashAttention. |
When to upgrade your hardware
Upgrade LLaMA Factory hardware when the selected model/method crosses a VRAM tier or when training stability and throughput show that the current GPU has insufficient headroom.
You move from 4-bit QLoRA to LoRA or full tuning
You move from 7B/14B into 30B or 70B models
Sequence length, batch size or multimodal data grows
Official 4-bit QLoRA estimates rise from about 6 GB at 7B and 12 GB at 14B to 24 GB at 30B and 48 GB at 70B. LoRA rises even faster, making multi-GPU or data-center-class memory increasingly likely.
For users scaling model capability while keeping the same training method.The upstream hardware table is explicitly estimated and cannot capture every runtime variable. Longer sequences, larger batches, optimizer states and vision/audio inputs can raise peak VRAM and system-memory use beyond the simple parameter-size tier.
For long-context fine-tuning, VLM datasets, multimodal training and throughput-oriented batch increases.Plan hardware growth with confidence
LLaMA Factory scales best when GPU memory, host memory, model/checkpoint storage and training datasets are planned as separate resources.
Use QLoRA before buying substantially more VRAM
LLaMA Factory officially supports 2/3/4/5/6/8-bit QLoRA paths. Four-bit QLoRA can reduce the estimated 7B GPU requirement to about 6 GB and a 14B model to about 12 GB, although training speed and exact compatibility still depend on the model/backend.
Use quantization and gradient checkpointing when the training goal allows it, then validate quality and speed before moving to a much larger GPU.Put models and checkpoints on fast SSD/NVMe
Fine-tuning repeatedly reads base weights, datasets and caches and writes checkpoints, adapters and merged models. Slow or capacity-limited storage creates a separate bottleneck even when VRAM is sufficient.
Use NVMe or fast SSD storage for active models, Hugging Face caches, training datasets and checkpoints; archive older runs to larger capacity disks.Use multi-GPU only when the workload justifies it
LLaMA Factory supports DeepSpeed, FSDP and distributed examples. Multi-GPU training can make larger LoRA/full-tuning jobs possible, but introduces communication, power, cooling and software complexity.
Move to multi-GPU when one GPU cannot meet the target method/model tier or when training throughput is a measured business requirement.Separate the fine-tuning host from the home-server control plane
Training can saturate GPU, CPU, RAM and storage for hours. Keeping file services and ordinary ZimaOS apps separate from the training workload can improve stability and reduce resource contention.
Use Zima storage for datasets/checkpoints and a larger external GPU workstation or cloud node when the target model exceeds the installed local GPU.Can it run on ZimaOS?
ZimaOS currently lists a dedicated LLaMA Factory(Nvidia GPU) App. Because this package is explicitly NVIDIA-focused, the GPU and VRAM should be verified before using the official LLaMA Factory memory table to plan a training job.
Install LLaMA Factory(Nvidia GPU) from ZimaOS
The current ZimaOS App Store lists LLaMA Factory(Nvidia GPU) in the AI category for unified LLM fine-tuning. Treat the package as a GPU training environment rather than a lightweight CPU-only web application.
Open LLaMA Factory(Nvidia GPU) in the ZimaOS App StoreVerify the NVIDIA driver and container GPU path
Upstream CUDA support is only useful if the ZimaOS host exposes the installed NVIDIA GPU correctly to the container. Confirm nvidia-smi, available VRAM and PyTorch CUDA visibility before selecting a training recipe.
Read LLaMA Factory installation guidanceMatch the exact training recipe to available VRAM
Do not infer model support only from the App launching successfully. Compare the installed GPU's usable VRAM with the upstream model/method table and leave headroom for sequence length, batch size and other runtime memory.
Read the official LLaMA Factory hardware tableChoose Zima hardware for your LLaMA Factory workload
For the current ZimaOS NVIDIA App, GPU VRAM is the first hardware gate. ZimaBoard 2 and non-Creator ZimaCube 2 models can act as dataset, storage and controller platforms or accept a separately validated GPU expansion, while Creator Pack is the current preconfigured dedicated-GPU option.
Does the installed NVIDIA GPU have enough verified VRAM for your exact model and fine-tuning method?
No โ the Zima system has no suitable GPU or the VRAM is below the target tier
Use the Zima device for dataset storage, experiment management and model artifacts, then train on an external GPU workstation or cloud node. ZimaBoard 2 and ZimaCube 2 Standard/Pro should not be presented as complete LLaMA Factory GPU-training systems as shipped.
- Dataset/controller host with PCIe expansion pathZimaBoard 2 1664
- Large dataset/checkpoint storage plus stronger controller CPUZimaCube 2 Pro
Yes โ a compatible NVIDIA GPU is installed and its VRAM tier is verified
Use the official VRAM table to select 7B/14B/30B/70B and LoRA/QLoRA/full tuning. Creator Pack is the easiest preconfigured Zima GPU path, but its exact current RTX PRO 2000 VRAM must be verified before promising any specific training tier.
- Preconfigured NVIDIA GPU path for QLoRA/LoRA experimentsZimaCube 2 Creator Pack
- User-added NVIDIA GPU plus multi-drive dataset storageZimaCube 2 Pro
- Compact external-GPU experiment host where expansion is validatedZimaBoard 2 1664
This is a workload guide, not a guaranteed fine-tuning benchmark. LLaMA Factory's VRAM table is explicitly estimated; actual use depends on architecture, quantization, sequence length, batch size, optimizer, gradient checkpointing, multimodal inputs, CUDA/PyTorch versions and other ZimaOS workloads.
| Zima hardware | Best for | Example workload | Core configuration | Recommended boundary | Next step |
|---|---|---|---|---|---|
| ZimaBoard 2 832 | Dataset preparation, experiment control and remote/cloud LLaMA Factory workflows rather than substantial local GPU fine-tuning as shipped. | Downloading datasets/models, managing YAML recipes, storing adapters and controlling a separate GPU training machine. |
|
Eight GB system RAM and 32 GB eMMC are also tight for model preparation and repeated checkpoints. Do not present the stock 832 as a ready LLM fine-tuning workstation. | Get Now |
| ZimaBoard 2 1664 | A compact LLaMA Factory controller/data-prep host with more system RAM and a PCIe expansion path to a separately powered, validated NVIDIA GPU. | Model/dataset management, LLaMA Board UI, experiment setup, smaller preprocessing jobs and remote or expanded GPU training. |
|
The N150 and 16 GB host RAM are useful for control/preprocessing, but the actual trainable model size is still determined primarily by the attached GPU's VRAM and training method. | Get Now |
| ZimaCube 2 Standard | Large model/dataset storage and remote fine-tuning workflows, or a custom user-added GPU build. | Datasets, checkpoints, experiment archives, remote/cloud training coordination and other self-hosted AI services. |
|
Eight GB system RAM is not ideal for heavier training preprocessing, and stock hardware does not satisfy the NVIDIA-specific App's local training requirement by itself. | Get Now |
| ZimaCube 2 Pro | A stronger LLaMA Factory controller and dataset/checkpoint server with PCIe expansion for a separately selected NVIDIA GPU. | Large training datasets, experiment archives, preprocessing, model transfer over 10GbE and user-added GPU fine-tuning. |
|
Training capability depends on the added GPU. A stronger CPU and 10GbE do not compensate for insufficient GPU VRAM. | Get Now |
| ZimaCube 2 Creator Pack | The most direct current Zima configuration for LLaMA Factory(Nvidia GPU), with a dedicated NVIDIA GPU, 64 GB system RAM and large SSD capacity. | Local QLoRA/LoRA experimentation, model preparation, dataset processing, checkpoint storage and GPU-accelerated AI workloads when the exact GPU VRAM fits the chosen recipe. |
|
The current ZimaCube 2 product page names RTX PRO 2000 but does not publish VRAM in its text specification. Therefore do not promise even the official 7B LoRA 16-GB tier until the exact installed GPU memory is confirmed. | Get Now |
What the Press Says
Highlights from trusted reviewers worldwide.
โZimaCube 2: Not just another NAS, tested with 25TB storage, local AI agents, 4K transcoding, and real homelab workflows.โRead full review
โThe ZimaBoard 2 is a compact x86 server board that can be turned into a mini NAS, home server, media box, or self-hosting hub.โRead full review
โZimaCube 2: A modern, high-performance NAS with plenty of room to growโbuilt for users who want more than basic storage.โRead full review
โCoverage focused on ZimaCube 2's open hardware foundation, no monthly fee, and self-hosting flexibility.โRead full review
Loved by the Community
Stories and reviews from people who build with Zima every day.
โ
โ
โ
โ
โ
Zima Blade Little yet Powerful
Maybe I am not digital natives but I live with PCs since 12 years old in 1984 when IBM PC clone come to my home. Many years have passed and many operating system I've tried. For me Zima blade and CasaOS was a quantum leap for home PC enthusiast and server lab machine to make me stay curious and relevant for this era.
โ
โ
โ
โ
โ
Very good!!
I use ZimaCube Pro as 5th Proxmox cluster node. It runs several VMs and containers, including a VM with GPU passthrough to run a self-hosted LLM. A specific LXC container runs a Samba server for NAS capabilities using four of six RAID 6 SATA HDDs with ZFS.
โ
โ
โ
โ
โ
Great innovation for mini server!
It is very useful and makes a powerful mini server for many purposes, including university and college students in engineering and electronics. Thank you so much for making this server.
โ
โ
โ
โ
โ
Avaliaรงรฃo ZimaBoard 2
Construรญ um servidor de uso pessoal. O desempenho estรก muito bom e funciona perfeitamente onde quer que eu esteja. A surpresa รฉ nรฃo dependermos de grandes estruturas para termos nosso prรณprio servidor de dados. Como iniciante, estou gostando bastante do ZimaOS, pois ele รฉ simples e eficiente.
Frequently asked questions
These answers use LLaMA Factory's current upstream GPU-memory table as the primary source and separate official estimates from GPU-provider and third-party planning advice.
How much VRAM does LLaMA Factory need?
There is no single VRAM minimum. Current official estimates depend on model size and fine-tuning method. For 7B, LoRA is about 16 GB and 4-bit QLoRA about 6 GB. For 14B, LoRA is about 32 GB and 4-bit QLoRA about 12 GB. For 30B 4-bit QLoRA is about 24 GB, while 70B 4-bit QLoRA is about 48 GB.
Is 16 GB VRAM enough for LLaMA Factory?
It is enough for some official estimated tiers, such as 7B LoRA, 7B QLoRA and approximately 30B 2-bit QLoRA. It is not a universal threshold: 14B LoRA is estimated around 32 GB, and 30B 4-bit QLoRA around 24 GB.
Can LLaMA Factory fine-tune a 7B model on an 8 GB GPU?
Potentially with an efficient method. The official table estimates 7B 4-bit QLoRA at about 6 GB and 2-bit QLoRA at about 4 GB, but actual peak memory depends on architecture, sequence length, batch size and other settings. Leave practical headroom rather than targeting the estimate exactly.
How much VRAM does 14B fine-tuning need?
Current upstream estimates are approximately 32 GB for LoRA, 20 GB for 8-bit QLoRA, 12 GB for 4-bit QLoRA and 8 GB for 2-bit QLoRA. Full tuning is far heavier, with pure-bf16 estimated around 120 GB.
Does LLaMA Factory require an NVIDIA GPU?
Upstream LLaMA Factory supports multiple accelerator paths, including CUDA, AMD ROCm and Ascend NPU. However, the current ZimaOS package is explicitly LLaMA Factory(Nvidia GPU), so this ZimaOS deployment should be planned around a compatible NVIDIA GPU and CUDA/container path.
Can ZimaBoard 2 run LLaMA Factory GPU training?
Not as a complete stock GPU-training system. ZimaBoard 2 has PCIe expansion and can act as a dataset/controller host or support a custom externally powered GPU setup, but the stock board has no dedicated NVIDIA GPU. The attached GPU's VRAMโnot the N150 or system RAMโsets the fine-tuning tier.
Why are Clore.ai's GPU recommendations higher than LLaMA Factory's 4-bit QLoRA table?
Clore.ai recommends RTX 3090-class hardware as a practical minimum for a 7B LLM fine-tuning cloud workload and A100 40 GB as recommended, while upstream LLaMA Factory estimates that a narrowly configured 7B 4-bit QLoRA job can fit around 6 GB VRAM. Clore's figures include a more comfortable provider-oriented training target; the upstream table is the canonical method-specific minimum estimate.
When should I choose ZimaCube 2 Creator Pack for LLaMA Factory?
Choose Creator Pack when local NVIDIA training is genuinely part of the workload and you also benefit from 64 GB system RAM, 1 TB system storage and multi-drive dataset/checkpoint capacity. Verify the exact RTX PRO 2000 VRAM before matching it to the official 6/12/16/24/48-GB training tiers.
What sources and further reading informed this LLaMA Factory GPU hardware guide?
The official hiyouga/LlamaFactory repository is the primary authority for current software requirements and the estimated VRAM matrix across full tuning, LoRA and QLoRA. The official Read the Docs installation page supports the CUDA-based setup guidance but is less complete than the GitHub hardware table. Clore.ai provides cloud-GPU deployment guidanceโsuch as RTX 3090 for 7B fine-tuning and A100 40 GB as a stronger targetโwhich is useful operational context but not a replacement for upstream method-specific VRAM estimates. VoltAgent correctly explains that hardware requirements vary strongly by model and fine-tuning method and points readers back to the upstream hardware table; its article is explanatory rather than canonical.
