LLM fine-tuning workstation, sized to the training method.
Fine-tuning needs far more GPU memory than running the same model. How much depends on the method: full fine-tuning keeps gradients and optimizer state for every parameter, while LoRA and QLoRA train small adapters on a frozen base model. This page covers the arithmetic and which of our builds fits each case.
- ~16 bytesper parameter, full fine-tuning
- ~2 bytesper parameter, LoRA on a 16-bit base
- ~0.5 bytesper parameter, QLoRA on a 4-bit base
- 384GBVRAM, 4× RTX PRO 6000
GPU memory by fine-tuning method
Approximate memory for the model state alone. Activations add to every figure and grow with sequence length and batch size.
| Method | Held in GPU memory | Per parameter |
|---|---|---|
| Full fine-tuning, mixed precision, Adam | 16-bit weights and gradients, a 32-bit master copy of the weights, and two 32-bit optimizer states | About 16 bytes |
| LoRA | Frozen 16-bit base weights, plus small trainable adapter matrices and their optimizer state | About 2 bytes, plus adapters |
| QLoRA | Frozen base weights quantized to 4-bit, plus 16-bit adapters | About 0.5 bytes, plus adapters |
Full fine-tuning: 2 bytes (weights) + 2 (gradients) + 4 (32-bit master weights) + 8 (two Adam moments) = 16 bytes per parameter. Sharding across GPUs, gradient checkpointing, and memory-saving optimizers lower the per-GPU figure; activations and framework overhead raise it.
Model size, method, and the build that fits
Rules of thumb for the model state with working headroom. Long sequences need more.
| Model | Method | Model state | Fits on |
|---|---|---|---|
| 7–8B | LoRA | about 16GB | One 32GB GPU: RTX 5090 or RTX PRO 4500 |
| 13–14B | LoRA | about 26–28GB | One 48GB RTX PRO 5000 or larger |
| 30–34B | QLoRA | about 15–17GB | One 32GB or 48GB GPU |
| 70B | QLoRA | about 35GB | One 72GB RTX PRO 5000 or 96GB RTX PRO 6000; a 48GB card at short sequence lengths |
| 70B | LoRA, 16-bit base | about 140GB | Two RTX PRO 6000 (192GB) |
| 7–8B | Full fine-tuning | about 112–128GB | Two RTX PRO 6000 (192GB), with sharded training |
| 70B | Full fine-tuning | about 1.1TB | Beyond a four-GPU tower: see AI servers |
Training on more than one GPU
Data parallel
Each GPU holds a full copy of the trainable state and works through its own batches; gradients are averaged across the cards every step. It shortens training when the model already fits on one GPU.
Sharded training
Sharding, for example with FSDP or DeepSpeed ZeRO, splits weights, gradients, and optimizer state across the GPUs, so the model state can be larger than any single card. That is how full fine-tuning of a 7B or 8B model fits on two or four GPUs.
A full x16 link per GPU
Both approaches move data between GPUs every step. In the JNCS AI PRO and AI MAX every GPU runs on a dedicated PCIe 5.0 x16 slot: 80 lanes on Threadripper 9000, 128 on Threadripper PRO 9000.
The same card in every slot
Each build uses one, two, or four of the same GPU, so memory and speed match across cards and the work divides evenly.
Checkpoints, memory, and long runs
Checkpoints
A full fine-tuning checkpoint that includes optimizer state is roughly 12 to 16 bytes per parameter: about 85GB to 110GB for a 7B model. Every build offers Samsung 9100 PRO PCIe 5.0 NVMe drives of 1TB to 8TB in several M.2 slots.
System memory
Data loading, tokenization, and CPU offload use system RAM. A common guideline is system memory at least equal to total VRAM; the AI PRO and AI MAX use registered ECC DDR5 up to 512GB and 1TB.
Bulk storage
Optional Seagate Exos 7200 RPM enterprise hard drives, 12TB to 28TB each, hold raw datasets and older checkpoints, as independent volumes or a RAID 1 or RAID 0 pair.
Multi-day runs
Every system is burn-in tested under sustained full load before it ships, so a training run that lasts days is not the first time the machine has been held at full power.
Power for training loads
Training holds every GPU near full power for hours at a time.
A single-GPU system, or two RTX PRO 4500 or 5000 cards, runs on a standard 15A, 120V circuit. Two RTX PRO 6000 cards, or any four-GPU configuration, require a dedicated 20A, 110–120V circuit, with a 30A, 220–240V circuit recommended. Four RTX PRO 6000 cards draw approximately 2,850W and require a dedicated 30A, 220–240V circuit. The configurator shows the requirement for every configuration as you build it.
Assembly, burn-in, and delivery
Each system is assembled by hand at our shop in Fairport, NY, by a shop that has built computers for more than thirty years.
Fit and power validated
The configurator checks physical fit, memory, PCIe slots, and power on every selection. The power supply is sized automatically to the GPU count and estimated draw, with a 12V-2x6 cable for every GPU.
Burn-in under load
Every system is stress-tested under sustained full load and firmware-updated before it leaves the bench, so the first long job is not the first test.
Windows 11 Pro installed
Windows 11 Pro is installed and updated before shipment.
Warranty and support
Every build carries a 3-year warranty and lifetime technical support from the shop that built it.
Quoting and payment
The configurator shows a price for every part and an estimated total, including assembly, burn-in, and testing, as the build is configured. GPU and memory pricing changes frequently, so a technician confirms GPU availability and final pricing by email, with lead time, typically within 1-2 business days. There is no cost and no obligation to request a quote.
Payment for AI workstations is by ACH bank transfer; credit cards are not accepted for these systems. An invoice with bank details is issued once the quote is approved. Systems ship across the United States from Fairport, NY, with local pickup available.
LLM fine-tuning hardware questions
How much VRAM do I need to fine-tune an LLM?
Can I fine-tune a 70B model on one GPU?
Is an RTX 5090 good for fine-tuning?
Do multiple GPUs help with fine-tuning?
How much storage does a fine-tuning workstation need?
How is an AI workstation ordered and paid for?
Related AI workstation pages
Local LLM workstation guide
How much VRAM a model needs, and which build provides it.
Read →Multi-GPU AI workstation
Up to four RTX PRO Blackwell GPUs, up to 384GB VRAM, Threadripper PRO 9000.
Read →Dual RTX PRO 6000 workstation
192GB of VRAM on two 96GB GPUs, and the circuit it needs.
Read →Deep learning workstation
Choosing GPUs, CPU cores, memory, and storage for training.
Read →RTX PRO 6000 Blackwell workstation
One or two RTX PRO Blackwell GPUs, up to 192GB VRAM, Threadripper 9000.
Read →Size the build to the training job.
Select a build and GPU count. The configurator shows total VRAM, system memory, estimated power draw, and the circuit the system needs as you go.
Have an exact parts list or a department spec sheet? Send the whole list. For more than four GPUs or rackmount systems, see AI servers and larger systems.