Running Large Local Open Source and Open Weights LLM Models on a Quad RTX-3060 12GB VRAM Threadripper 64GB AI Workstation with 48GB VRAM Each for CrewAI Agents running in a Home Lab

,

During my Summer break from teaching at Northwest University, I upgraded my home AI workstations with more memory to run larger local Open Source and Open Weights 48GB LLM Models using CrewAI.  I also needed to bake lighting in some of my very large Unity 3D open world environments (e.g. Exodus Vigil, Castle Quest) using a lot more Multi-GPU VRAM.

CrewAI is an open-source Python framework for building teams of AI agents that collaborate on a job, instead of stuffing everything into one giant prompt. You define people-like roles (“researcher,” “writer,” “reviewer”, “developer”, “tester”), give each a goal and tools, then CrewAI handles handoffs, context passing, and the process that runs them. CrewAI is not an LLM itself. It is the orchestration layer. The actual thinking and reasoning is done by whatever model you point it at.

I upgraded a couple of Azrock Taichi 399 AMD Threadripper AI Workstations with Quad Multi-GPU Gigabyte RTX-3060 12GB VRAM video cards with a total of 48GB VRAM in each system with 64GB of DDR4 system RAM.  This allowed me to run multiple Crew AI Agents on each system working simultaneously together on several projects that I was working on.  This allowed me to start baking my larger Unity 3D open world scenes using Bakery with multi-GPU light baking.   Now I can light bake very large levels and render larger and longer cutscene videos faster.

The largest models that fit comfortably in the Quad RTX-3060 with 48GB of VRAM are ~70B parameter model quantized to Q4_K_M or Q5_K_M.  That is roughly 35–45 GB total VRAM needed after overhead.  With 4 × 12GB = 48 GB total VRAM, I can run up to a 70B–90B model in Q4/Q5, or a smaller dense model like a good 34B–70B at higher quantization for better quality output.

The Azrock Taichi x399 Threadripper systems provide plenty of PCIe lanes for good multi-GPU scaling.  12GB per card is proving excellent for layer offloading using Ollama or LM Studio.  These factors have increased my local AI model sizing over my single GPU RTX-3080 10GB VRAM Alienware R11 system.  This is proving to be a very economical and capable local AI system for the AI work I want to do over my summer break.  So far, the multi-GPU systems are proving excellent for running large models for offline chatting, coding assistance, and light fine-tuning for coding.

I plan to test the following AI Models locally on it:

Rank
Model (Size)
Best For
Why It’s Good for Game Dev
VRAM Needed (Q4/Q5)
1
Llama 3.3 70B or Llama 4 70B
All-around best
Excellent coding, reasoning, long context
~40–45 GB
2
Qwen 2.5 Coder 32B
Programming & Scripting
Top-tier code generation (C#, C++, Blueprints, HLSL)
~20–25 GB
3
DeepSeek R1 or DeepSeek Coder V2 236B (MoE)
Complex coding & architecture
Outstanding for game systems & optimization
~35–45 GB
4
Command R+ (104B)
Creative writing & game design
Strong narrative, quests, dialogue, world-building
~45–50 GB
5
Mistral Large or Mixtral 8x22B
Balanced creative + technical
Good all-rounder for Unity/Unreal
~30–40 GB
For Unity development (C# focus):
  • Primary: Qwen 2.5 Coder 32B or Llama 3.3 70B
  • Great for Unity-specific tasks: Editor scripts, Addressables, DOTS/ECS, Shader Graph, UI Toolkit.
For Unreal Engine (C++ + Blueprints):
  • Primary: Llama 3.3/4 70B or DeepSeek Coder
  • Good for Unreal-specific: Niagara, Behavior Trees, GAS (Gameplay Ability System), Slate/UI, C++ performance optimization.

In case you are curious, here are the hardware components that make up my two Azrock Taichi 399 AMD Threadripper AI workstations:

 

I’ll follow up with another post at the end of summer to let you know how my local AI co-development went using these two AMD Threadripper Quad RTX-3060 48GB VRAM workstations and how many tokens/sec I got using each model.

 

/by