Answer · confidence: partial
How has the biggest AI you can run at home changed in recent years?
Research question: How has the size of the largest AI model a person can run on a home computer changed over the past few years?
The records show a rise from roughly 50 billion parameters on consumer GPUs in 2024, when SpecExec ran such models with RAM offloading at 4–6 tokens per second using 4-bit quantization 25. By 2026, a 70-billion-parameter model ran on Apple Silicon unified memory at about 18.9 tokens per second without multiple GPUs 1, and a vendor claim said a Ryzen AI Max PRO 400 system could handle 300-billion-parameter models at 4-bit with a 160GB allocation 22. The facts do not give a consistent year-by-year record of the largest model a home user can run, and the hardware and claims differ, so this is only a partial picture. The newest facts are dated September 2026, about one month before the current date of October 2026 122.
Checked 2026-10-21. Only documented and supported facts are used.
Dig deeper on this questionSources
[1] supported
Apple Silicon unified memory can run 70-billion-parameter models (Llama-3-70B at Q4_K_M) at about 18.9 tokens per second without needing multiple GPUs.
"Apple Silicon unified memory successfully executes 70-billion-parameter models (Llama-3-70B at Q4_K_M) at an interactive 18.9 tokens per second natively without requiring multi-GPU tensor-parallel clustering."Quantization, Memory Bandwidth Bottlenecks, and Speculative Decoding in Local Large Language Model (LLM) Inference: An Empirical Benchmark of 4-bit/8-bit Transformers (Llama 3, Mistral) on Consumer GPU, Apple Silicon, and Embedded NPU Architectures (2026)
[2] supported
Using the SpecExec method, the authors ran models with more than 50 billion parameters on consumer GPUs with RAM offloading at 4–6 tokens per second with 4-bit quantization.
"UsingSpecExec,wedemonstrateinferenceof50B+parameter"SpecExec: Massively Parallel Speculative Decoding for Interactive LLM Inference on Consumer Devices (2024)
[3] supported
Qwen's 72B model weights were released for download in December 2023, the first time its largest model of that era was made available openly.
"In December 2023, it released its 72B and 1.8B models for download, while Qwen 7B weights were released in August."Qwen
[4] supported
The benchmarked Apple M3 Max high-capacity unified memory configuration has 128 GB of memory.
"high-capacity unified memory (Apple M3 Max, 128 GB Unified LPDDR5X, 400 GB/s bandwidth)"Quantization, Memory Bandwidth Bottlenecks, and Speculative Decoding in Local Large Language Model (LLM) Inference: An Empirical Benchmark of 4-bit/8-bit Transformers (Llama 3, Mistral) on Consumer GPU, Apple Silicon, and Embedded NPU Architectures (2026)
[5] supported
Running the largest language models on consumer hardware requires either extreme compression or offloading to CPU RAM.
"Runningthelargestlanguagemodelsinthissetuprequireseitherextremecompressionoroffloading."SpecExec: Massively Parallel Speculative Decoding for Interactive LLM Inference on Consumer Devices (2024)
[6] supported
The February 2025 version of the llama.cpp README listed supported GLM models ranging from 1.5 billion to 9 billion parameters, which is only a rough indication of model sizes and does not show how the largest runnable model has changed.
"ChatGLM3-6b + ChatGLM4-9b + GLMEdge-1.5b + GLMEdge-4b"What changed on github.com/ggml-org/llama.cpp between 2025-02-16 and 2026-10-03
[7] supported
The Qwen3 family released in April 2025 had dense models up to 32B parameters and a mixture-of-experts model with 235B total parameters but only 22B active.
"The sizes of the dense models include 0.6B, 1.7B, 4B, 8B, 14B, and 32B, and the MoE models include 30B-A3B (30B with 3B activated parameters) and 235B-A22B (235B with 22B activated parameters)."Qwen
[8] supported
The benchmarked discrete consumer GPU, the NVIDIA GeForce RTX 4090, has 24 GB of GDDR6X memory.
"discrete consumer GPU (NVIDIA GeForce RTX 4090, 24 GB GDDR6X, 1,008 GB/s bandwidth)"Quantization, Memory Bandwidth Bottlenecks, and Speculative Decoding in Local Large Language Model (LLM) Inference: An Empirical Benchmark of 4-bit/8-bit Transformers (Llama 3, Mistral) on Consumer GPU, Apple Silicon, and Embedded NPU Architectures (2026)
[9] supported
In January 2025, the Qwen2.5-VL vision-language line came in sizes of 3, 7, 32, and 72 billion parameters.
"In January 2025, Qwen2.5-VL was released with variants of 3, 7, 32, and 72 billion parameters."Qwen
[10] supported
Qwen's latest flagship open-weights model is a 2.4 trillion parameter model, the second largest open-weights LLM released as of August 2026, behind Kimi K3.
"Its latest version, Qwen3.8; the 2.4 trillion parameter model was the second largest and second most powerful open weights LLM and Chinese LLM released by 12 August 2026, after Kimi K3 ."Qwen
[11] supported
A distilled 27B version of Qwen3.8 was released under the more permissive Apache License, making a much smaller model available for broad use alongside the 2.4 trillion parameter model.
"The distilled Qwen3.8 27B was released under a more permissive Apache License ."Qwen
[12] supported
Quantization reduces numerical precision from 32-bit to lower-bit formats such as 4-bit, which is described as a way to shrink model size.
"quantization—reducing numerical precision from 32-bit to lower-bit formats such as 4-bit—has emerged as an effective compression method to minimize model size and enhance inference efficiency."A Survey on Quantization Techniques for Large Language Models (2026)
[13] supported
The sizes of transformer-based language models increased dramatically in the years before 2023.
"Along with the performance improvement in NLP domain, the sizes of transformer-based language models (TLM) are also dramatically increased."Boost Transformer-based Language Models with GPU-Friendly Sparsity and Quantization (2023)
[14] supported
The review reports that quantization effectively reduces model size and memory consumption while keeping accuracy near the original level.
"Comparative analysis across selected papers highlights that quantization effectively reduces model size and memory consumption while maintaining near-original accuracy and significant inference speedups."A Survey on Quantization Techniques for Large Language Models (2026)
[15] supported
A typical laptop discrete GPU can see only about 8 to 24GB of its own VRAM, which limits the size of model it can load.
"On a standard laptop with a discrete GPU, the GPU can typically see 8–24GB of its own VRAM."Ryzen AI Max PRO 400 : 192GB Unified Memory Puts 300B - Parameter LLMs on Your Desk Without the Cloud (techtimes.com, 2026-09)
[16] supported
The guide's recommended largest model for a single 24 GB VRAM card is a 32B-parameter model such as Qwen3 32B or DeepSeek-R1-Distill-32B.
"24 GB VRAM : Qwen3 32B or DeepSeek-R1-Distill-32B"The Best Local LLM Models to Run in 2026
[17] supported
A 70-billion-parameter model at FP16 precision needs about 140GB, far beyond any discrete laptop card, but at 4-bit quantization it shrinks to roughly 38 to 40GB.
"Loading a 70-billion-parameter model at FP16 precision requires approximately 140GB — far beyond any discrete laptop card. At 4-bit quantization, a lossy compression, that same 70B model drops to roughly 38–40GB, which fits in a 48GB workstation card but still exceeds most portable hardware."Ryzen AI Max PRO 400 : 192GB Unified Memory Puts 300B - Parameter LLMs on Your Desk Without the Cloud (techtimes.com, 2026-09)
[18] supported
The guide says a workstation with 48 GB or more of VRAM can run 70B-parameter models such as Llama 3.3 70B and DeepSeek-R1 Distill 70B.
"48 GB+ VRAM : Llama 3.3 70B for general use, DeepSeek-R1 Distill 70B for harder reasoning"The Best Local LLM Models to Run in 2026
[19] supported
NVIDIA's DGX Spark, a deskside personal AI computer shipping since late 2025, is said to run models of up to 200 billion parameters.
"NVIDIA says it can run models of up to 200 billion parameters."Ryzen AI Max PRO 400 : 192GB Unified Memory Puts 300B - Parameter LLMs on Your Desk Without the Cloud (techtimes.com, 2026-09)
[20] supported
Mixtral 8x7B has 47B total parameters but only 13B active per token, yet still needs about 26 GB of VRAM to load its weights at Q4.
"The catch is simple: you still need enough VRAM to load the weights, which comes to about 26 GB at Q4."The Best Local LLM Models to Run in 2026
[21] supported
Q4_K_M quantization makes a model about 50% smaller than 8-bit with only about a 1–3% drop on benchmark scores, which is why it lets larger models fit in limited memory.
"Q4_K_M is the usual pick because it cuts model size a lot with only a small hit to output quality - about 50% smaller than 8-bit, with only around a 1–3% drop on benchmark scores."The Best Local LLM Models to Run in 2026
[22] supported
The 300-billion-parameter claim applies specifically at 4-bit quantization, which the article estimates needs about 150GB of weights within a 160GB VRAM allocation.
"AMD's 300-billion-parameter claim is specifically at 4-bit quantization."Ryzen AI Max PRO 400 : 192GB Unified Memory Puts 300B - Parameter LLMs on Your Desk Without the Cloud (techtimes.com, 2026-09)