Capacity belongs in affordable storage.
The complete model should not need to occupy the most expensive memory tier.
Large language models are learned data structures: enormous collections of learned information and capability. Today, that information is stored and accessed inefficiently, forcing increasingly large models onto increasingly expensive GPU systems.
We are building models that keep most of their capacity in affordable storage and activate only what each request needs.
Much larger models. Much smaller machines.
A 750M-parameter GLM, served primarily from disk, achieved:
1,300 prefill tokens per second on a single NVIDIA RTX 5090
GPU memory should be the working set, not the model boundary.
A language model stores learned information, associations, skills and behaviour. Current architectures tightly couple that stored capacity to expensive accelerator memory and per-request computation.
We are separating them.
The complete model should not need to occupy the most expensive memory tier.
Each request should use a small relevant part of a much larger model.
Total parameters determine capacity. Active parameters determine serving cost.
AI inference is not only limited by arithmetic. It is limited by where model parameters are stored and how much data must move through expensive memory.
GPU memory is fast, but limited and costly. Storage is abundant and affordable. Models should be designed around this hierarchy rather than forced entirely into its smallest tier.
The memory hierarchy — width indicates relative capacity
Model capacity is designed to live across the memory hierarchy rather than entirely inside accelerator memory.
Only the parameters relevant to the current request are activated.
The model and runtime work together to move and execute a compact active working set efficiently.
This allows total model capacity to grow without requiring proportional growth in GPU memory or per-request computation.
Model capacity can exceed the memory of the GPU serving it.
Serving cost is tied to the active working set rather than the complete stored model.
High-capacity models become practical on smaller private systems and more accessible infrastructure.
We are looking for researchers and engineers working across language models, sparse architectures, model distillation, storage systems, compilers, GPU kernels and efficient inference.
For partnerships, research, investment and general enquiries.