Quantized Model Checkpoint Size: Read the Bytes, Not Badges
Learn why model precision badges can mislead, how to calculate checkpoint size from repository files, and what to check before planning storage.
The Model Says 8-Bit. The Checkpoint Works Out to 4.49 Bits
A model page can show an "8-bit precision" badge, even when the checkpoint you'll download ends up much smaller than an 8-bit-per-parameter back-of-the-envelope would suggest. That's not automatically a contradiction inside the model files - it usually just means the badge, the model card, and the downloadable checkpoint are talking about different things.
I ran into this when I was estimating storage for Kimi K3.1 Pro. The model card listed MXFP4 weights and MXFP8 activations. The platform page, though, showed an 8-bit badge. But the checkpoint came as 96 top-level shards totaling about 1,561 GB. Divide that by the platform's reported 2.78 trillion parameters, and you get about 4.49 bits per parameter - not 8.
This matters when you're planning a download, picking a disk, or comparing formats. A label is useful context, but it can't tell you exactly how many bytes you'll store. If you want the real number, you have to look at the files you'll actually download, what's in the repo, and what your averaging calculation does (and doesn't) measure.
This article walks through that process, including an example where counting duplicate copies led to a confident but wildly wrong size estimate. It also points out a small arithmetic mismatch in one breakdown - because storage estimates only stay useful if the numbers reconcile.
Why can an 8-bit badge describe a smaller checkpoint
"Precision" is one of those words people use like it means one clear thing. In practice, a model page can group several different properties under a similar label - weight format, activation format, tensor storage type, or some platform-generated summary. They're related, but they aren't interchangeable.
For Kimi K3.1 Pro, the model card says MXFP4 weights and MXFP8 activations. Those labels describe different parts of computation. Meanwhile, the repo's platform badge says 8-bit. By itself, neither label tells you the exact total size of the checkpoint you'll download.
If you assume every one of the model's 2.78 trillion parameters takes eight bits, you'd estimate about 2,780 GB. If you use four bits per parameter, you'd get roughly 1,390 GB. The repo's selected 96 shards total about 1,561.3 GB. So yes, the gap from a simple four-bit estimate is real - but it doesn't prove the checkpoint is "secretly 8-bit."
One model can have several relevant precisions
Weight format is about how weights are stored. Activation precision is about values used during computation. The formats can differ because storing a model and running it are separate problems, with different tradeoffs for memory, speed, and output quality.
Also, checkpoints can include tensors that don't match the "main" quantized weight format. Some layers might stay at 16-bit or 32-bit (either to preserve quality or because quantizing small tensors isn't worth it). On top of that, quantized formats often carry extra metadata - block scales and similar bookkeeping - which take up space too.
So "4-bit model" is usually shorthand for the dominant weight representation, not a promise that every byte in the repository works out to exactly four bits per parameter. Once you add higher-precision tensors and quantization metadata, the average can end up above four bits.
A badge is a summary, not a disk measurement
The Kimi checkpoint and OpenAI's gpt-oss-120b repository were both reported as 8-bit by the platform, even though their MXFP4-related files or model details point elsewhere. That's helpful information, but it doesn't tell you how the badge is computed. If you don't know the platform's exact logic, it's guessing to tie the badge to a particular internal field.
One possible explanation (not a guaranteed one) is that packed 4-bit values live inside an 8-bit container. But the point is simpler: don't use the summary badge as your download-size calculation. Measure the files under the repository path and revision you plan to use.
How do you calculate checkpoint size from repository files
For a practical first-pass estimate, you need two inputs:
- the byte sizes of the checkpoint files you intend to download
- the model's reported parameter count
Add the bytes up, then compute the average storage cost using:
Bits per parameter = total checkpoint bytes × 8 ÷ parameter count
For the Kimi example, the selected files total 1,561,293,160,568 bytes. The platform reports 2,780,110,363,392 parameters. That comes out to about 4.4928 bits per parameter. This is an average across the selected files - it's not a claim that every tensor uses the same bit width.
Inspect file sizes through the repository API
The approach here uses the Hugging Face model API with blob info enabled, then sums the sizes of the intended Safetensors files. The exact script isn't the important part - the key is being explicit about which files count.
The API has to return a size for each file. If you only get filenames, you can't produce a byte total. Then you filter for the file format and repo location that match the download you're estimating.
In the Kimi repository, the check kept top-level Safetensors shards rather than files nested inside subdirectories.
A useful workflow is:
- Get the repository's file listing with per-file sizes.
- Select the exact files and revision you plan to download.
- Sum their byte sizes.
- Pull the parameter count from the model metadata.
- Calculate bytes per parameter / bits per parameter.
- Compare against the model's stated formats and dig into any big mismatch.
This is a storage estimate, not a benchmark of inference quality or speed. It doesn't directly tell you how much GPU memory the model needs, how fast it will run, or how good it will be.
Avoid counting duplicate copies
Repo structure can break an otherwise correct sum. A model repository is a directory, not necessarily "one set of weights." It may contain multiple formats, converted copies, examples, or files in subfolders. If you sum everything that looks like Safetensors, you might count the same underlying model more than once.
That happened with gpt-oss-120b. When all matching files were counted, the total came out to 195.8 GB and an average of 13.405 bits per parameter - clearly not right for what the repo is claiming. The repository included top-level MXFP4 shards plus copies under original/ and metal/, each about 65.2 GB. Restricting the count to the 15 intended top-level shards gave 65.2 GB and 4.4680 bits per parameter.
A "top-level only" rule won't work universally. It worked for that layout; another repo may put the intended files in a subfolder. The reliable rule is: match your calculation to the files your download command will actually fetch. When possible, use the same include/exclude patterns as your downloader and confirm you're selecting one encoding set rather than several.
What does 4.49 bits per parameter actually tell you
The average is a good diagnostic. It can tell you whether the checkpoint's total size is broadly consistent with a quantized representation, and it helps catch missing files or duplicate copies. What it doesn't do is reveal every tensor's dtype or show how the format packs data internally.
For Kimi K3.1 Pro, the breakdown estimates three major contributors:
| Component | Approximate size |
|---|---|
| Four-bit weights at half a byte per weight | 1,361.4 GB |
| Tensors retained at 16 or 32 bits | 114.8 GB |
| Block scales, assuming one byte per 32 weights | 85.1 GB |
| Estimated total | 1,561.2 GB |
| Selected checkpoint files | 1,561.3 GB |
The total estimated from the three components is 1,561,217,377,280 bytes. The listed file total is 1,561,293,160,568 bytes. That leaves a difference of about 75.8 million bytes - about 75.8 MB in decimal units, not 138.6 MB. It's still small compared to a ~1.56 TB download, but the rows don't support the larger residual that sometimes gets reported with them.
Treat the breakdown as an explanation, not a full inspection
The four-bit weight estimate comes from the half-byte-per-parameter assumption. The higher-precision tensor estimate comes from published dtype counts. The scale estimate assumes one byte per 32 weights, a common pattern for block quantization.
There's an important limit: the API data used here doesn't expose the scale layout per tensor. So the scale row is an assumption that fits the total closely - not a direct measurement. Change the block size or packing details, and the internal breakdown can shift while keeping the overall file total plausible.
That's the distinction to keep clear: the total size is directly measurable from the selected files. The "why the total is what it is" comes from dtype counts plus a quantization assumption. A close match boosts confidence, but it doesn't magically turn assumptions into a verified tensor-by-tensor audit.
Check the units before planning a disk
These numbers use decimal gigabytes: 1 GB = 1,000,000,000 bytes. Tools and operating systems may show binary units instead.
A 1,561.3 GB download is about 1.42 TiB, where 1 TiB = 1,099,511,627,776 bytes.
That difference can trip you up when you compare a model's published size with the free-space display on a machine. Also leave headroom for incomplete downloads, temporary files, checksums, and any extracted or converted copy you plan to keep. The checkpoint file total is the minimum storage for those files, not a complete "working environment" estimate.
How should you size storage for a model download
Start with what your download process actually retrieves, not the precision label by itself. If you can query repo metadata before downloading, sum the relevant file sizes. If you can't, use the repo's listed file sizes - but double-check whether that includes alternate formats or duplicate directories.
Then decide what "enough storage" means for your task. A one-time download just needs room for the checkpoint. A workflow that converts formats might need space for both the original and converted files at the same time. A deployment machine also needs space for the OS, runtime, caches, logs, and any model-specific working data.
Separate download size from runtime memory
Disk space and inference memory are different budgets. The checkpoint file total tells you how much storage the checkpoint occupies. It doesn't include runtime allocations, computation buffers, or the context-dependent KV cache. Usable memory requirements also depend on implementation, parallelism, and serving setup.
This matters most with very large models and mixture-of-experts models. Sometimes you activate only part of the network for each token, so you can't size GPU memory from active parameters alone. The stored files still contain the weights represented in that checkpoint.
So: use the file sum for disk planning, and treat it as only one input to deployment planning. Don't infer GPU memory, throughput, or performance from average bits per parameter. Those need measurements (or at least calculations) against the environment and configuration you plan to run.
Use a decision checklist
Before you commit to a download or allocate disk space, verify:
- Repository and revision: Are you measuring the exact model version you'll use?
- File selection: Does your sum include only the intended checkpoint format, not duplicate or alternate copies?
- Parameter count: Is the count for the same model represented by those files?
- Units: Are the listed sizes decimal GB or binary GiB/TiB?
- Working space: Will you need the original and converted checkpoints at the same time?
- Runtime budget: Have you estimated memory separately from disk storage?
If the calculated average is roughly twice what you expected, look first for duplicate files or a mismatch between the selected encoding and the parameter count. If it's far lower, check for missing shards. A weird result is a prompt to inspect inputs - not a reason to force your conclusion to match a badge.
Conclusion
A model page's precision badge and a checkpoint's average storage cost can describe different things. In the Kimi example, summing the intended shards and dividing by the published parameter count gives about 4.49 bits per parameter, even though the platform displays an 8-bit badge. The repository's actual files - not the badge - are what determine disk space.
For a dependable estimate, count the files you'll download, exclude duplicate encodings, use a parameter count that matches those files, and keep disk storage separate from runtime memory. Then treat any precision-based breakdown as something to test, not as a substitute for the file total. That simple distinction keeps a label from turning into a bad storage plan.