Find the weights that matter

In a 70-billion parameter model, protecting just 50 weights
during 4-bit quantization recovers 90-99% of lost quality.

See the resultsData-free. Single forward pass.

Weights Protected

99%

Quality Recovery

Forward Pass

Calibration Data

How PeakWeights works

A simple scoring function finds the needles in 70 billion haystacks.

score = |weight| × |max_activation|

Captures worst-case quantization error.

In 70 billion parameters, only ~50 are critical.

They follow a power law distribution.

Keep critical weights in FP16, quantize the rest to 4-bit.

Recover 90-99% of lost quality.

Tested across 5 architectures from the research paper

Model	FP16 PPL	4-bit PPL	+PeakWeights	Recovery
Qwen2.5-7B	10.62	11.51	10.63	99%
Mistral-7B-v0.3	13.44	13.8	13.58	61%
SmolLM2-1.7B	17.92	24.56	17.99	99%
DeepSeek-R1-Distill-Qwen-7B	70.73	73.52	70.91	93%
Phi-3-mini	11.65	12.67	11.68	97%

Perplexity on WikiText-103. Lower is better.

Install and run in under a minute

1. Install

pip install peakweights

2. Find critical weights

peakweights Qwen/Qwen2.5-7B --top_k 50

3. Or use Python

from peakweights import find
critical = find("Qwen/Qwen2.5-7B", k=50)

Pre-computed peak weights for the most popular open-source LLMs in 2026

DeepSeek

Qwen3

Alibaba

Llama 4 Maverick

“PeakWeights: Data-Free Discovery of Critical LLM Parameters”

Thiyagarajan M (Kalmantic Labs) & Vamshi Ambati (CMU, IIIT Hyderabad)