AutoAWQ
4-bit AWQ quantization for LLMs — deprecated upstream, but still widely integrated.
Retired
AutoAWQ appears to be discontinued or shut down — standalone project deprecated; moved to vllm-project/llm-compressor This page is kept as an archive record and is excluded from search engines.
About
AutoAWQ implements Activation-aware Weight Quantization (AWQ), a 4-bit quantization method from MIT's Han Lab that preserves salient weights for better accuracy at low bit-widths, claiming 2–3x inference speedups across around 35 model architectures. It became one of the most widely used LLM quantization paths, with native integration in Hugging Face Transformers (via optimum) and broad community tooling support.
The important context: the standalone AutoAWQ project is officially deprecated — AWQ functionality has moved to the vllm-project/llm-compressor repository. New projects should quantize through llm-compressor or Transformers' native AWQ support rather than the old autoawq package. It is included here as a catalog entry because the format remains pervasive in model hubs; treat it as a historical reference, not a fresh dependency.
What it can do
- 4-bit activation-aware weight quantization
- 2-3x inference speedup claims
- 35 supported model architectures
- Transformers integration
- Calibration-based quantization
Setup
Deprecated: use llm-compressor or Transformers' built-in AWQ quantization instead of pip install autoawq.
Prerequisites
A supported LLM checkpoint; GPU for quantized inference.
Use cases
Works with
Trust score — 7/10
Scored 7/10 (metrics tier, metrics-v1). Evidence: Verified via vendor site/docs/GitHub and reputable coverage, wave 85; Trending score 72/100 from researcher assessment.
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.