Guanaco-65B

Guanaco is the model family produced by QLoRA (arXiv 2305.14314, May 2023), the efficient fine-tuning method from the University of Washington's NLP group. QLoRA backpropagates gradients through a frozen 4-bit quantized model into low-rank adapters, using innovations like 4-bit NormalFloat, double quantization, and paged optimizers to cut fine-tuning memory for a 65B model from over 780GB to under 48GB.

The largest Guanaco model reached 99.3% of ChatGPT's performance on the Vicuna benchmark after 24 hours of fine-tuning on a single professional GPU, essentially closing the gap on that benchmark at the time. The paper also notes current chatbot benchmarks are not fully trustworthy, with a lemon-picked failure analysis included. License caveat: Guanaco builds on LLaMA weights, inheriting Meta's non-commercial community license.