FlashRank

FlashRank is an open-source Python library by Prithiviraj Damodaran that adds re-ranking to search and retrieval pipelines using tiny ONNX cross-encoder models. The default nano model is about 4MB and runs in milliseconds on CPU, while the small (~34MB) and medium (~110–150MB) options trade speed for ranking precision. It supports both pairwise cross-encoder reranking and LLM-based listwise reranking, and has become a popular drop-in second stage for RAG systems that need better precision than bi-encoder retrieval alone without paying per-call rerank API fees.

The honest picture: tiny cross-encoders cannot match the ranking quality of large LLM rerankers or hosted APIs like Cohere Rerank on hard queries — FlashRank is a speed/cost play, not a quality-max play. Model selection and max_length tuning meaningfully affect results, and the defaults suit English best. But for CPU-only deployments and latency-sensitive pipelines, it is one of the most practical open reranking options.