A recent paper reported by arXiv cs.CL outlines a method for turning decoder-only large language models into dense retrievers without requiring standard retriever training. Instead of adapting model weights through gradient descent, the approach relies on prompting the model with a few in-context examples. The technique, called RICE, conditions large language models on examples that establish a shared context for both query and document encoding.
Understanding Training-Free Retrieval
In standard information retrieval pipelines, embedding models must be explicitly trained or fine-tuned to map queries and documents into a shared vector space where semantic similarity can be computed. This usually requires significant computational resources and labeled training datasets. By attempting to elicit these dense representations purely through prompt engineering and in-context learning, methods like RICE aim to bypass the training phase entirely, leveraging the existing zero-shot or few-shot capabilities inherent in modern decoder-only architectures.
Unknowns and Implementation Details
Despite the introduction of the RICE approach, the announcement did not specify the exact computational overhead required when processing long contexts for query and document encoding. Furthermore, the available information does not detail how the technique scales across different model sizes, nor does it provide concrete performance benchmarks compared against state-of-the-art fully trained dense retrievers. Future updates from the researchers may clarify these missing metrics.
The development highlights ongoing academic exploration into extracting specialized capabilities from large language models using only prompt-based methods.
Lead image: Christina Morillo / Pexels
