cross-encoder/ms-marco-MiniLM-L6-v2 · Hugging Face

Overview

The cross-encoder/ms-marco-MiniLM-L6-v2 model is a cross-encoder designed for text ranking, specifically fine-tuned on the MS MARCO passage ranking dataset. It is based on the MiniLM architecture with 6 layers, offering a balance between efficiency and accuracy. The model is available on Hugging Face under the identifier cross-encoder/ms-marco-MiniLM-L6-v2 and has accumulated over 79.9 million downloads, indicating its widespread adoption in the NLP community. The model card is hosted at Hugging Face.

Capabilities

This model excels at re-ranking passages given a query. It takes a query and a passage as input and outputs a relevance score. Key capabilities include:

• Text Ranking: Assigns a score (logit) indicating the relevance of a passage to a query. In the example from the model card, scores like 8.6 and -4.3 are produced, where higher positive values indicate higher relevance.
• Cross-Encoder Architecture: Unlike bi-encoders, cross-encoders jointly encode the query and passage, leading to higher accuracy at the cost of slower inference. This makes it suitable for second-stage re-ranking where a smaller set of candidates is scored.
• Multi-Framework Support: Available in PyTorch, JAX, ONNX, SafeTensors, and OpenVINO formats, enabling deployment across various environments. The model card lists these formats under the model tags.
• Integration with Sentence-Transformers: Can be used with the sentence-transformers library for easy integration into existing pipelines. The CrossEncoder class provides a simple API for scoring pairs.
• Integration with Transformers: Can be loaded directly using AutoModelForSequenceClassification from the transformers library, allowing custom inference pipelines.

Use cases

• Search Engine Re-Ranking: As a second-stage re-ranker to improve the relevance of initial search results from a bi-encoder or BM25. The model card explicitly mentions usage with ElasticSearch for retrieval and re-ranking.
• Question Answering: Ranking candidate passages for extractive QA systems. The example in the model card uses a query about Berlin's population.
• Information Retrieval: Enhancing retrieval pipelines in domains like legal, medical, or enterprise search. The model is trained on MS MARCO, which is a general-domain passage ranking dataset.
• Research: Benchmarking cross-encoder performance on ranking tasks. The model is a popular baseline for re-ranking experiments.

License & deployment

The model is released under the Apache-2.0 license, as indicated on the Hugging Face model page. This allows for commercial and non-commercial use with attribution. Deployment options include:

• Hugging Face Transformers: Directly load the model using AutoModelForSequenceClassification and AutoTokenizer. Example code is provided in the model card.
• Sentence-Transformers: Use the CrossEncoder class for easy inference. This is the recommended approach for most users.
• ONNX Runtime: Optimize for production with ONNX format. The model is available in ONNX format on Hugging Face.
• OpenVINO: Deploy on Intel hardware for accelerated inference. The model card includes OpenVINO as a supported format.
• Inference Providers: The model can be deployed via Hugging Face Inference Endpoints or other cloud services.

Hardware considerations: The model is relatively small (6 layers) and can run on CPU, but GPU acceleration is recommended for production workloads. The exact memory footprint is not specified in the source, but typical MiniLM-L6 models require around 200-300 MB of RAM.

Alternatives

• ms-marco-MiniLM-L12-v2: A larger 12-layer version for higher accuracy, also available on Hugging Face.
• ms-marco-TinyBERT-L2-v2: A smaller, faster model for resource-constrained environments.
• Cohere rerank: Proprietary API-based re-ranking model.
• ColBERT: Late interaction model that balances efficiency and accuracy, often used as an alternative to cross-encoders.
• Other cross-encoders: Models like cross-encoder/stsb-roberta-large for semantic textual similarity, or domain-specific variants.

FAQ

Q: What is the difference between a cross-encoder and a bi-encoder? A: A cross-encoder processes the query and passage together as a single input, allowing full interaction between tokens, which leads to higher accuracy but slower inference. A bi-encoder encodes query and passage separately into vectors and uses cosine similarity, which is faster but less accurate. Cross-encoders are typically used for re-ranking a small set of candidates.

Q: What is the output range of the scores? A: The model outputs raw logits (not normalized between 0 and 1). In the example, scores like 8.6 and -4.3 are produced. For ranking, the relative order matters more than the absolute values. To obtain probabilities, one can apply a sigmoid function.

Q: Can this model be used for zero-shot classification? A: While primarily designed for ranking, it can be adapted for classification by treating class labels as passages. However, it is not optimized for that task.

Q: What is the training data? A: The model was fine-tuned on the MS MARCO Passage Ranking dataset, which consists of over 8.8 million passages and 1 million queries from Bing search logs.

Q: Is the model available in other languages? A: The model is trained on English data only. For multilingual ranking, consider models like cross-encoder/mmarco-mMiniLMv2-L6-H384.

Q: How to deploy with ONNX? A: The model is available in ONNX format on Hugging Face. You can load it using onnxruntime and run inference. The model card does not provide specific ONNX deployment code, but the ONNX file can be downloaded from the model repository.

Q: What are the safety considerations? A: The model may reflect biases present in the training data (MS MARCO). Users should evaluate for fairness and potential harmful outputs in their specific use case. No explicit safety evaluation is provided in the source.