A legal search system can return a result that is semantically relevant but legally useless.
Consider a lawyer searching a supply agreement for:
Confidential Information
A typical semantic search system might return dozens of paragraphs discussing confidentiality or secrecy. But the most important result may be the definition clause — the clause that determines what all those other references actually mean.
If that definition appears at rank 31, the search technically worked. For the lawyer, it failed.
Legal documents are full of defined terms, party names, statute numbers, exceptions, carve-outs, and near-duplicate clauses. This makes the embedding model one of the most important components of a legal retrieval system.
In this article, we'll look at seven open-weight embedding models that can be used commercially, along with several hosted alternatives. We'll compare them based on context length, retrieval capabilities, multilingual support, model size, and practical deployment considerations.
Note: The benchmark numbers mentioned below come from the respective model cards and are not directly comparable because different models report results on different datasets.
Why embedding models matter in legal search
An embedding model converts text into vectors that represent its meaning.
For a general search application, semantic similarity may be enough. Legal search is more complicated.
For example, these two clauses might be almost identical:
The liability of the Supplier shall not exceed...
and
The liability of the Supplier shall not exceed...
except for losses arising from...
That small negotiated difference can completely change the legal meaning.
A good legal retrieval system therefore needs to handle both:
Semantic similarity — finding conceptually related clauses
Lexical similarity — matching exact terms, names, numbers, and defined expressions
This is why hybrid retrieval, reranking, and clause-level chunking are often just as important as the embedding model itself.
The 7 Open-Weight Models to Consider
The models below all have licenses that permit commercial use.
Several popular models were excluded because they use CC-BY-NC-4.0, including:
jina-colbert-v2
Linq-Embed-Mistral
SFR-Embedding
Reason-ModernColBERT
For a commercial legal-tech product, licensing is a practical constraint, not an afterthought.
- Qwen3-Embedding-8B
License: Apache-2.0
Parameters: ~8B
Context: 32K tokens
Languages: 100+
Deployment: Open weights; self-hosted
Qwen3-Embedding-8B is an LLM-based dense embedding model with a 32K-token context window and support for more than 100 languages.
According to Qwen's model card, it ranked #1 on the MTEB multilingual leaderboard as of June 5, 2025, with a score of 70.58. It reports 69.44 on MTEB English v2 retrieval.
These are the highest MTEB figures among the models covered here, although they shouldn't be treated as a direct apples-to-apples comparison because the other models report results on different benchmarks.
Why it is interesting for legal documents
The 32K context window gives the model enough room to process long clauses containing:
Sub-clauses
Exceptions
Carve-outs
Qualifications
Related definitions
However, bigger context isn't automatically better.
Embedding an entire multi-page schedule into one vector can dilute the importance of the operative sentence. For legal retrieval, clause-level chunks will often work better than embedding entire schedules.
Qwen3-Embedding-8B is also instruction-aware. You can tell the model what type of search you're performing, such as looking specifically for a limitation-of-liability clause.
The model card notes that omitting the instruction can reduce retrieval performance by 1–5%.
The trade-off
The model's biggest downside is its size.
At around 8 billion parameters, it requires a capable GPU and can increase indexing latency. This matters when re-embedding a large contract archive.
Best fit: Teams prioritizing retrieval quality and long context and that have enough GPU capacity to run a larger model.
- BAAI bge-m3
License: MIT
Context: 8,192 tokens
Languages: 100+
Deployment: Open weights; Cloud and self-hosted
bge-m3 is particularly attractive for legal search because it produces three types of representations:
Dense vectors
Sparse lexical weights
ColBERT-style multi-vectors
This is useful because legal search frequently depends on exact matching.
Consider:
"Confidential Information"
"Section 12.4"
"Party A"
"Article 7(b)"
These aren't just concepts. They're precise references.
A purely dense embedding can sometimes blur these terms into broader semantic topics. Sparse retrieval gives the system another signal.
Why bge-m3 is a strong baseline
bge-m3 supports 8,192 tokens and more than 100 languages.
BAAI reports results on MIRACL and MLDR and recommends combining retrieval with reranking.
A practical architecture looks like:
Query
│
├── Dense retrieval
│
└── Sparse retrieval
│
▼
Candidate set
│
▼
Reranker
│
▼
Final results
The advantage is that one model can provide both dense and sparse representations.
The trade-off
bge-m3 dates from 2024, and newer models report stronger results on some general benchmarks.
Still, it's an excellent model to use as a baseline.
If another model can't clearly beat bge-m3 on your own legal gold set, there's little reason to switch simply because a newer model exists.
Best fit: Legal search where exact terms, defined terms, multilingual documents, and hybrid retrieval matter.
- Snowflake arctic-embed-l-v2.0
License: Apache-2.0
Parameters: 568M
Context: 8,192 tokens
Vector size: 1,024 dimensions
Deployment: Open weights; Cloud and self-hosted
If Qwen3-Embedding-8B is the heavyweight option, arctic-embed-l-v2.0 is the efficiency-focused alternative.
Snowflake reports:
BEIR: 55.6
MIRACL: 55.8
CLEF focused: 52.9
CLEF full: 54.3
Again, these numbers shouldn't be directly compared with benchmarks from other model cards.
Why efficiency matters
Embedding models aren't only used during search.
They're also used whenever documents are indexed or re-indexed.
Imagine a legal platform with hundreds of thousands of contracts.
If you change your chunking strategy, you may need to re-embed the entire archive.
A smaller model can make that process faster and cheaper.
Its multilingual capabilities also make it useful for agreements written in multiple languages.
The trade-off
The primary reason to choose arctic-embed-l-v2.0 is efficiency rather than simply chasing the highest benchmark score.
It makes sense for teams that:
Re-index frequently
Need lower latency
Have large document archives
Plan to use a reranker for additional precision
Best fit: Large legal archives where indexing speed and efficiency matter.
- intfloat multilingual-e5-large-instruct
License: MIT
Parameters: 560M
Context: 512 tokens
Languages: ~100
Deployment: Open weights; self-hosted
multilingual-e5-large-instruct is the established baseline in this comparison.
It has:
560M parameters
24 layers
1,024 dimensions
Around 100 languages
Instruction prefixes for queries
Its long public evaluation history makes it useful as a control when testing newer embedding models.
The problem: 512 tokens
The biggest limitation is its 512-token context window.
That's short for legal documents.
A limitation-of-liability clause can easily exceed 512 tokens once exceptions and carve-outs are included.
You therefore need more aggressive chunking.
For example:
Long clause
↓
Chunk 1
Chunk 2
Chunk 3
↓
Embed separately
That can work, but splitting a clause also means potentially losing relationships between sentences.
The model card also notes that performance degrades for low-resource languages.
Best fit: Teams looking for a proven baseline with relatively short clause-level chunks.
- nomic-embed-text-v2-moe
License: Apache-2.0
Parameters: 475M total / 305M active per token
Context: 512 tokens
Languages: ~100
Deployment: Open weights; self-hosted
nomic-embed-text-v2-moe uses a mixture-of-experts architecture.
Nomic reports:
BEIR: 52.86
MIRACL: 65.80
One of its most interesting features is Matryoshka embeddings.
The vector dimensions can be reduced from 768 down to 256, allowing teams to trade some representation capacity for a smaller index.
That can become valuable when the document archive grows.
Why transparency matters
Nomic also publishes its training data and code.
For legal-tech vendors, this can be useful when enterprise customers ask questions about the model:
What was the model trained on?
How transparent is the training data?
Can we inspect the implementation?
Can we control the model ourselves?
Those questions can be especially important when selling into regulated or security-conscious organizations.
The trade-off
Like multilingual-e5-large-instruct, this model has a 512-token maximum sequence length.
Long contractual provisions therefore require aggressive chunking.
Best fit: Multilingual collections with shorter chunks and teams that value transparency and smaller indexes.
- Alibaba-NLP gte-modernbert-base
License: Apache-2.0
Parameters: 149M
Context: 8,192 tokens
Language: English
Deployment: Open weights; self-hosted
If speed is your priority, gte-modernbert-base deserves attention.
At only 149M parameters, it is one of the smallest models in this comparison.
Yet it supports an 8,192-token context window.
Alibaba reports:
MTEB English: 64.38
BEIR: 55.33
LoCo: 87.57
CoIR: 79.31
Why this combination is useful
Legal search is often interactive.
A lawyer enters a query, checks the results, changes the wording, and searches again.
That makes latency important.
A small model that can still process long English clauses can provide a useful balance between:
Retrieval speed
Indexing speed
GPU requirements
Context length
The limitation
The model is English-only.
So it isn't suitable as the sole embedding model for multilingual contract collections.
Best fit: Fast English legal search where GPU resources are limited.
- lightonai GTE-ModernColBERT-v1
License: Apache-2.0
Context: Up to 32K with configuration
Deployment: Open weights; Cloud and self-hosted
GTE-ModernColBERT-v1 uses a late-interaction approach.
Instead of representing an entire document chunk with one vector, it maintains token-level vectors and matches query tokens against document tokens.
This can be particularly useful for legal documents containing near-duplicate clauses.
Consider two indemnity clauses that are identical except for one negotiated phrase.
A single-vector embedding might consider both almost identical.
Token-level matching gives the system another way to identify the difference.
Long-document considerations
LightOn reports:
BEIR average: 54.67
LongEmbed average: 88.39 at 32K document length
However, there's an important caveat.
The model was trained using a 300-token document length by default. Longer-document evaluation requires changing the setting, and the model card notes that lengths beyond 8K aren't guaranteed.
So if long-document retrieval is important, benchmark this model against your own contracts.
The trade-off
Late interaction comes with a larger multi-vector index.
The model is also English-only.
Best fit: Systems where the main retrieval problem is distinguishing near-duplicate contractual language.
Hosted Alternatives
Open-weight models give you control, but hosted APIs can significantly simplify infrastructure.
Here are several alternatives worth considering.
Model
Provider
Listed price
Key consideration
voyage-law-2
Voyage AI
$0.12 / 1M tokens
Legal-focused embedding
Kanon 2 Embedder
Isaacus
$0.35 / 1M tokens
Strong MLEB result
text-embedding-3-large
OpenAI
$0.13 / 1M tokens
Simple hosted API
Embed 5
Cohere
Varies
128K context + private deployment
Voyage AI
voyage-law-2 is a hosted model trained on legal text and costs $0.12 per 1M tokens according to the listed pricing.
Voyage models can also be deployed within an AWS VPC through AWS Marketplace. If this matters for your architecture, verify that the specific legal model is available through that deployment route.
Isaacus
Kanon 2 Embedder is listed at $0.35 per 1M tokens.
Isaacus reports that it ranks first on its MLEB legal benchmark at 86% NDCG@10.
The MLEB datasets and evaluation code are public, making the benchmark reproducible.
Isaacus also offers private air-gapped deployment on its advanced plan.
OpenAI
text-embedding-3-large costs $0.13 per 1M tokens.
For teams that don't want to manage GPUs, the simplicity is the main attraction.
Cohere
Cohere's Embed 5 models support a 128K-token context window.
Cohere also provides private VPC and on-premises deployment options.
Open-Weight vs. Hosted Embeddings
The choice generally comes down to control versus convenience.
Hosted API
You get:
No GPU management
Simple API integration
Faster initial deployment
Less infrastructure maintenance
Open weights
You get:
Control over the model
Control over where it runs
Ability to fine-tune
More deployment flexibility
Less dependence on a vendor's model lifecycle
For legal technology, the deployment question can be especially important.
Some customers may require documents to remain within a specific cloud environment. Others may prohibit third-party processing entirely.
Running an open-weight model in your own infrastructure gives you considerably more control over those requirements.
What Happens When You Move Away From OpenAI Embeddings?
Switching embedding models isn't simply a configuration change.
If your existing application uses text-embedding-3-large, moving to another embedding model generally requires rebuilding the vector index.
For example:
Model
Vector dimensions
OpenAI text-embedding-3-large
3,072
bge-m3
1,024
arctic-embed-l-v2.0
1,024
Because the dimensions can change, your vector database will often require a new collection rather than an in-place update.
A typical migration looks like:
Existing documents
│
├── Old embedding model → Existing index
│
└── New embedding model → New index
│
▼
Compare results
│
▼
Gold-set testing
│
▼
Cut over
For a large legal archive, re-embedding can be a significant operation.
The safest approach is to run both indexes side by side and compare them against a representative evaluation set before switching production traffic.
How to Test These Models
Benchmark scores are useful, but your own legal documents should make the final decision.
A practical evaluation workflow is:
- Build a representative document set
Include different types of documents:
NDAs
Supply agreements
Employment agreements
MSAs
DPAs
Vendor contracts
Regulatory documents
- Create real search queries
Use the types of searches your users actually perform.
For example:
"Find the limitation of liability clause"
"Who can terminate the agreement?"
"What are the confidentiality obligations?"
"Find the governing law provision"
"Which party owns the IP?"
- Define the expected results
Have lawyers or domain experts identify the relevant clauses.
- Test multiple models
Run the same queries through:
bge-m3
Qwen3-Embedding-8B
arctic-embed-l-v2.0
GTE-ModernColBERT-v1
and any hosted model you're considering.
- Measure more than relevance
Track:
Recall
Precision
NDCG
Top-k retrieval accuracy
Query latency
Indexing speed
Index size
GPU requirements
Cost per document/query
A model that wins a general benchmark may not win on your contracts.
Why Run These Models on Superlinked Inference Engine?
An inference model doesn't exist in isolation.
A production legal search pipeline may look like:
Documents
│
▼
OCR
│
▼
Chunking
│
▼
Embedding
│
▼
Vector Database
│
▼
Candidate Retrieval
│
▼
Reranking
│
▼
Final Results
Superlinked Inference Engine provides an inference layer for several of these model workloads, including embeddings, reranking, OCR, and extraction.
The models discussed above can be tested through the SIE model catalog, with SIE available as:
Managed SIE Cloud
Self-hosted SIE on Kubernetes
Local development
The self-hosted deployment is particularly relevant when contracts must remain inside your own environment.
So, Which Embedding Model Should You Choose?
There isn't one universal winner.
A practical shortlist would be:
🥇 bge-m3
Start here if your legal search needs dense + sparse retrieval, exact-term matching, and multilingual support.
🥈 Qwen3-Embedding-8B
Test this if retrieval quality and long context are your priorities and you have enough GPU capacity.
🥉 arctic-embed-l-v2.0
Choose this when efficiency, indexing speed, and lower latency are important.
Also test GTE-ModernColBERT-v1
Add this if your biggest problem is near-duplicate clauses and subtle negotiated differences.
Final Takeaway
Choosing an embedding model for legal search isn't about picking the model with the highest leaderboard score.
Legal retrieval has a different standard.
The system needs to find the specific clause that matters, even when that clause contains a defined term, a negotiated exception, or only a few words that distinguish it from dozens of similar clauses.
That's why a strong legal retrieval stack usually combines:
Good chunking
+
Dense retrieval
+
Sparse / lexical retrieval
+
Reranking
+
A representative legal evaluation set
For most teams, I'd start by testing bge-m3, Qwen3-Embedding-8B, and arctic-embed-l-v2.0 on real contract data.
If near-duplicate clauses are a major source of retrieval failures, add GTE-ModernColBERT-v1.
And before making the final decision, benchmark everything against your own gold set.
Because in legal search, the best embedding model isn't necessarily the one with the best benchmark score.
It's the one that consistently puts the clause your lawyer is looking for at the top of the results.
