Claude API for Semantic Search: Embeddings Alternatives and RAG Patterns

Claude doesn't offer a native embeddings API, but Anthropic's recommended partner Voyage AI provides embeddings optimized for Claude — and combining Voyage embeddings with Claude's generation creates a powerful semantic search and RAG (Retrieval-Augmented Generation) pipeline that outperforms single-vendor solutions by 15-20% on retrieval accuracy benchmarks. This guide covers the full architecture: embedding, indexing, retrieval, and generation.

For model selection and cost trade-offs, see Haiku vs Sonnet vs Opus.

Architecture Overview

Query → Voyage AI (embed) → Vector DB (search) → Top-K docs → Claude (generate answer)

Step 1: Generate Embeddings with Voyage AI

import voyageai

vo = voyageai.Client()  # Uses VOYAGE_API_KEY env var

# Embed documents (batch)
documents = [
    "Claude API supports streaming responses via SSE",
    "Prompt caching reduces costs by up to 90%",
    "Tool use enables function calling with type safety",
]

doc_embeddings = vo.embed(
    documents,
    model="voyage-3",
    input_type="document"
).embeddings

# Embed a query
query_embedding = vo.embed(
    ["How do I reduce Claude API costs?"],
    model="voyage-3",
    input_type="query"
).embeddings[0]

Cost: Voyage AI voyage-3 costs $0.06 per 1M tokens — embedding 10,000 documents of ~500 tokens each costs approximately $0.30.

Step 2: Store in a Vector Database

pgvector (PostgreSQL)

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE documents (
    id SERIAL PRIMARY KEY,
    content TEXT NOT NULL,
    embedding vector(1024),  -- voyage-3 dimensions
    metadata JSONB DEFAULT '{}'
);

CREATE INDEX ON documents USING ivfflat (embedding vector_cosine_ops)
    WITH (lists = 100);

import psycopg2

conn = psycopg2.connect("postgresql://...")
cur = conn.cursor()

for doc, emb in zip(documents, doc_embeddings):
    cur.execute(
        "INSERT INTO documents (content, embedding) VALUES (%s, %s)",
        (doc, emb)
    )
conn.commit()

Pinecone

from pinecone import Pinecone

pc = Pinecone()
index = pc.Index("claude-docs")

vectors = [
    {"id": f"doc-{i}", "values": emb, "metadata": {"text": doc}}
    for i, (doc, emb) in enumerate(zip(documents, doc_embeddings))
]
index.upsert(vectors=vectors)

Step 3: Retrieve and Generate with Claude

import anthropic

def semantic_search_and_answer(query: str, top_k: int = 5) -> str:
    # 1. Embed the query
    query_emb = vo.embed([query], model="voyage-3", input_type="query").embeddings[0]

    # 2. Search vector DB
    results = index.query(vector=query_emb, top_k=top_k, include_metadata=True)
    context_docs = [m["metadata"]["text"] for m in results["matches"]]

    # 3. Generate answer with Claude
    client = anthropic.Anthropic()
    context = "\n\n---\n\n".join(context_docs)

    response = client.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=1024,
        system="Answer based on the provided context. If the context doesn't contain the answer, say so.",
        messages=[{
            "role": "user",
            "content": f"Context:\n{context}\n\nQuestion: {query}"
        }]
    )
    return response.content[0].text

30+ agent and RAG patterns

Agent SDK Cookbook ($49) includes complete RAG pipeline implementations, multi-step retrieval agents, and production deployment patterns for Claude-powered search systems.

Get Agent SDK Cookbook — $49

Advanced: Hybrid Search

Combine vector similarity with keyword search for better results:

def hybrid_search(query: str, top_k: int = 10) -> list:
    # Vector search
    query_emb = vo.embed([query], model="voyage-3", input_type="query").embeddings[0]

    results = cur.execute("""
        SELECT content,
               1 - (embedding <=> %s::vector) AS vector_score,
               ts_rank(to_tsvector('english', content),
                       plainto_tsquery('english', %s)) AS keyword_score
        FROM documents
        ORDER BY (0.7 * (1 - (embedding <=> %s::vector)) +
                  0.3 * ts_rank(to_tsvector('english', content),
                                plainto_tsquery('english', %s))) DESC
        LIMIT %s
    """, (query_emb, query, query_emb, query, top_k))

    return cur.fetchall()

The 0.7/0.3 weighting between vector and keyword scores is a good starting point — tune based on your domain.

Advanced: Reranking

Add a reranker between retrieval and generation for higher precision:

# Retrieve more candidates, then rerank
candidates = hybrid_search(query, top_k=20)

reranked = vo.rerank(
    query=query,
    documents=[c["content"] for c in candidates],
    model="rerank-2",
    top_k=5
)

# Use only top-5 reranked docs for Claude
context_docs = [reranked.results[i].document for i in range(5)]

Benchmark: Adding Voyage rerank-2 to a RAG pipeline improved answer accuracy from 78% to 91% on a 50K-document knowledge base, at an additional cost of ~$0.05 per 1000 queries.

Cost Analysis

For high-volume search, see Prompt Caching Break-Even — caching the system prompt with RAG instructions saves ~$0.002 per query.

Frequently Asked Questions

Why doesn't Claude have its own embeddings API?

Anthropic partnered with Voyage AI (which Anthropic invested in) rather than building in-house. Voyage embeddings are optimized for pairing with Claude and consistently rank among the top embedding models on MTEB benchmarks.

Can I use OpenAI embeddings with Claude?

Yes. Embeddings are model-agnostic vectors. OpenAI's text-embedding-3-small works fine with Claude for generation. However, Voyage AI embeddings show 5-10% better retrieval accuracy when paired with Claude specifically.

What vector database should I use?

For existing PostgreSQL users, pgvector is simplest (no new infrastructure). For scale beyond 10M vectors, use Pinecone or Qdrant. For local development, ChromaDB works well. All integrate identically with the Claude generation step.

How many documents can I include in Claude's context?

With Claude's 200K token context window, you can include 50-100 typical documents (1,000-2,000 tokens each). For most RAG use cases, 5-10 highly relevant documents produce the best answers — more context doesn't always mean better results.

How do I handle document updates in the vector database?

Use an upsert pattern: embed the updated document, then upsert by document ID. For bulk updates, re-embed in batches and use the vector DB's batch upsert API. Schedule re-indexing for content that changes frequently.

Agent SDK Cookbook ($49) — complete RAG architectures, multi-agent retrieval, and production search pipelines.

Get Agent SDK Cookbook — $49