Vector Embeddings — Cheat Sheet

Tags: embeddings, vector-search, nlp, cosine-similarity, pgvector, rag

Level: intermediate

TL;DR: Vector embeddings convert unstructured data into dense semantic numerical vectors, enabling fast approximate nearest neighbor similarity searches via distance metrics like cosine similarity.

Key concepts

Dense numerical representation of data points where geometric proximity reflects semantic similarity.

Converts words, sentences, or images into fixed-size floating-point arrays (e.g., 384, 768, 1536 dimensions).

Distance metrics include Cosine Similarity, Dot Product, and Euclidean (L2) distance.

Normalized vectors make Dot Product and Cosine Similarity mathematically identical.

High dimensionality captures fine-grained nuances but increases memory overhead and compute latency.

Vector databases index embeddings using algorithms like HNSW or IVF for Approximate Nearest Neighbor (ANN) search.

Used in Retrieval-Augmented Generation (RAG) to fetch relevant context chunks before prompting LLMs.

Asymmetric search optimizes embeddings when queries are short and documents are long paragraphs.

Typical questions and ready answers

Q: How do you choose between Cosine Similarity and Euclidean Distance for text embeddings?R: Cosine similarity evaluates the angle between vectors regardless of text length, making it ideal for variable-length documents. Euclidean distance is sensitive to magnitude unless the vectors are strictly unit-normalized.

Q: What is the difference between exact KNN search and ANN search in production systems?R: Exact KNN computes distances against every single vector, which guarantees perfect accuracy but scales poorly (O(N)O(N)O(N)). ANN algorithms like HNSW approximate neighbors in sub-linear time, sacrificing marginal recall for massive latency gains.

Q: What causes semantic drift or poor retrieval accuracy in vector search?R: Common causes are mismatched chunk boundaries breaking context, domain shifts between training corpora and query terms, and comparing unnormalized vectors using dot product.

Pitfalls and common mistakes

Mixing embeddings generated by different model versions or completely different embedding architectures.

Storing un-normalized embeddings while querying with inner product search.

Chunking text too aggressively, which strips essential semantic context from isolated chunks.

Over-relying on vector search alone instead of hybrid search (combining dense vectors with BM25 keyword search).

Forgetting to index high-cardinality metadata alongside vector columns for filtered ANN queries.

Snippets and commandsSQL

```-- pgvector setup and index creation in PostgreSQL

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE document_embeddings (

id SERIAL PRIMARY KEY,

content TEXT NOT NULL,

embedding vector(1536)

);

-- HNSW index for cosine distance

CREATE INDEX ON document_embeddings

USING hnsw (embedding vector_cosine_ops)

WITH (m = 16, ef_construction = 64);

-- Query nearest 5 documents by cosine distance (<=>)

SELECT id, content, 1 - (embedding <=> '[0.012, -0.045, ...]'::vector) AS similarity

FROM document_embeddings

ORDER BY embedding <=> '[0.012, -0.045, ...]'::vector

LIMIT 5;

// Cosine similarity implementation in TypeScript

function cosineSimilarity(a: number[], b: number[]): number {

let dotProduct = 0;

let normA = 0;

let normB = 0;

for (let i = 0; i < a.length; i++) {

dotProduct += a[i] * b[i];

normA += a[i] * a[i];

normB += b[i] * b[i];

}

if (normA === 0 || normB === 0) return 0;

return dotProduct / (Math.sqrt(normA) * Math.sqrt(normB));

}

```

To review

[ ] HNSW graph parameters: tuning M and efConstruction tradeoffs between build time and recall.

[ ] Quantization techniques (Scalar Quantization, Product Quantization) for large-scale memory reduction.

[x] Cosine similarity vs dot product on normalized vectors.

[x] Hybrid search combining dense embeddings with sparse BM25 lexical ranking.