Definition
Embeddings are lists of numbers (vectors) produced by a machine learning model to represent the meaning of a piece of content, such as a sentence, a passage, an image or a product listing, so that content with similar meaning gets similar vectors. They let software compare meaning with arithmetic: the distance between two embeddings, usually measured by cosine similarity, indicates how related the two pieces of content are. 'How to train a puppy' lands near 'dog obedience tips' and far from 'quarterly earnings guidance'.
Embeddings are the input to vector search and the retrieval layer of most retrieval-augmented generation systems. They are also used for clustering, deduplication, classification and recommendations.
An embedding model is trained so that related texts end up close together and unrelated texts far apart. The output has a fixed number of dimensions, often between a few hundred and a few thousand. Google's Gemini embedding models output 3,072 dimensions by default and are trained with Matryoshka Representation Learning, so the vectors can be truncated to 1,536 or 768 dimensions to save storage with little quality loss (Google AI for Developers). Every document and every query must be embedded with the same model; vectors from different models are not comparable.
The model market in October 2026 includes:
- Google's gemini-embedding-001 for text and gemini-embedding-2, which Google calls the first multimodal embedding model in the Gemini API, mapping text, images, video, audio and documents into one space across more than 100 languages.
- OpenAI's text-embedding-3-small and text-embedding-3-large.
- Voyage AI's voyage-4 family, Cohere's Embed v4 and Jina's embedding models.
- Open-weight models such as Alibaba's Qwen3-Embedding series (0.6B, 4B and 8B parameters, Apache 2.0 licensed), which rank near the top of the public MTEB leaderboards.
Benchmarks guide the choice. MTEB, the Massive Text Embedding Benchmark, averages performance over many tasks; RTEB is a newer retrieval-only leaderboard that keeps part of its test data private to discourage tuning to the test. In Anthropic's 2024 retrieval experiments, Gemini and Voyage embeddings performed best of those it tried.
Embeddings have limits that matter for search. They compress a passage into one point, so long or multi-topic chunks produce blurred vectors that sit near nothing in particular. They are weak at exact identifiers, which is why retrieval systems pair them with BM25 keyword matching in hybrid search. And a chunk embedded without its surrounding context can lose the very facts that make it findable.
For GEO, you cannot see or control the embedding models AI engines use, but you control the text they embed. Passages that name their subject, stay on one topic and state their point in plain language produce embeddings close to the questions they answer. Passages that rely on earlier context, mix several topics or hide meaning behind slogans produce embeddings that match poorly. Because AI systems embed chunks rather than pages, this applies section by section.
Examples of Embeddings
- An online store embeds 200,000 product descriptions with text-embedding-3-small so that a search for 'warm waterproof jacket for hiking' finds insulated rain shells regardless of brand terminology.
- A media company uses gemini-embedding-2 to place photos and captions in one vector space, letting editors search the photo archive with plain-language descriptions.
- A support team embeds help articles at 768 dimensions instead of 3,072 to cut storage costs, and its retrieval tests show almost no drop in accuracy.
- A content team splits a 6,000-word guide into focused sections after noticing that the whole-page embedding matched none of its target questions closely.
