Large Language Model (LLM)

A large language model (LLM) is a neural network trained on very large amounts of text, and often images, audio and code, to predict the next token in a sequence. Scaled up and then tuned with human and automated feedback, that single objective produces systems that can answer questions, write, summarize, reason through problems and call tools. LLMs are the engines behind ChatGPT, Claude, Gemini, Perplexity, Microsoft Copilot and Google's AI Overviews and AI Mode.

Nearly all current LLMs use the transformer architecture introduced by Google researchers in 2017, whose attention mechanism lets the model weigh every part of the input against every other part. Many frontier models are sparse mixture-of-experts designs, which activate only a fraction of their parameters for each token. Google's model card for Gemini 4 Argon, for example, describes it as a large sparse mixture-of-experts transformer.

Building an LLM happens in stages:

• Pre-training on trillions of tokens of web pages, books, code and licensed data, learning to predict what comes next
• Supervised fine-tuning on curated instruction and answer pairs
• Reinforcement learning, including RLHF and training on tasks with checkable answers such as math and code
• Safety tuning and evaluation before release

Modern models also 'think' before answering. What were once separate reasoning models are now standard: Anthropic's Claude models use adaptive thinking that is always on for its top tiers, OpenAI exposes reasoning effort settings, and Gemini 4 Argon offers low, medium and high thinking levels.

The frontier as of October 2026 looks like this. OpenAI released GPT-6 Astra on September 3, 2026, followed by GPT-6 Sol, GPT-6 Luna and GPT-6.1 Sol. Anthropic's current lineup is Claude Fable 5.1, Opus 5.5, Sonnet 5.5 and Haiku 5.5, all with 1 million token context windows and a June 2026 knowledge cutoff. Google made Gemini 4 Argon generally available on October 13, 2026, with a 1,048,576 token window shared between input and output. Strong open-weight options include Moonshot's Kimi K3, Zhipu's GLM-5 series, Alibaba's Qwen3.8 and DeepSeek's V4 models (see open source LLMs). Names change every few months, so check providers' model pages rather than relying on any list.

LLMs answer from two sources of knowledge. Parametric knowledge is what the model absorbed during training, frozen at its cutoff date. Retrieved knowledge comes from search or documents fetched at answer time through retrieval-augmented generation. Google's guidance on AI features in Search describes exactly this: AI Overviews and AI Mode use RAG on Google's index and query fan-out, where the model issues several related searches to cover a question. ChatGPT, Copilot and Perplexity work in a similar way with their own search stacks. Parametric answers rarely cite sources; retrieved answers usually do.

That split decides how brands appear. If a prompt triggers search, visibility depends on whether your pages are crawlable, rank for the queries the model generates and state the answer clearly enough to be quoted. If it does not, the answer reflects what the web said about you before the model's cutoff, which is why reviews, documentation and third-party coverage keep mattering long after they are published. LLMs can also hallucinate, stating false details with confidence, so brands need to check what models actually say. With ChatGPT alone reporting 1.2 billion weekly users in September 2026, those answers reach a large audience. Our analysis of how ChatGPT searches the web shows the retrieval side in practice.