The problem with most AI glossaries is that they're written for people who already understand the concepts. This guide is written for technical leaders who need working definitions — precise enough to use in architecture discussions and product decisions, but not padded with academic qualifications. We've focused on the vocabulary you need to evaluate vendors, direct engineering work, and have informed conversations about AI strategy. Each definition includes the practical implication that matters most for building products.
LLM (Large Language Model)
A large language model is a neural network trained on massive text datasets to predict the next word in a sequence. GPT-4, Claude 3.5, Gemini, and Llama 3 are all LLMs. Practical implication: LLMs are not databases — they generate plausible text based on training, which is why they can write and reason but also make confident errors. Understanding this distinction shapes every architecture decision you make around AI.
Context Window
The context window is the maximum amount of text an LLM can process in a single API call — both input and output combined. Claude 3.5 Sonnet: 200K tokens. GPT-4o: 128K tokens. A token is roughly 0.75 words in English. Practical implication: Context window size determines what you can fit in a single prompt. If your use case requires reasoning across large documents or codebases, context window size is a primary model selection criterion. Performance degrades at the far end of large contexts.
Hallucination
Hallucination describes when an LLM generates factually incorrect text stated with confidence. The model doesn't "know" it's wrong — it's producing a statistically plausible output that happens to be false. Practical implication: Any AI output acted on without human review needs a verification layer. Measure hallucination rates on your specific use case — don't rely on published benchmarks.
Embeddings
Embeddings are numerical (vector) representations of text where semantically similar content produces mathematically close vectors. You can find documents similar in meaning to a query even without shared keywords. Practical implication: Embeddings power semantic search and are the foundation of RAG. Embedding models are separate from generation models and substantially cheaper to call.
RAG (Retrieval-Augmented Generation)
RAG is an architecture pattern where you retrieve relevant information from your own data and include it in the prompt before asking the LLM to respond. Instead of relying on training knowledge, you give the model the specific context it needs. Practical implication: RAG is the practical solution to hallucination for domain-specific questions. It's simpler than fine-tuning, can be updated in real time, and is the right starting point for most enterprise knowledge-base applications.
Fine-Tuning
Fine-tuning continues training a pre-trained model on your specific data, adjusting its weights to improve performance on a specific task. It changes the model's parameters, not just the instructions it receives. Practical implication: Fine-tuning is worth considering for consistent output format or style that prompting doesn't reliably achieve. For most enterprise use cases, RAG + good prompt engineering is simpler, cheaper, and more maintainable.
AI Agents
An AI agent is a system where an LLM is given tools (web search, code execution, API calls) and autonomously decides which tools to use and in what sequence to accomplish a goal. The LLM acts as a reasoning engine that plans and executes multi-step tasks. Practical implication: Agents are significantly more powerful for complex tasks, but significantly harder to test and debug. Start simple — single agent with a small, well-defined tool set — and only add complexity when simpler designs fail.
Inference
Inference is running a trained model to generate an output, as opposed to training, which teaches the model from data. When you call an AI API, you're running inference. Practical implication: Inference cost and latency are the engineering variables you optimise in production. Model routing (using smaller models for tasks that don't need frontier capability) is the primary cost management lever.
What This Means for Engineering Teams
Working vocabulary matters in AI development because terminology maps to real architectural decisions. When a product manager asks for "AI that uses our docs," that's a RAG question. When a VP asks why the model "made something up," that's a hallucination and verification question. Shared precise vocabulary reduces translation work in every technical discussion.
If your engineering team is building with AI and needs developers who apply these concepts to real product problems, our AI automation team can accelerate your implementation. You can also hire AI engineers who bring this technical vocabulary and the judgment to apply it correctly.
Frequently Asked Questions
What is the difference between RAG and fine-tuning?
RAG retrieves relevant information at query time and includes it in the prompt — knowledge can be updated in real time, and the model's weights don't change. Fine-tuning modifies the model's weights by training on your data — knowledge becomes baked in and static until re-training. RAG is simpler, cheaper, and more maintainable for most enterprise use cases.
Can you eliminate hallucinations with better prompting?
You can reduce hallucination rates significantly with better prompting (asking the model to acknowledge uncertainty, providing source material via RAG, chain-of-thought prompting) but you cannot eliminate them entirely. For any use case where factual accuracy is critical, you need an explicit verification layer — human review, automated fact-checking, or constrained output formats.
What is the difference between an AI model and an AI agent?
A model generates a response to a single prompt. An agent uses a model as a reasoning engine to autonomously plan and execute multi-step tasks using tools. Agents are more powerful but harder to control and debug. Start with the simplest architecture that solves the problem.
What does "tokens" mean and why does it matter for pricing?
Tokens are the basic units LLMs process — roughly 0.75 words in English. AI API pricing is based on input tokens plus output tokens, with output tokens typically 3–5x more expensive. Monitoring your average tokens per API call is the starting point for AI cost optimisation.
What is the difference between a small language model and a large language model?
The distinction is primarily parameter count. SLMs (up to ~13B parameters) are faster, cheaper, and can run on consumer hardware. LLMs perform better on complex reasoning but cost more. For many common enterprise tasks in 2026, SLMs are now good enough — and significantly cheaper per call.