Every year, “AI” goes from being a demo you chat with to infrastructure you build on. To use it well, a handful of terms decide everything you pay and everything you can do — and tokens and API keys explained clearly are the foundation. The mental model is simple: you send text to a remote service, get text back, and pay for what flows through. The currency, the middlemen, and the price list are the three things worth understanding before you spend a cent.
Inference
Inference is the industry word for the text-in, text-out process. You send a prompt, the model generates a response, and the provider bills you for the tokens that crossed the wire. That is all inference means. Everything else in this primer is about controlling that process: who hosts it, how you authenticate, how you route it, and what you pay.
Tokens
A token is the unit the model actually reads and writes. Before input reaches the model, your text is chopped into smaller pieces by a tokenizer. Think of it as a chopper: roughly three quarters of an English word become one token, but the exact ratio varies. Common words like “the” are usually one token; a rare word can become several tokens, and code and other languages often tokenize differently. The model never sees full words or characters; it sees a sequence of token IDs.
Tokens matter for three reasons. First, cost: virtually every provider bills per token, for both the text you send in and the text the model returns, and output usually costs more than input. Second, context: a model’s context window is measured in tokens. That window holds the entire conversation, any files you attach, and the system prompt together. The system prompt is the hidden instructions you give the model at the start of a conversation to set its behavior. Long documents eat context fast. Third, performance: throughput is measured in tokens per second, so bigger requests are slower and more expensive in direct proportion.
Providers
A provider is the company or organization that trains a model and runs the servers that serve it. Anthropic produces Claude, OpenAI produces GPT, Google produces Gemini, Meta produces Llama, and there are many more. Providers expose models over a web interface called an API, which stands for Application Programming Interface. That is simply a structured way for one piece of software to talk to another. Providers set the prices, the rate limits, and the terms. A rate limit is the maximum number of requests or tokens you may use in a given period.
There is a second, looser use of the word: a provider can also be any service that actually serves you model output, including resellers and hosting platforms. Open-weight models like Llama and DeepSeek are trained by one group but hosted by dozens of companies. Open-weight means the model’s internal parameters, or weights, are publicly released, so anyone can inspect, modify, or host them. So “which provider?” really asks two questions: who built the model, and who is serving it to you. Usually the answer to both is the same; when it is not, you are buying from a gateway.
API keys
An API key is a credential. It is a long random string that identifies you to the provider’s servers and, more importantly, links your requests to your billing account. You send it in an Authorization header on every request. A header is just a piece of metadata attached to a web request, similar to putting a return address on an envelope. The provider uses the key to decide who you are, how fast you may go, and whose bill to add this call to.
Treat an API key like a password, because it is one: anyone who holds it can spend your money and burn through your rate limits. Keys are scoped to a provider, meaning an Anthropic key works only with Anthropic. Providers offer ways to restrict them, rotate them, and watch for leaks. In modern setups, keys are often managed by a local tool or a backend service rather than pasted into a browser or committed to a repository.
Endpoints
An endpoint is the web address where you send your request. It is the front door of the API. For OpenAI, the endpoint is https://api.openai.com/v1/chat/completions; for Anthropic it is https://api.anthropic.com/v1/messages; for OpenRouter it is https://openrouter.ai/api/v1/chat/completions. The path after the domain tells the server which operation you want, and the request body contains the model name, your messages, and any options. When someone says “switch to a different endpoint,” they mean change that front-door address, usually because you are pointing at a different provider or gateway.
OpenRouter
OpenRouter is a gateway. Instead of holding one API key per provider and learning one API format per provider, you hold a single OpenRouter key and call a single endpoint. You pick any model by name, and OpenRouter routes your request to whichever provider actually hosts that model. It handles the billing on your behalf and returns the response in one uniform format.
The practical payoff: you can try dozens of models with one integration, compare them side by side, and switch models without touching your code. Because OpenRouter fronts many providers, it also shows live pricing, context sizes, and throughput for every model it carries, plus community ratings. The trade-off is a thin layer of indirection — a third party in the middle of your requests — which matters to people who care about latency, privacy, or data residency.
Major model families
A handful of families dominate the landscape, and each has a personality.
- Claude (Anthropic) — strong reasoning and long-form writing, famously capable at coding and agentic tool use. The current lineup runs from Haiku (fast, cheap) through Sonnet to Opus 5 (frontier reasoning and coding) and Fable 5 (the overall quality leader on independent leaderboards). Powers Cursor, Windsurf, and Claude Code.
- GPT (OpenAI) — the GPT-5.6 generation (Sol, Terra, Luna) covers autonomous agents, coding, and general use. Sol currently leads independent agent benchmarks and carries a 1.1M context window.
- Gemini (Google) — built for multimodal input from day one. Gemini 3.1 Pro offers 1M+ tokens, native Google Search integration, and competitive pricing; Gemini 3.5 Flash is the fast, cheap sibling.
- Grok (xAI) — Grok 4.6 is one of the strongest models at the frontier and significantly undercuts comparable flagships on output pricing, making it a serious cost-efficiency option at the top tier.
- DeepSeek (DeepSeek) — open-weight models that matched premium reasoning at dramatically lower cost. DeepSeek V4 Flash currently offers one of the best quality-to-cost ratios available and is a popular self-hostable option.
- Llama (Meta) — the open-weight flagship. Llama 4 Scout ships with a 10M token context window and is free to download and self-host. The default for organizations that need local deployment for privacy, compliance, or cost reasons.
- Kimi (Moonshot AI) — Kimi K3 leads among open-weight models on independent reasoning benchmarks. Self-hosting requires serious hardware (~594GB); the hosted API is more accessible.
- GLM (Z.AI) — GLM-5.3 competes with the frontier cluster at a significantly lower per-token price, making it worth evaluating for cost-sensitive agentic workloads.
- Qwen (Alibaba) — open-weight models spanning tiny to huge, strong in coding and math. Qwen3.8-Max is the current flagship; smaller variants run well on modest hardware.
- Mistral (Mistral AI) — open-weight European models with strong multilingual capability and efficient small sizes that run well locally.
For most people the practical choice is a small set: a frontier model when quality is critical, a mid-tier or fast budget model when it is not, and an open-weight model when you need to self-host.
For most people the practical choice is a small set: a frontier model when quality is critical, a mid-tier or fast budget model when it is not, and an open-weight model when you need to self-host.
Rough cost tiers
Pricing moves constantly and is quoted per million tokens, but the shape is stable. A usable model today ranges from a few cents to a few dollars per million input tokens, with output tokens costing more, often several times more.
- Very low — open-weight and small models, the DeepSeek family, and the smallest tiers of the big labs. Fractions of a dollar per million input tokens.
- Low — mid-size and older flagship models, plus the lite tiers of the majors. Well under a dollar per million input tokens.
- Mid — the current balanced flagships: Sonnet-class, GPT-4-class, and Gemini Pro-class. Single-digit dollars per million input tokens.
- High — the top reasoning and flagship tiers: Opus-class and the largest reasoning models. Tens of dollars per million input tokens, and more for output.
The real cost of a request depends on how many tokens you actually push through: a small, focused prompt on a cheap model costs a rounding error, while a million-token codebase on a premium model adds up quickly.
Six models at a glance
The table below is a snapshot — September 2026. Rankings, prices, and context sizes shift constantly; check live pricing pages before committing to any model.
| Model | Provider | Context | Relative cost | Typical strength |
|---|---|---|---|---|
| Claude Fable 5 | Anthropic | 200K | $$$$ | Overall quality leader (LMArena), coding, agentic work |
| GPT-5.6 Sol | OpenAI | 1.1M | $$$ | Autonomous agents, GPQA Diamond leader, long context |
| Gemini 3.1 Pro | 1M+ | $$ | Long-document, multimodal, native Search integration | |
| Grok 4.6 | xAI | 256K | $$ | Frontier quality at the lowest price in the top tier |
| DeepSeek V4 Flash | DeepSeek (open) | 128K | $ | Best quality-to-cost ratio, self-hostable |
| Llama 4 Scout | Meta (open) | 10M | Free (self-host) | Largest open-weight context window, privacy-first |
The pattern to take away: at the frontier the performance gap between models has narrowed dramatically — the decisive differences are now mostly price, context window, and which specific task you are running. Open-weight families give you control and zero per-token cost at the price of hardware; proprietary flagships charge a premium for the last few percentage points of quality on the hardest tasks. Because gateways and per-model pricing make switching cheap, the practical skill is knowing which axis your workload is sensitive to — and choosing accordingly.
Agentic AI: what changes when the model uses tools
So far this primer describes chat-style usage: you send a prompt, the model answers. Agentic AI is the next step. Instead of returning text only, the model can call tools, run code, search the web, and chain multiple steps together until a task is done. Under the hood this still works with the same tokens, keys, providers, and gateways. What changes is that the conversation becomes a loop: the model decides what to do next, executes it, reads the result, and continues. That makes agentic workflows more powerful, but also more sensitive to context-window size, tool reliability, and cost, because each extra step burns more tokens. The same primer applies; you are just spending more tokens per task.
Credits stretch further than you think
A useful sanity check on cost tiers comes from credit-deal math. A $10 OpenRouter credit bundle can stretch to roughly $20 of MiniMax M3 usage or about $50 of MiMo V2.5 Pro usage, depending on the exact bundle terms. Those multipliers do not mean the models are free — they mean the underlying per-token prices are low enough that promotional credit goes a long way. It also means two things at once: first, you can benchmark cheaply; second, you should still watch actual token volume, because a million-token request on any model still adds up.
If you are ready to move from understanding to building, the series on building your own agentic AI assistant picks up where this primer leaves off.
John Lock
Liked this? There's more where it came from.
Get our digest — articles worth your time, no spam, unsubscribe in one click.
Subscribe to Factnetize →
