Training, post-training, inference, retrieval, and agents—without the mathematics, with the failure modes each step creates.
From data to a deployed assistant
- Pre-training on large corpora. Developers train a neural network—today usually a transformer—on large collections of text and, for multimodal models, images or other media. The objective is next-token (or similar) prediction. The model is not given a verified database of facts; facts appear only insofar as they were consistent in the data.
- Post-training (including RLHF and related methods). Developers further adjust the model with supervised examples and preference feedback so it follows instructions, refuses certain requests, and matches a product’s tone. Reinforcement learning from human feedback (RLHF) is one common family of methods. Post-training shapes helpfulness and refusal behavior; it does not install a truth oracle.
- Inference. At use time the model receives a system prompt and user prompt, then samples tokens. Settings such as temperature change how varied outputs are. There is no separate reasoning engine that checks a ledger unless the product adds tools.
- Optional retrieval and tools (RAG, search, code execution). Many products first retrieve documents from the web or a private corpus, then condition generation on those passages (retrieval-augmented generation, RAG). Agents may call calendars, browsers, or internal APIs. Quality then depends on retrieval coverage and tool permissions as much as on the base model.
- Logging, evaluation, and monitoring. Enterprises and governments that deploy AI typically keep inventories, run evaluations (benchmarks, red teams, bias tests), and monitor incidents. NIST’s AI RMF frames these activities as Govern, Map, Measure, and Manage (as of the Jan 2023 AI RMF 1.0 and the Jul 2024 Generative AI Profile).[25,26]
Scale, tokens, and cost vocabulary
Model size is often discussed in parameters (billions of weights). Training cost is discussed in FLOPs or GPU-hours. Inference cost is usually metered in tokens: pieces of text the model reads and writes. A long PDF stuffed into the prompt consumes context window and budget even before the model answers. Multimodal inputs (images, audio) are likewise converted into tokens or embeddings. Officials comparing vendor quotes are usually comparing seat licenses, API token rates, minimum commitments, and whether retrieval or agent tool-calls are billed separately.
Post-training alignment methods (including RLHF and related preference optimization) change how models refuse or comply; security teams still test for jailbreaks because those controls are imperfect. When products add browsing or code-execution tools, evaluations must cover tool misuse, not only chat refusals.
Open-weight and closed deployment
Closed service. The vendor hosts the model. Buyers gain ease of use and frequent updates. Data-processing location, subprocessors, training-on-inputs terms, and export of logs become contract questions.
Open weights on own hardware or a chosen cloud. The organization can keep prompts inside a chosen boundary and inspect configurations. It also accepts patching, access control, evaluation, and incident response as its own duties. “Open weight” is not identical to “open source”; licenses vary.
Hybrid patterns. Some agencies use a closed model for general drafting and a local or virtual-private-cloud model for records that cannot leave a designated environment. Suite-embedded assistants inherit the identity and permission model of the office suite.
Cost drivers. Training frontier models is capital-intensive. Inference cost scales with tokens, context length, tool calls, and concurrency. Self-hosting shifts spend from API invoices to hardware, power, and staff time.
What each design choice implies for government
| Design choice | Typical implication for a public body |
|---|---|
| Base model only (no retrieval) | Answers reflect training patterns and cut-off; harder to cite primary sources. |
| RAG over approved corpora | Answers can point to policies you control; errors track corpus gaps and retrieval misses. |
| Web-grounded answer engine | Better for current events if citations are shown; still requires human verification before reliance. |
| Agent with write permissions | Can change systems of record; needs least-privilege access, logging, and kill switches. |
| Consumer free tier | Often weaker contractual limits on training-on-inputs and retention than enterprise/government SKUs. |