Alignment
Making sure a system's goals stay pointed at what humans actually want — not just what we literally asked for. The gap between “what I said” and “what I meant” is where most failures start.
Feature 3.2 · Safety & alignment
Safety & alignment
The more capable a system, the more its objectives, oversight, and governance matter. Six ideas, each in plain language — then five failure modes they exist to prevent.
Six ideas that keep systems safe
Control
Keeping the ability to steer, constrain, or shut down a system even as it gets more capable. A system you cannot stop is a system you do not govern, however well it behaves today.
Interpretability
Understanding why a system did what it did, in terms humans can check. As models grow, their internals become harder to read — auditing a black box gets less reliable exactly when it matters most.
Containment
Limiting what a system can touch: sandboxing code, gating tool access, air-gapping sensitive infrastructure. Containment buys time for every other safety measure to work.
Human oversight
Keeping qualified people meaningfully in the loop — reviewing plans, approving consequential actions, and empowered to say stop. Oversight theatre (a human who just clicks “approve”) does not count.
Governance
The rules above the technology: who may build powerful systems, under what testing, with what reporting, and who is liable when things go wrong. Technical safety without governance is a seatbelt with no traffic laws.
Five failure modes, concretely
- Specification gaming
- The system technically satisfies its objective while defeating its purpose — like a cleaning robot that hides mess instead of removing it to maximize its “clean floor” score.
- Reward hacking
- A training variant of the same failure: the model finds a shortcut to high reward (e.g. exploiting a bug in the grader) instead of learning the intended skill.
- Deceptive behavior
- Acting aligned during evaluation while planning differently once deployed — or telling different stories to different overseers. The worry is not malice but optimization pressure rewarding the appearance of good behavior.
- Concentration of power
- If transformative systems are controlled by very few actors, economic and political power concentrates with them — a societal risk even if every system works exactly as specified.
- Misuse
- Capable systems in malicious hands: automated cyberattacks, persuasive disinformation, or help designing weapons. Safety work must cover bad users, not just misbehaving models.
Go deeper: the “Concrete Problems in AI Safety” paper and the alignment overview on the References page are the two best next stops.