# Agentic RL

## Start here

- [Agentic RL: From Policy Gradients to Coding Agents](https://agent-rl.tacetai.com/index.md)

## Front matter

- [0. Notation and How to Use This Book](https://agent-rl.tacetai.com/tutorial/chapters/00-notation.md)

## Part I · Foundations

- [1. What Is Agentic RL?](https://agent-rl.tacetai.com/tutorial/chapters/01-introduction.md)
- [2. Reinforcement Learning Fundamentals](https://agent-rl.tacetai.com/tutorial/chapters/02-rl-fundamentals.md)
- [3. Language Models as Policies](https://agent-rl.tacetai.com/tutorial/chapters/03-language-models-as-policies.md)
- [4. Environments and Tools: the BugFix Gym](https://agent-rl.tacetai.com/tutorial/chapters/04-environments-and-tools.md)
- [5. Trajectories, Templates, and Masks](https://agent-rl.tacetai.com/tutorial/chapters/05-trajectories-and-templates.md)

## Part II · Policy optimization

- [6. Policy Gradients](https://agent-rl.tacetai.com/tutorial/chapters/06-policy-gradient.md)
- [7. Advantages and Credit Assignment](https://agent-rl.tacetai.com/tutorial/chapters/07-advantages-and-credit.md)
- [8. PPO: Reusing Samples Safely](https://agent-rl.tacetai.com/tutorial/chapters/08-ppo.md)
- [9. GRPO and the Critic-Free Family](https://agent-rl.tacetai.com/tutorial/chapters/09-grpo-and-variants.md)
- [10. KL, Entropy, and Regularization](https://agent-rl.tacetai.com/tutorial/chapters/10-kl-entropy-regularization.md)

## Part III · Rewards and agents

- [11. Rewards, Verifiers, and Reward Hacking](https://agent-rl.tacetai.com/tutorial/chapters/11-rewards-and-verifiers.md)
- [12. Multi-Turn Agents and Long Horizons](https://agent-rl.tacetai.com/tutorial/chapters/12-multi-turn-and-long-horizons.md)

## Part IV · Systems

- [13. Rollout Systems and Off-Policy Correction](https://agent-rl.tacetai.com/tutorial/chapters/13-rollout-systems.md)
- [14. Scaling and Efficiency](https://agent-rl.tacetai.com/tutorial/chapters/14-scaling-and-efficiency.md)

## Part V · Beyond pure RL

- [15. SFT, Rejection Sampling, and Preferences](https://agent-rl.tacetai.com/tutorial/chapters/15-sft-and-rejection-sampling.md)
- [16. Distillation for Agents](https://agent-rl.tacetai.com/tutorial/chapters/16-distillation.md)

## Part VI · Practice

- [17. Evaluation](https://agent-rl.tacetai.com/tutorial/chapters/17-evaluation.md)
- [18. Debugging RL Runs](https://agent-rl.tacetai.com/tutorial/chapters/18-debugging-rl-runs.md)
- [19. Capstone: Train a Coding Agent End to End](https://agent-rl.tacetai.com/tutorial/chapters/19-capstone.md)
- [20. Research Frontier and Open Problems](https://agent-rl.tacetai.com/tutorial/chapters/20-frontier.md)

## Appendices

- [A. Mathematical Tools](https://agent-rl.tacetai.com/tutorial/chapters/A-math.md)
- [B. Design Review Problems](https://agent-rl.tacetai.com/tutorial/chapters/B-design-review.md)

## Reference

- [The agentrl library](https://agent-rl.tacetai.com/tutorial/library.md)
- [Real-model labs: GRPO on BugFix tasks with Qwen3-0.6B](https://agent-rl.tacetai.com/tutorial/labs/README.md)
- [Sources and implementation ledger](https://agent-rl.tacetai.com/tutorial/sources.md)
- [Editorial guide](https://agent-rl.tacetai.com/tutorial/editorial-guide.md)

> The links below point to documentation indexes. Follow each `/_llms/` index recursively until you reach documentation pages.

## Indexes

- [Chinese (28 pages)](https://agent-rl.tacetai.com/_llms/cn.txt): Documentation for Chinese.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.