1. Introduction
3. The description
4. 1. Overview
5. 2. Definitions
6. 3. Before tokenising
7. 4. Tokenising
8. 1. 4.1. Pretokenising
   2. 1. 4.1.1. Output: pretokens
      2. 4.1.2. Patterns
      3. 4.1.3. List of pretokenisation rules
   3. 4.2. Reprocessing
   4. 1. 4.2.1. Output: fine-grained tokens
      2. 4.2.2. Escape processing
      3. 4.2.3. List of reprocessing cases
10. Discussion
11. 5. Rationale for this model
12. 6. Open questions
13. 7. Rustc oddities