- Introduction
- The description
- 1. Overview
- 2. Definitions
- 2.1. Parsing Expression Grammars
- 3. Before tokenising
- 4. Tokenising
- 4.1. Fine-grained tokens
- 4.2. Token nonterminals
- 4.3. Whitespace and comment tokens
- 4.4. String and byte literal tokens
- 4.4.1. Escape processing
- 4.5. Numeric literal tokens
- 4.6. Identifier-like tokens
- 4.7. Punctuation tokens
- 5. Lowering doc-comments
- Appendices
- 6. Frontmatter grammar
- 7. Complete tokenisation grammar
- 8. Reimplementation CLI
- Discussion
- 9. Rationale for this model
- 10. Grammar for raw string literals
- 11. Open questions
- 12. Rustc oddities