Writeup of Rust's lexer

Common definitions

Some grammar definitions which are needed on the following pages appear below.

Identifier-like forms

Grammar

IDENT = { IDENT_START ~ XID_CONTINUE * }
IDENT_START = { XID_START | "_" }
SUFFIX = { IDENT }