DEV Community

Discussion on: What is a Lexer, Anyway?

cad97 profile image
Christopher Durham Author

I'm glad it could help! Out of curiosity, what kind of improvements have you spotted?

adam_cyclones profile image
Adam Crockett • Edited on

Specifically my tokens are defined in structs Token and stored in a Btree Map where the key is the token and the value is Token struct which I had planned to provide callbacks to handle what is expected after the next token. But I'm struggling to implement (new to rust) I wanted to do something like Marpa parser.

I like how your tokens are just a match statement.