History | View | Annotate | Download (993 Bytes)
json-lexer: make lexer error-recovery more deterministic
Currently when we reach an error state we effectively flush everythingfed to the lexer, which can put us in a state where we keep feedingtokens into the parser at arbitrary offsets in the stream. This makes it...
Add a lexer for JSON
Our JSON parser is a three stage parser. The first stage tokenizes the streaminto a set of lexical tokens. Since the lexical grammar is regular, we canuse a finite state machine to model it. The state machine will emit tokensas they are identified....