How to reconcile overlapping tokens. #440
-
I have have a stupid and naive question. I am building a VDM to Go transpiler and VDM uses a lot of keywords and especially keyword operators (and even a bunch of multi word operators). What is the recommended way, when using participle, to-handle the issue of overlapping tokens.
What is the recommended to make statements like that lex as something akin to : This issue seems to have been touched upon in some other issues but I have not seen a coherent solution. As I said, I may very well be misguided in this question, I am not super well educated and do not have much understanding into compilers. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
It's relatively straightforward and you can see an example in the SQL lexer here. Basically, just ensure your keywords come before your identifier in the lexer patterns. They are attempted in order, so if an identifier isn't a keyword it will match as an identifier. |
Beta Was this translation helpful? Give feedback.
It's relatively straightforward and you can see an example in the SQL lexer here. Basically, just ensure your keywords come before your identifier in the lexer patterns. They are attempted in order, so if an identifier isn't a keyword it will match as an identifier.