Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

You can, but see, the actual contents of the strings are really only used at the code-gen stage, where it's put verbatim in the .rodata section. Arguably, it could have been discarded by the dead-code elimination pass by that time but that's unlikely: most of the string constants do actually see use in the program.

Pushing that part of the lexer's task so late on the off-chance it probably won't be needed IMHO only makes sense if you're writing a parser for an IDE/LSP in which case you don't need to unescape the string at all.



If the string contents are only used at the code-gen stage, then I see that as an argument FOR being lazy.

On the other hand, the principle of validating your input up front suggests that you should at least recognize the escapes, if not store their decoded from.

That is, you can be lazy by only parsing \\ and \" to find the closing quote, and producing 1 token. But that's nearly the same thing as doing a full-decoding. The downsides are that it allows invalid input deeper into the compiler, and it doesn't alert the user to errors as quickly as possible.

most of the string constants do actually see use in the program

[citation needed] :) I don't see how that can possibly hold across languages (Go, Java, Swift, etc.), and I'm not even sure that's true for C/C++.

only makes sense if you're writing a parser for an IDE/LSP in which case you don't need to unescape the string at all.

Clang's front end is used both for LSP and for code gen, so that distinction doesn't apply in all cases. It seems like most projects want to reuse their front ends these days.

And again the LSP will want to warn about invalid escapes like \777 and \u{9999999}.

----

EDIT: Although I guess in C, emitting all the errors up front, and then storing the decoded form in place, is probably about the same cost. If there's no allocation, then I might treat eager decoding as "free".

I think the problem is that you lose info, and most languages have more structure in their string literals, e.g. Swift has \(var) interpolation, and IDEs in Rust want to do things with {} format! strings, etc.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: