should we disallow all Unicode control characters in character literals?

[zygoloid]

Summary of issue:

Per #1964, we disallow the control characters U+0000 - U+001F from appearing directly as the contents of a character literal, and require an escape sequence to be used instead. However, we don't apply the same restriction to the other Unicode control characters U+007F - U+009F. Should we?

Details:

I think the reason we chose to reject these characters was to ensure that character literals are readable; we might want to exclude other kinds of non-printable or invisible characters here too if that is the rationale. Another question to consider is whether we want consistent restrictions between character and string literals. There might be an argument for not using the same restrictions for things like LTR markers, where a matched set in a string literal might be reasonable but a lone marker in a character literal might not.

Any other information that you want to share?

#5960 makes the toolchain reject both kinds of control character.

[josh11b]

What do we do about the problematic characters mentioned in: https://www.tbray.org/ongoing/When/202x/2025/08/14/RFC9839 https://www.rfc-editor.org/rfc/rfc9839.html ?

[zygoloid]

What do we do about the problematic characters mentioned in: https://www.tbray.org/ongoing/When/202x/2025/08/14/RFC9839 https://www.rfc-editor.org/rfc/rfc9839.html ?

2.2.1 Surrogates: Our input is UTF-8, in which surrogate encodings are not valid. We reject these already.
2.2.2 Control Codes: These are the subject of this issue. (Toolchain rejects, design only rejects half of them.)
2.2.3 Noncharacters: Looks like we should reject these too: U+aFFFE and U+aFFFF for any a, plus U+FDD0 - U+FDEF.

Therefore I suggest we change our list of "disallowed in character literals" characters to include at least the following ranges:

U+0000 - U+001F
U+007F - U+009F
U+FDD0 - U+FDEF
U+aaFFFE - U+aaFFFF for 0 <= aa <= 10