Invisible characters in code: what you see is not always what runs
Bidirectional control characters change the order in which text is displayed without changing the text itself. In source code that means a reviewer can read one program while the compiler runs another. This page shows the effect safely, and how to check your own code for it.
The mechanism
Unicode has to render text containing both left-to-right and right-to-left scripts. The rules for doing so are the Unicode bidirectional algorithm, specified in Unicode Standard Annex #9. Part of that specification is a set of invisible control characters that override the algorithm's automatic decisions:
| Character | Effect |
|---|---|
| U+202D | Left-to-right override: force everything after this point to render left to right |
| U+202E | Right-to-left override: force everything after this point to render right to left |
| U+2066 – U+2069 | Isolates: open and close a run whose direction is fixed independently of its surroundings |
| U+202A – U+202C | Embeddings and the pop that closes them, the older mechanism |
These characters are invisible. They occupy no width and produce no glyph. Their only effect is on the order in which the surrounding characters are painted.
A compiler does not apply the bidirectional algorithm. It reads the code points in logical order. Your editor, your terminal, your code review interface and your browser all do apply it. The gap between those two readings is the entire problem, and it was documented in detail by Nicholas Boucher and Ross Anderson of Cambridge University in 2021, in research they titled "Trojan Source".
A safe demonstration
The block below is ordinary text, it cannot run, and copying it is harmless. Your browser renders it with bidirectional processing active, exactly as most editors and review tools would:
Loading demo…Read that as a reviewer would. The comment appears to say the block is skipped for non-admins, and the condition appears to close before the call. Now load the same string into the checker below and open the x-ray view: the red chips mark U+202E and the isolate characters, and the cleaned output shows the code in the order a compiler actually reads it.
More: what was found, x-ray view and options
X-ray view
Your text with every hidden character exposed as a labelled chip. Hover a chip for its Unicode name.
What was found
| Code point | Character | Count | Action |
|---|
The three patterns worth recognising
Early return
An override makes a return statement appear to sit inside a comment or a disabled branch. The reviewer sees dead code; the compiler sees a live early return that skips a check.
Commenting out
The variant shown in the demo. A comment terminator appears to be in one place and is in another, so code that looks commented out is executed, or code that looks live is not.
Stretched string
A string literal appears to end earlier than it does. Text the reviewer reads as executable code is actually inside the string, or the reverse. This one is particularly effective in languages with rich string interpolation.
Why reviewers miss it. The problem targets the review step, not the runtime. It matters most for code that arrives from outside, a pull request, a dependency update, a pasted snippet, where the reviewer is reading unfamiliar code and trusting their eyes. Nothing about the code is technically invalid, which is exactly why eyes alone are not enough.
The related trick: lookalike letters
The same research documents a second pattern using characters that look identical to Latin letters but are not: the Cyrillic small letter е next to the Latin e, for instance. Two functions with names that render identically can coexist, and a call can be routed to the wrong one.
Lookalikes are a different problem from hidden characters, because the characters are visible, they just are not what they appear to be. The checker on this page does not rewrite them, since they are legitimate letters in their own scripts. Detecting them reliably means comparing scripts within an identifier, which is a linter's job rather than a text cleaner's.
How to protect your project
- Reject unterminated bidirectional controls in source files. Most current compilers warn on them; several treat them as errors. Turn the warning into an error in CI.
- Make them visible in review. Major code hosting platforms now flag bidirectional characters in diffs. Check that the flagging is on for your organisation.
- Configure editors to render invisible characters. The whole problem depends on the reviewer not seeing them.
- Scan at the commit boundary. A pre-commit hook that rejects bidirectional and tag characters outside string literals costs nothing and closes the class.
- Check pasted code before you commit it. Snippets from issue trackers, documentation sites and forums are the usual entry route. The checker above does this in one paste.
The hidden characters guide covers the wider class of invisible characters, most of which are disruptive rather than dangerous. Bidirectional controls are the ones that justify checking code as a habit.
Questions about invisible characters in code
Which programming languages are affected?
Essentially all of them. The behaviour comes from Unicode rendering rather than any language feature, so it applies wherever a compiler reads code points in logical order and a human reads them after bidirectional reordering, which is every mainstream language.
Is this still a real risk today?
The class is well known and tooling has improved: compilers warn, and code hosting platforms flag bidirectional characters in diffs. What has not changed is the underlying behaviour of Unicode rendering, so the risk moves to places where those defences are not applied, pasted snippets, generated files, and text pipelines outside code review.
Can I just delete every bidirectional character from my codebase?
In source code, yes: they have no legitimate purpose in identifiers or syntax. In string literals containing right-to-left text they occasionally do real work, so removing them blindly can change how a user-facing string displays. Reviewing each occurrence is quick, because there are usually very few.
Does this affect anything other than source code?
Yes. Any context where display order carries meaning is affected: filenames, where a right-to-left override can make an executable appear to be a document, chat messages, log files and configuration.
Where can I read the original research?
The paper is "Trojan Source: Invisible Vulnerabilities" by Nicholas Boucher and Ross Anderson, published in 2021 and presented at USENIX Security. The behaviour it describes is specified in Unicode Standard Annex #9, the Unicode bidirectional algorithm.
Related pages
- Hidden characters in text - the full class, its origins and its failure modes.
- The invisible Unicode list - every bidirectional control, with what the cleaner does to it.
- Copilot output and code suggestions - where pasted code most often enters a project.