Short answer: Unicode often lets you spell the same character two ways — é can be one code point or e plus a combining accent. They look identical and compare as different. Normalising both sides to NFC before comparing fixes it. Use NFC for storing and displaying text; use NFKC only for search keys, because it changes the text.
Need to see what’s actually in a string? Paste it into the Symbol Identifier — it lists every code point, which is the only way to tell these apart by eye.
The same character, two spellings
There are two ways to write é:
| Spelling | Code points | Length |
|---|---|---|
| Composed | U+00E9 | 1 character |
| Decomposed | U+0065 U+0301 | 2 characters |
Both render as é. Both are correct. And in almost every programming language, "é" == "é" returns false when one is composed and the other decomposed, because string comparison works on code points, not on what the pixels look like.
Unicode calls these two spellings canonically equivalent — genuinely the same text, written differently. Normalisation is the process of converting to one agreed spelling so comparison works. Convert both to NFC, and the comparison returns true.
This is the root cause of an enormous number of “but it’s obviously the same!” bugs: a search that misses, a duplicate row in a database, a file that can’t be found, a password that won’t authenticate. Background on the encoding model is in what Unicode is.
The four forms
| Form | What it does | Safe for |
|---|---|---|
| NFC | Composes — combines base + accent into a single character where one exists | Storage, display, the web |
| NFD | Decomposes — splits characters into base + combining marks | Sorting, accent-stripping |
| NFKC | Composes and replaces compatibility characters | Search keys, identifiers |
| NFKD | Decomposes and replaces compatibility characters | Search keys, matching |
The important split is the K. NFC and NFD are canonical: they change the spelling but never the meaning, and you can convert back and forth without losing anything. NFKC and NFKD are compatibility: they deliberately rewrite characters into plainer equivalents, and that’s a one-way trip.
What NFC and NFD do
Canonical normalisation only reshuffles things that were always the same character:
| Input | NFC | NFD |
|---|---|---|
| é | U+00E9 | U+0065 U+0301 |
| Å (U+00C5, the letter) | U+00C5 | U+0041 U+030A |
| Å (U+212B, the ångström sign) | U+00C5 | U+0041 U+030A |
That third row is worth a pause. The ångström sign U+212B is a separate character that exists for compatibility with older standards, and NFC folds it into the ordinary letter U+00C5. Two different code points, same normalised result — because Unicode considers them the same character.
What NFKC does — and why it’s dangerous
Compatibility normalisation rewrites your text. Every one of these was derived, not guessed:
| Character | Name | NFKC result |
|---|---|---|
| fi | Latin small ligature fi | fi — two letters |
| µ | Micro sign | μ — Greek small letter mu |
| Ω | Ohm sign | Ω — Greek capital omega |
| ① | Circled digit one | 1 |
| ½ | Vulgar fraction one half | 1⁄2 — three characters |
| ⁵ | Superscript five | 5 |
| ＊ | Fullwidth asterisk | * |
Look at the micro sign. µ U+00B5 becomes μ U+03BC under NFKC — the exact pair that causes trouble in statistics symbols and on the mu symbol page. Normalisation is why the two are so often silently interchanged: any system running NFKC on its input has already converted one into the other.
And notice what happens to ½ and ⁵: superscripts and fractions lose their formatting entirely. Run NFKC over a recipe and half your quantities change meaning.
Use NFKC to build a search key, never to store the user’s text. Keep the original; match against the normalised copy.
One specific trap
If you use fullwidth lookalikes to get around blocked characters in filenames — the trick in invalid filename characters — NFKC turns them straight back into the characters you were avoiding. ＊ becomes *, ？ becomes ?, ｜ becomes |. A sync client or upload handler that normalises with NFKC will reintroduce exactly the problem you worked around.
Where this actually bites
Filenames on a Mac. The old HFS+ filesystem stored every filename in a decomposed, NFD-like form — so a file you named Café was written to disk as Cafe + combining accent. APFS behaves differently: it preserves whatever normalisation you give it, and since High Sierra it’s normalisation-insensitive, using hash-based matching so a file can be found under any spelling and two variants can’t coexist in one folder.
That solves it at the filesystem level and not above it. Anything that compares filenames byte for byte — git, zip archives, rsync, build tools — still sees two different strings. This class of bug is still being found: in 2026 a Python packaging flaw let exclusion rules silently fail, because the patterns were written NFC while the on-disk names were NFD, so files meant to be excluded were published anyway.
Cross-platform archives. Zip a folder of accented filenames on a Mac, unzip on Windows, and the names may arrive decomposed — visually identical, sorted differently, and no longer matching any script that looks for them.
Databases. Two rows that look identical, both “José”, one composed and one decomposed. A unique index won’t stop it, because they genuinely are different byte sequences. PostgreSQL has normalize() for exactly this.
Search. A user types the accent with a dead key (composed) and your content was pasted from a Mac (decomposed). Zero results, no error, no clue.
Usernames and security. If you normalise with NFKC at signup but not at login, or vice versa, authentication silently fails. Worse, characters that become identical after normalisation can be used to impersonate an existing account, which is why identifier normalisation must happen at exactly one point and be applied everywhere.
Which form should you use?
- Storing and displaying text: NFC. It’s what the web standards recommend for content, it’s what most systems already produce, and it keeps text shortest.
- Comparing user input: NFC on both sides, every time, before the comparison.
- Search indexes and identifiers: NFKC, as a derived key alongside the original — never as a replacement for it.
- Stripping accents: NFD first, then drop the combining marks (
U+0300–U+036F). This is the correct way to turnJoséintoJose, and it only works because decomposition separates the mark from the letter. - Never store NFKD. You’ll have destroyed formatting you can’t recover.
In code
python
import unicodedata as u
u.normalize("NFC", text) == u.normalize("NFC", other) # correct comparisonjavascript
"é".normalize("NFC") === "é".normalize("NFC") // true
"Café".normalize("NFD").replace(/[\u0300-\u036f]/g, "") // "Cafe"In PHP it’s Normalizer::normalize() from the intl extension; in PostgreSQL, normalize(text, NFC) and the IS NFC NORMALIZED predicate; in Java, java.text.Normalizer.
Normalise at the boundary. Do it once, as data enters your system, rather than sprinkling calls through the code — and if you store a normalised copy for matching, keep the user’s original text intact for display.
Three things normalisation does not fix
This is where people over-trust it.
It doesn’t invent characters that don’t exist. x̄ — the statistics sample mean — is x plus a combining macron, and NFC leaves it as two code points because there is no precomposed “x with macron” in Unicode. ȳ does have one (U+0233), so it composes to a single character. Same-looking pair, different behaviour, and NFC is why.
It doesn’t fix case-mapping. Lowercasing Turkish İ produces i plus a combining dot above — two code points — and normalising the result doesn’t collapse them back. Neither does case folding. That’s a casing problem, not a normalisation one.
It doesn’t make lookalikes equal. Characters that merely resemble each other are unaffected: Greek ο and Latin o, the tilde operator vs the ASCII tilde, a hyphen and an en dash. Normalisation only merges characters Unicode has declared equivalent. Everything else stays different, forever — which is the whole basis of homograph attacks.
Related traps that look like normalisation problems but aren’t: invisible characters and characters that won’t render.
Accessibility
Screen readers announce decomposed text character by character, so Café stored decomposed can be read as “Cafe” then “combining acute accent” instead of as a word. Composed text reads naturally.
This is one more reason NFC is the right form for anything a person will read, alongside setting a correct lang attribute so the reader applies the right pronunciation rules. How screen readers pronounce symbols covers the pattern.
Troubleshooting
| Problem | Fix |
|---|---|
| Two identical-looking strings won’t match | Normalise both to NFC before comparing. |
| Duplicate rows that look the same | One composed, one decomposed. Normalise on the way in and de-duplicate. |
| Search misses accented words | Index a normalised copy; normalise the query the same way. |
| Filenames break after zipping on a Mac | NFD names from HFS+ lineage. Normalise names before comparing them. |
| Accented character counts as two | It’s decomposed. NFC composes it — unless no precomposed form exists. |
| NFKC changed my superscripts and fractions | Expected. NFKC is lossy. Use it for keys only, never for stored text. |
| Fullwidth filename substitutes turned back into blocked characters | Something in the pipeline ran NFKC. Use a different substitution strategy. |
| Normalising didn’t fix my Turkish casing bug | Different problem. Use an invariant locale for the comparison. |
Recap
| Situation | Form |
|---|---|
| Storing and displaying text | NFC |
| Comparing two strings | NFC on both sides |
| Building a search key | NFKC, kept alongside the original |
| Stripping accents | NFD, then remove U+0300–U+036F |
| Storing user text long-term | Never NFKD |
| Filenames across platforms | Normalise before comparing, not after |
Frequently asked questions
What is Unicode normalization?
Converting text to one agreed spelling so that characters which can be written more than one way compare as equal. The accented é, for example, can be a single code point or a letter plus a combining accent — normalising makes both the same.
What’s the difference between NFC and NFD?
NFC composes characters, combining a base letter and its accent into one code point where one exists. NFD decomposes them into separate pieces. Both preserve meaning exactly; NFC is shorter and is what you should store.
When should I use NFKC?
For search keys and identifier matching, and only as a derived copy. NFKC rewrites compatibility characters — ligatures become separate letters, superscripts lose their formatting, the micro sign becomes Greek mu — so it must never replace the text you store.
Why do accented filenames break between Mac and Windows?
The older Mac filesystem stored names in a decomposed form. Anything that compares filenames byte for byte, such as git or a zip archive, sees a decomposed name as different from a composed one even though they display identically.
Does normalization fix lookalike characters?
No. It only merges characters Unicode declares equivalent. A Greek letter that looks like a Latin one stays different after any normalisation, which is exactly what makes homograph attacks possible.
How do I strip accents from text?
Normalise to NFD, then remove the combining marks in the range U+0300 to U+036F. Decomposition is what separates the accent from the letter, so it has to happen first.
Written by Igor R. Tested on Windows 11, macOS and Linux, in Python, JavaScript and PostgreSQL. Last updated September 2026. Symbol code points follow the Unicode Standard.