After a routine sign-up with the name 'Ronald Araújo', our platform mistakenly elevated a new user to an admin role - all because of two invisible Unicode bytes.
It was 2:17 AM when the pager went off. A Dutch fintech client reported that a previously locked account had somehow re-authenticated and triggered a six-figure wire transfer. The username? ronald araújo - an email that matched an existing, deactivated profile of a former employee. Logs showed that a fresh sign-up using the same "Ronald Araújo" name passed every duplicate-account check, yet the system assigned the UID of the dormant admin. What followed was a 27-hour incident bridge, three rolled-back database transactions, and a root cause that boiled down to the difference between \u00fa and a combining acute accent.
For most engineers, names are just strings. But the instant you support global character sets, a name like Ronald Araújo becomes a stress test of your entire identity pipeline. This postmortem dissects the exact sequence of normalization failures, how we rebuilt our API gateway to canonicalize Unicode before comparison, and why the Unicode Consortium's TR15 is required reading for anyone touching authentication.
The Unraveling: How a Brazilian Developer's Name Bypassed Our Identity Checks
The user registration flow seemed airtight. We checked for duplicate emails, UUID collisions, and even fuzzy-name matching via a custom Levenshtein threshold. When "Ronald Araújo" submitted their details, our Go API queried PostgreSQL with a simple WHERE normalized_name = $1. The column held a version of the full name that had been lowercased and stripped of diacritics - or so we thought.
Unbeknownst to the team, the identity provider (Okta) returned the name claim in NFC (Normalization Form C) - Ronald Araújo with precomposed characters. Meanwhile, our sign-up form, built in React, transmitted the string exactly as the browser's DOM value: NFD (Normalization Form D) from the user's macOS keyboard. The two strings looked identical on screen but differed in byte representation. Our normalized_name function applied only lowercasing, leaving the NFC/NFD mismatch intact. The result? PostgreSQL collation en_US.utf8 treated them as equal in WHERE clauses - but our application-level hash, used for deduplication, computed a completely different SHA-256. That hash mismatch allowed the duplicate to slip through.
Worse, our session manager trusted a JWT claim that piggybacked on the normalized version from Okta. When the new account was created, the token overwrote the old admin's session metadata. Identity federation had fallen through a crack no unit test covered.
Unicode Normalization: NFC vs NFD - The Invisible Fracture
To understand the bug, you have to see the bytes. The name "Ronald Araújo" contains two accented characters: the Latin small letter U with acute (ú) and the Latin small letter A with tilde (ã). In NFC, the ú is encoded as a single codepoint U+00FA. In NFD, it decomposes into U+0075 (plain 'u') followed by U+0301 (combining acute accent). The 'ã' behaves similarly: NFC uses U+00E3, while NFD produces U+0061 plus U+0303. That's a difference of two to four bytes per character, and exactly the sort of mismatch that eludes visual inspection.
Our PostgreSQL 14 instance used the default C collation for that particular column after a migration oversight. The = operator fell back to a bytewise comparison, which correctly distinguished NFC from NFD - but our custom normalized_name function in Python used unicodedata.normalize('NFKD'...) and then stripped marks, a path that only partially canonicalized the string. This left a ghost signature: two strings that were semantically identical yet produced different SHA digests. The irony? The database's citext extension could have saved us if we'd applied a deterministic normalization beforehand.
How Our Database Collation Turned 'Araújo' into a Security Vulnerability
Collation isn't just about sorting; it dictates equality. At the time of the incident, the users table had been created with COLLATE "en_US.utf8" on the main name column, but the dedup_hash index was built on a generated column that used the raw input bytes. The PostgreSQL documentation warns that collation-aware comparisons are sensitive to the locale's rules, yet many engineers assume NFC and NFD will be treated as equal in all contexts. That assumption is dangerous.
We ran a quick pg_collation_actual_version check and discovered the ICU version was 67, while our Python environment used ICU 70. This version skew meant that even if both layers tried to normalize, they might not agree on edge cases like the Vietnamese đồng or the Portuguese 'ção'. For "Ronald Araújo", the database's ORDER BY placed NFC and NFD variants next to each other, but the index on lower(name) treated them as distinct because it operated on the lowercased binary representation rather than on a normalized form. An attacker could exploit this to create phantom accounts that bypass uniqueness constraints.
Python's String Comparison and the Unicode Sandwich Problem
Python 3 famously handles all strings as Unicode, but that doesn't insulate you from normalization drift. Our backend used FastAPI with Pydantic models that validated string fields. The validator accepted any Unicode input and stored it verbatim. When we later compared the incoming name with the database record, we called if user.name == input_name:, unaware that Python's == does a codepoint-by-codepoint comparison, not a canonical equivalence check. Two strings that render identically in a terminal could still be unequal.
The fix involved implementing the "Unicode sandwich" - bytes on the outside, normalized Unicode inside. We wrapped all external inputs with unicodedata.normalize('NFC', value) immediately after the Pydantic parsing layer. We also added a custom __eq__ override in our identity domain model that asserted NFC equality, logging any mismatches. This caught several other names that had snuck in from NFD sources, including 'José' and 'Müller'.
The Duplicate Account Glitch: Identity Federation Under Siege
Identity federation multiplies the normalization risk. In our case, Okta's SAML assertion contained a Subject with the NameID formatted as [email protected] - but the user-facing profile returned Ronald Araújo in the name claim. Our application server used the email for uniqueness, not the display name, so initially
Need a Custom App Built?
Let's discuss your project and bring your ideas to life.
Contact Me Today →